Skip to content

Beautiful, AI-native markdown IDE and LLM wiki

Treat a server.lock whose pid was reused as stale (PRD-9354) (#6374)

Andrew Mikofalvy9e3f2f13f7c3
diff --git a/.changeset/stale-server-lock-pid-reuse.md b/.changeset/stale-server-lock-pid-reuse.md
new file mode 100644
index 000000000..3c9d4b375
--- /dev/null
+++ b/.changeset/stale-server-lock-pid-reuse.md
@@ -0,0 +1,5 @@
+---
+"@inkeep/open-knowledge": minor
+---
+
+`ok start` and `ok clean` now treat a server lock as stale once another program reuses its process ID, and `ok stop` and the desktop app no longer signal that program. `ok stop` exits `1` if it cannot confirm the process; stop 0.25.0-or-earlier servers yourself, then restart them.
diff --git a/AGENTS.md b/AGENTS.md
index 89bc8b0a0..7902b7d28 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -51,7 +51,7 @@ pnpm run dev
## Process safety
-Never signal a pid you did not spawn. Signal a held `ChildProcess` before it exits (its group only then); never a pid from a file, `ps` or stdout, a `?? <n>` fallback or literal `0`/`1`/`-1`. Two acts in the maintainers' gate tooling, which this repository does not carry, go further, and nothing else does: the lifeline watcher the guard of each gate step forks into the step's process group may run `kill -s KILL 0`, which ends the group it belongs to; and after the step's first process exits, its supervisor may still signal the step's group while that lifeline, a live member, holds the group. Gate: `ok/no-sentinel-signal-target`.
+Never signal a pid you did not spawn. Signal a held `ChildProcess` before it exits (its group only then); never a pid from a file, `ps` or stdout, a `?? <n>` fallback or literal `0`/`1`/`-1`. The one pid from a file you may signal is a `server.lock` pid at the two places that stop a server through its lock, `runStop` in the CLI and `terminateServerByPid` in the desktop app, and only after `assessLockHolder` confirms, immediately before each signal, that the process holding that pid is the one that wrote the lock. Two acts in the maintainers' gate tooling, which this repository does not carry, go further, and nothing else does: the lifeline watcher the guard of each gate step forks into the step's process group may run `kill -s KILL 0`, which ends the group it belongs to; and after the step's first process exits, its supervisor may still signal the step's group while that lifeline, a live member, holds the group. Gate: `ok/no-sentinel-signal-target`.
## Comment policy
diff --git a/docs/content/reference/cli.mdx b/docs/content/reference/cli.mdx
index 831e9f737..c88525654 100644
--- a/docs/content/reference/cli.mdx
+++ b/docs/content/reference/cli.mdx
@@ -72,7 +72,7 @@ Restart all existing OpenKnowledge servers after updating so every runtime uses
If startup reports a registry version mismatch, update all OpenKnowledge runtimes to compatible versions. For a permissions error, make the OS account's home writable; protection is never silently disabled. The shared registry must be on a local filesystem that honors SQLite file locks. Network-mounted homes and filesystems with broken locks do not provide this protection.
-After a server crashes, is forcibly terminated or the computer restarts, its released filesystem lease lets the next server recover ownership automatically on macOS, Windows and Linux. Recovery never depends on a PID, hostname or timeout. A missing or damaged lease leaves only its recorded content protected; disjoint projects can still open. A corrupt registry or a retired, nonempty pre-release format affects the whole registry and needs manual recovery. Run `ok stop all` and confirm **every OpenKnowledge server under that OS account** has exited, including desktop, terminal, MCP-spawned and preview sessions. Then back up and remove the registry file named in the error (`~/.ok/local/server-authority.sqlite`) together with its `-journal` sidecar if present, keeping them together as one backup, and restart the desired projects. Never remove or replace the registry while a server is running: doing so would discard its ownership claim.
+After a server crashes, is forcibly terminated or the computer restarts, its released filesystem lease lets the next server recover ownership automatically on macOS, Windows and Linux. Recovery never depends on a PID, hostname or timeout. A missing or damaged lease leaves only its recorded content protected; disjoint projects can still open. A corrupt registry or a retired, nonempty pre-release format affects the whole registry and needs manual recovery. Run `ok stop all`, or on a machine shared with accounts you do not trust, list servers with `ok ps` and run `ok stop` in each listed project of yours. Either way, confirm **every OpenKnowledge server under that OS account** has exited, including desktop, terminal, MCP-spawned and preview sessions. Then back up and remove the registry file named in the error (`~/.ok/local/server-authority.sqlite`) together with its `-journal` sidecar if present, keeping them together as one backup, and restart the desired projects. Never remove or replace the registry while a server is running: doing so would discard its ownership claim.
## Open a doc, folder, or file
@@ -144,10 +144,27 @@ Each project runs its own local server. These commands let you see and stop them
| --- | --- |
| `ok status` | Show whether a server and UI are running for the current project |
| `ok ps` | List every running OpenKnowledge server on your machine |
-| `ok stop` | Stop the server for the current directory — or pass a port, path, or PID to stop a specific server, or `all` to stop every server. Declines when the server's lock records a different OpenKnowledge app (Stable, Beta, or a `dev` build run from source) than [the CLI's own](/docs/reference/what-open-knowledge-writes#beta-uses-its-own-entry), naming that app, and when something is still connected to the target, naming how many clients are attached. Either way it exits `1`; pass `--force` to stop it anyway. A lock with no recorded app is stopped as before, and a server it cannot reach is stopped without the connected-clients check |
+| `ok stop` | Stop the server for the current directory — or pass a port, path, or PID to stop a specific server, or `all` to stop every server. Declines when the server's lock records a different OpenKnowledge app (Stable, Beta, or a `dev` build run from source) than [the CLI's own](/docs/reference/what-open-knowledge-writes#beta-uses-its-own-entry), naming that app, and when something is still connected to the target, naming how many clients are attached. Either way it exits `1`; pass `--force` to stop it anyway. `--force` does not override the process identity check described below. A lock with no recorded app is stopped as before, and a server it cannot reach is stopped without the connected-clients check |
| `ok clean` | Remove a stale lock file left behind by a crash (never touches live servers) |
-`ok clean` retains a lock and exits `1` if it cannot read it, it records another machine, or it has no recorded owner and its PID is still running locally. A PID that has exited is not running even while it lingers as a defunct entry in the process table, so `ok clean` prunes the lock naming it. The error names the lock and explains what to check. For an unverified owner, inspect the named local PID and stop it if it is an OpenKnowledge server; do not delete the lock while that process is running.
+`ok clean` retains a lock and exits `1` if it cannot read it, it records a machine ID or host name that is not this machine's, it has no recorded owner and its PID is still running locally, or it changed or was removed between `ok clean` checking it and removing it. A PID that has exited is not running even while it lingers as a defunct entry in the process table, so `ok clean` prunes the lock naming it. The error names the lock and explains what to check. For a lock that records another machine ID or host name, run `ok stop` from the account, home folder, or machine that started the server while it is running, or `ok clean` there once it has exited. For an unverified owner, inspect the named local PID and stop it if it is an OpenKnowledge server; do not delete the lock while that process is running.
+
+A lock from this machine whose PID now belongs to a process that started after the lock was written, as after a restart, is stale too: `ok clean` prunes it, `ok start` replaces it, and `ok stop` leaves that process unsignalled and exits `0`, as it does for a lock whose process has exited. `ok status --json` reports such a lock as `state: "reused-pid"` with `alive: false`. `ok ps --json` reports it as `server.status: "reused-pid"` and `displayStatus: "stale"`, with no command or usage; a lock whose process has exited shows the same `displayStatus` with `server.status: "dead-pid"`.
+
+`ok stop` signals a server only after confirming that its PID still belongs to the process that wrote the lock, by comparing when that process started with the start the lock records. This check detects a reused PID; it does not establish who wrote the lock. `ok stop` with no argument or with a directory reads that project's lock, so keep the project's `.ok/local` folder writable only by accounts you trust. `ok stop all`, `ok stop` with a port or PID, and `ok uninstall` act on locks found through running processes on this machine, including other accounts' processes, or under the directory you run them from; `ok ps` lists the same set. On a machine shared with accounts you do not trust, list servers with `ok ps` and run `ok stop` in each listed project of yours instead. `ok uninstall` has no per-project form, so on such a machine run it without `--yes` and check the servers its plan lists before you confirm it, or preview them first with `--dry-run`. `ok uninstall --yes`, and the macOS app's **Uninstall OpenKnowledge** command, stop every server they find without listing them. When that process is still running and cannot be confirmed, `ok stop` exits `1` without signalling, even with `--force`, and the error gives one of these reasons:
+
+- It could not read when the process started. On macOS this happens when `ps` cannot run, as inside some sandboxes. On Linux it happens when `/proc/<pid>/stat` or `/proc/stat` cannot be read; restore read access and retry. On Windows it happens when OpenKnowledge's bundled native component cannot be loaded, which reinstalling fixes, or when the component loaded but Windows refused the query, which reinstalling does not fix: rerun from an account that can inspect the server process. The Windows failure detail is recorded in `~/.ok/logs` (`~/.ok-beta/logs` for `ok-beta`), not printed.
+- The process started before the start the lock records.
+- The process start cannot be compared with the start the lock records, as when the lock records neither a process start nor a valid `startedAt`.
+- The lock records a `startedAt` well after the current time on this machine, as after the clock was set back, so it is not compared at all.
+- The lock records this machine's host name but neither a machine ID nor a process start, as locks written by version 0.25.0 and earlier do, so nothing in it confirms the running process as its server. Restarting the server on version 0.26.0 or later records its machine ID, so `ok stop` can confirm it.
+- The lock records a machine ID different from the one in `~/.ok/machine-id`, and no process start that matches the running process. Each home folder holds its own machine ID, so this also happens when the server was started on this machine from another account or with a different home folder. If it was started from another account, run `ok stop` as that account. If it was started from your account with a different home folder, run `ok stop` with `HOME` (`USERPROFILE` on Windows) set to that folder. Otherwise stop the server from the machine that owns the lock.
+- The lock records only another host name and no process start. If that host is another machine, stop the server there. If it is an earlier name of this machine, confirm that the process is an OpenKnowledge server before ending it yourself, and remove the lock file once no OpenKnowledge server is running for the project.
+- The lock does not record its owning machine and its process is still running. Check whether that process is an OpenKnowledge server; if it is, stop it yourself, and do not remove the lock while it is running. `ok stop all` and `ok stop` with a port or PID skip such a lock instead.
+
+For the first five, confirm that the process the error names is an OpenKnowledge server before ending it yourself, then remove the lock file the error names once no OpenKnowledge server is running for the project. In those five cases `ok status`, `ok ps` and the server-lock health check still report the server as running (`state: "alive"` in `ok status --json`) and `ok clean` keeps its lock; only `ok stop` says why the process cannot be confirmed.
+
+A lock records a process start, as its `processStart` field, only when the server that wrote it could read its own start. Locks written before that field existed have none, and on Linux a server records one only while `/proc/sys/kernel/random/boot_id` is readable. For a lock without a recorded start, `ok stop` treats the running process as the lock's server only when the lock carries this machine's ID and the process did not start well after the lock's `startedAt`; a lock from this machine whose process started well after it is stale, as described above. It refuses a lock whose `startedAt` is well after the current time, a lock that records no machine ID, and a lock that carries another machine's ID. That ID is read from `~/.ok/machine-id` in the home folder of whoever runs the command, so running `ok stop` with a different home folder from the server's (`HOME`, or `USERPROFILE` on Windows) reads its lock as another machine's. Restarting the server on a release that writes `processStart` records its start, after which `ok stop` can confirm it from either home folder. "Well after" means more than 10 seconds on macOS and Windows and more than 5 minutes on Linux, where a process's start time is derived from the boot time and shifts whenever the clock is set forward. So on Linux, a still-running server whose lock has no `processStart` reads as stale once the clock has moved forward by more than 5 minutes since it wrote its lock, as after a long VM or WSL2 pause or a clock set by hand. `ok clean` then removes its lock, and if that server predates the filesystem lease described above, `ok start` can start a second server for the same project. Restart such servers so that their lock records `processStart` to rule this out.
`ok status`, `ok ps`, and the server-lock health check distinguish an unverified owner from a foreign host. `ok status --json` uses `state: "unverified-owner"` and includes the PID, with server liveness reported as unknown. `ok ps --json` uses the same status, `displayStatus: "unverified"`, and `null` for unrecorded host, port, and start time. The directory is the project root for current `.ok/local` and legacy `.ok` locks; it is `null` when a custom lock location does not establish a project root. Such records are not signal authority for `ok stop`, including with `--force`.
@@ -188,13 +205,15 @@ The rest of the CLI exists mostly for tooling and automation. Your AI editor spa
Both `ok deinit` and `ok uninstall` wait for servers to exit before deleting their state. If a server cannot be safely stopped or an integration cannot be cleaned up, the command reports the problem and retains state needed for repair and retry. Applied cleanup exits `1` when anything remains unresolved. With `--yes --json`, inspect `failed[]` for those items and their reasons; `skipped[]` covers intentionally preserved entries. Repair the reported problem and rerun the command.
-On Windows, confirming that a running server really is the process the lock names uses OpenKnowledge's bundled native component. Removal refuses rather than signalling a process it has not identified, and the refusal names which of two things went wrong. If the component could not be loaded, reinstalling OpenKnowledge restores it — the same remedy as for [its TOML writer](#set-up-a-project). If the component loaded but Windows refused the query, reinstalling will not help: that is usually a permissions difference, so rerun from an account that can inspect the server process. Either way you can instead stop the server yourself and confirm it has exited, then rerun. The failure detail itself is recorded in `~/.ok/logs` (`~/.ok-beta/logs` for `ok-beta`), not printed.
+Removal checks a server more strictly than `ok stop` when its lock is from this machine and records no process start. `ok deinit` and `ok uninstall`, including the `ok uninstall --yes` that the macOS app's **Uninstall OpenKnowledge** command runs, refuse a process that started after the lock's `startedAt` at all, even one that did not start "well after" it as [Manage local servers](#manage-local-servers) defines that term: more than 10 seconds on macOS and Windows, more than 5 minutes on Linux. The refusal says the process "started after the server lock" and "was not signalled". Confirm that the process it names is an OpenKnowledge server before ending it yourself, then, once no OpenKnowledge server is running for the project, remove the lock file the refusal names and rerun.
+
+On Windows, confirming that a running server really is the process the lock names uses OpenKnowledge's bundled native component. Removal refuses rather than signalling a process it has not identified, and the refusal names which of two things went wrong. If the component could not be loaded, reinstalling OpenKnowledge restores it — the same remedy as for [its TOML writer](#set-up-a-project). If the component loaded but Windows refused the query, reinstalling will not help: that is usually a permissions difference, so rerun from an account that can inspect the server process. Either way you can instead check whether the process is an OpenKnowledge server and, if it is, stop it yourself; once no OpenKnowledge server is running for the project, remove the lock file the refusal names, then rerun. The failure detail itself is recorded in `~/.ok/logs` (`~/.ok-beta/logs` for `ok-beta`), not printed.
`ok deinit` needs the project's `.ok/` directory; once that directory is gone, it reports nothing to remove. Remove any retained Pi trust entry manually using its reported exact key and file, as described in [installation details](./what-open-knowledge-writes.mdx#pi-folder-trust).
Malformed filesystem residue does not by itself establish a running server. Cleanup can skip it after process and listener inspection succeeds and finds no associated live candidate. A read failure, incomplete inspection, or associated live server remains a blocker. A process that has exited is not a live server, and its unreadable working directory does not make the inspection incomplete, even while the process lingers as a defunct entry in the process table. Global-only uninstall can retain foreign-owned or ownership-unverified project state after the same local inspection; selecting project deinitialization keeps those ownership checks strict. Dead PID-only locks are recoverable. Human output lists root blockers once; JSON `failed[]` includes both direct failures and blocked dependent operations, with `blocked[]` providing the dependent subset.
-Both commands print their plan and ask before acting; `--dry-run` previews without changing anything. Once `ok uninstall` has finished successfully, an interactive run also asks one optional question about why you're leaving — you can skip it, and `--yes`, `--json`, and non-interactive runs never ask. Anything you do answer is [sent to us](/docs/reference/what-open-knowledge-writes#what-leaves-your-machine).
+Without `--yes`, both commands print their plan and ask before acting; `--yes` skips both the plan and the question. `--dry-run` previews without changing anything. Once `ok uninstall` has finished successfully, an interactive run also asks one optional question about why you're leaving — you can skip it, and `--yes`, `--json`, and non-interactive runs never ask. Anything you do answer is [sent to us](/docs/reference/what-open-knowledge-writes#what-leaves-your-machine).
On the macOS desktop app, **App menu → Uninstall OpenKnowledge…** lets you choose projects to deinitialize, confirm removal, and answer or skip the optional "why are you leaving?" screen. The progress window stays visible while the original app quits and cleanup runs, so shutdown cannot recreate its settings. The same window then reports the result: successful cleanup guides you through dragging the app to the Trash; incomplete cleanup points to a retained log so you can repair the problem and retry.
diff --git a/docs/content/reference/what-open-knowledge-writes.mdx b/docs/content/reference/what-open-knowledge-writes.mdx
index 85d7a8280..08be9381c 100644
--- a/docs/content/reference/what-open-knowledge-writes.mdx
+++ b/docs/content/reference/what-open-knowledge-writes.mdx
@@ -220,7 +220,7 @@ Installing `@inkeep/open-knowledge` from npm (or invoking it with `npx`) drops t
| Path | What it is | Scope |
| --- | --- | --- |
| `ok`, `open-knowledge` (on your `PATH`) | The two CLI binaries (identical entry point) | Outside-project (npm global/bin) |
-| `<pkg>/dist/native/` | Bundled native addon, shipped inside the package: TOML editing on all platforms, plus a Windows-only reader that checks a running server's process creation time during `ok uninstall` / `ok deinit` | Outside-project (package dir) |
+| `<pkg>/dist/native/` | Bundled native addon, shipped inside the package: TOML editing on all platforms, plus a Windows-only reader of a process's creation time, which confirms that the process a server lock names is the server that wrote it | Outside-project (package dir) |
<Callout type="info" title="Opt out">
Installing the package writes nothing outside its own directory, so there is nothing to opt out of here. The user-global skills are installed by `ok init`, and `ok init --no-skills` skips installing them for that run.
diff --git a/lint-plugins/ok-rules/README.md b/lint-plugins/ok-rules/README.md
index 2dc31284c..13fa3500f 100644
--- a/lint-plugins/ok-rules/README.md
+++ b/lint-plugins/ok-rules/README.md
@@ -681,7 +681,7 @@ Owned-process signalling. `process.kill(target, signal)` — and the desktop tes
- **The lifeline's own `kill -s KILL 0`.** That watcher ends the group it belongs to with `kill -s KILL 0` when its supervisor's end of the lifeline closes. It is the only signal to a literal `0` the STOP rule allows, and these two are its only acts beyond signalling a handle before it exits.
- **Negation is not laundering.** `-<expr>` is judged by exactly the same tests as `<expr>`, so a negated `?? <n>` fallback fires just as the bare fallback does, and only a pid already taken from a handle survives the minus sign.
- **Filter a nullable pid out of a list** instead of substituting a sentinel.
-- **A pid parsed from a file or a `ps` row is not a signal target**, because `isValidLockPid()` checks its range, not its owner. Stop that process through its owner, or report it.
+- **A pid parsed from a file or a `ps` row is not a signal target**, because `isValidLockPid()` checks its range, not its owner. Stop that process through its owner, or report it. The one sanctioned exception is a `server.lock` pid at the two sites that stop a server through its lock, `runStop` (`packages/cli/src/commands/stop.ts`) and the desktop's `terminateServerByPid` (`packages/desktop/src/main/window-manager.ts`). Each signals only after `assessLockHolder` (`packages/server/src/process-identity.ts`) reads the pid's current holder as the process that wrote the lock, immediately before every signal, and refuses otherwise; a reused pid is never signalled. A new site that stops a server through its lock routes through one of these two rather than adding its own kill.
**The reaper seam is what is absent.** `signalOwnedGroup`, `signalOwnedPids`, `reapOwnedTree` and `recordingSignalSender` have no definition on this head, so the seam's own sanctioned shape — an injected sender — is the one thing you cannot write here yet. Once the seam lands, a guard test that must feed it a sentinel passes `recordingSignalSender().send` — not the recorder itself — in the sender slot, which is the third argument to `signalOwnedGroup` and `signalOwnedPids` but the fourth to `reapOwnedTree`, and asserts the recorded calls. The rule stops firing only when a real sender value occupies that slot: passing `undefined` or `void 0` explicitly injects nothing and still fires, and passing `process.kill` itself is no escape either.
diff --git a/packages/app/src/lib/install-server-drift-listener.test.ts b/packages/app/src/lib/install-server-drift-listener.test.ts
index b172c53dd..43a2c2555 100644
--- a/packages/app/src/lib/install-server-drift-listener.test.ts
+++ b/packages/app/src/lib/install-server-drift-listener.test.ts
@@ -138,6 +138,19 @@ describe('drift copy', () => {
);
});
+ test.each([
+ [
+ false,
+ "Couldn't restart the server: OpenKnowledge could not confirm that the process holding this project's server lock is still its server, so it was not stopped. Stop the server where it was started, or restart your computer, then reopen this project.",
+ ],
+ [
+ true,
+ "Couldn't restart the server: OpenKnowledge asked the process holding this project's server lock to stop, but could not confirm that it is this project's server, so it did not force it to quit. If the project is still locked, stop the server where it was started, or restart your computer, then reopen this project.",
+ ],
+ ])('an unverified holder (sigtermSent %s) names no CLI command', (sigtermSent, expected) => {
+ expect(restartFailureMessage({ ok: false, reason: 'unverified', sigtermSent })).toBe(expected);
+ });
+
test('other failure in a Beta window suggests `ok-beta stop all`', () => {
vi.stubGlobal('window', { okDesktop: { cliCommandName: 'ok-beta' } });
try {
diff --git a/packages/app/src/lib/install-server-drift-listener.ts b/packages/app/src/lib/install-server-drift-listener.ts
index 97d603d34..af7c7f302 100644
--- a/packages/app/src/lib/install-server-drift-listener.ts
+++ b/packages/app/src/lib/install-server-drift-listener.ts
@@ -9,7 +9,10 @@ import type {
OkServerRestartOutcome,
OkServerVersionDriftInfo,
} from '@/lib/desktop-bridge-types';
-import { otherChannelRestartMessage } from '@/lib/restart-collab-server';
+import {
+ otherChannelRestartMessage,
+ unverifiedHolderRestartMessage,
+} from '@/lib/restart-collab-server';
export function restartDisruptionWarning(): string {
return t`Restarting closes this project's server. Connected agents (Claude Code, Codex, Cursor) will see their OpenKnowledge MCP connection close unexpectedly — you may need to restart the agent, or toggle its OpenKnowledge MCP server off and on, to reconnect.`;
@@ -38,6 +41,8 @@ export function restartFailureMessage(failure: OkServerRestartFailure): string {
const cliCommand = okCliCommandName();
return t`Couldn't restart the server automatically. Try running \`${cliCommand} stop all\` in a terminal, then reopen this project — or restart your computer if it persists.`;
}
+ case 'unverified':
+ return unverifiedHolderRestartMessage(failure.sigtermSent);
}
}
diff --git a/packages/app/src/lib/restart-collab-server.test.ts b/packages/app/src/lib/restart-collab-server.test.ts
index 7225d809b..443e0c038 100644
--- a/packages/app/src/lib/restart-collab-server.test.ts
+++ b/packages/app/src/lib/restart-collab-server.test.ts
@@ -23,6 +23,21 @@ describe('restartServerFailureMessage', () => {
);
});
+ test.each([
+ [
+ false,
+ "Couldn't restart the server: OpenKnowledge could not confirm that the process holding this project's server lock is still its server, so it was not stopped. Stop the server where it was started, or restart your computer, then reopen this project.",
+ ],
+ [
+ true,
+ "Couldn't restart the server: OpenKnowledge asked the process holding this project's server lock to stop, but could not confirm that it is this project's server, so it did not force it to quit. If the project is still locked, stop the server where it was started, or restart your computer, then reopen this project.",
+ ],
+ ])('an unverified holder (sigtermSent %s) names no CLI command', (sigtermSent, expected) => {
+ expect(restartServerFailureMessage({ ok: false, reason: 'unverified', sigtermSent })).toBe(
+ expected,
+ );
+ });
+
test('other in a Beta window points at ok-beta start', () => {
vi.stubGlobal('window', { okDesktop: { cliCommandName: 'ok-beta' } });
try {
diff --git a/packages/app/src/lib/restart-collab-server.ts b/packages/app/src/lib/restart-collab-server.ts
index 5646f92b9..04b609205 100644
--- a/packages/app/src/lib/restart-collab-server.ts
+++ b/packages/app/src/lib/restart-collab-server.ts
@@ -13,9 +13,17 @@ export function restartServerFailureMessage(failure: OkServerRestartFailure): st
const cliCommand = okCliCommandName();
return t`Couldn't restart the server. Try \`${cliCommand} start\` in this folder.`;
}
+ case 'unverified':
+ return unverifiedHolderRestartMessage(failure.sigtermSent);
}
}
+export function unverifiedHolderRestartMessage(sigtermSent: boolean): string {
+ return sigtermSent
+ ? t`Couldn't restart the server: OpenKnowledge asked the process holding this project's server lock to stop, but could not confirm that it is this project's server, so it did not force it to quit. If the project is still locked, stop the server where it was started, or restart your computer, then reopen this project.`
+ : t`Couldn't restart the server: OpenKnowledge could not confirm that the process holding this project's server lock is still its server, so it was not stopped. Stop the server where it was started, or restart your computer, then reopen this project.`;
+}
+
export function otherChannelRestartMessage(holderChannel: string): string {
const holder = desktopChannelLabel(holderChannel);
return t`${holder} is serving this project, so this app won't stop it. Close this window and open the project again to stop ${holder}'s server and open the project here.`;
diff --git a/packages/app/src/locales/ar/messages.po b/packages/app/src/locales/ar/messages.po
index d3f735e1e..a301797f2 100644
--- a/packages/app/src/locales/ar/messages.po
+++ b/packages/app/src/locales/ar/messages.po
@@ -3163,6 +3163,12 @@ msgstr "تعذّرت إعادة تشغيل الخادم — يعمل تحت حس
msgid "Couldn't restart the server automatically. Try running `{cliCommand} stop all` in a terminal, then reopen this project — or restart your computer if it persists."
msgstr "تعذّرت إعادة تشغيل الخادم تلقائيًا. جرّب تشغيل `{cliCommand} stop all` في طرفية، ثم أعد فتح هذا المشروع — أو أعد تشغيل جهازك إن استمرت المشكلة."
+msgid "Couldn't restart the server: OpenKnowledge asked the process holding this project's server lock to stop, but could not confirm that it is this project's server, so it did not force it to quit. If the project is still locked, stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "تعذّرت إعادة تشغيل الخادم: طلب OpenKnowledge من العملية التي تحتفظ بقفل خادم هذا المشروع أن تتوقف، لكنه لم يتمكن من التأكد من أنها خادم هذا المشروع، لذلك لم يُجبرها على الإنهاء. إذا ظل المشروع مقفلًا، فأوقف الخادم من حيث بدأ تشغيله، أو أعد تشغيل جهازك، ثم أعد فتح هذا المشروع."
+
+msgid "Couldn't restart the server: OpenKnowledge could not confirm that the process holding this project's server lock is still its server, so it was not stopped. Stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "تعذّرت إعادة تشغيل الخادم: لم يتمكن OpenKnowledge من التأكد من أن العملية التي تحتفظ بقفل خادم هذا المشروع لا تزال خادمه، لذلك لم يوقفها. أوقف الخادم من حيث بدأ تشغيله، أو أعد تشغيل جهازك، ثم أعد فتح هذا المشروع."
+
msgid "Couldn't restart the server. Run {cliCommand} start in this folder to retry."
msgstr "تعذّر إعادة تشغيل الخادم. شغّل {cliCommand} start في هذا المجلد لإعادة المحاولة."
diff --git a/packages/app/src/locales/bn/messages.po b/packages/app/src/locales/bn/messages.po
index cd5078fea..0d14d6188 100644
--- a/packages/app/src/locales/bn/messages.po
+++ b/packages/app/src/locales/bn/messages.po
@@ -3159,6 +3159,12 @@ msgstr "সার্ভার পুনরায় চালু করা য
msgid "Couldn't restart the server automatically. Try running `{cliCommand} stop all` in a terminal, then reopen this project — or restart your computer if it persists."
msgstr "সার্ভার নিজে থেকে পুনরায় চালু করা যায়নি। টার্মিনালে `{cliCommand} stop all` চালিয়ে দেখুন, তারপর এই প্রকল্পটি আবার খুলুন — সমস্যা থাকলে কম্পিউটার পুনরায় চালু করুন।"
+msgid "Couldn't restart the server: OpenKnowledge asked the process holding this project's server lock to stop, but could not confirm that it is this project's server, so it did not force it to quit. If the project is still locked, stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "সার্ভার পুনরায় চালু করা যায়নি: OpenKnowledge এই প্রকল্পের সার্ভার লক ধরে রাখা প্রক্রিয়াটিকে থামতে বলেছিল, কিন্তু এটি এই প্রকল্পের সার্ভার কি না তা নিশ্চিত করতে পারেনি, তাই এটিকে জোর করে বন্ধ করেনি। প্রকল্পটি এখনও লক থাকলে, সার্ভারটি যেখানে চালু করা হয়েছিল সেখানে থামান, অথবা কম্পিউটার পুনরায় চালু করুন, তারপর এই প্রকল্পটি আবার খুলুন।"
+
+msgid "Couldn't restart the server: OpenKnowledge could not confirm that the process holding this project's server lock is still its server, so it was not stopped. Stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "সার্ভার পুনরায় চালু করা যায়নি: এই প্রকল্পের সার্ভার লক ধরে রাখা প্রক্রিয়াটি এখনও এর সার্ভার কি না তা OpenKnowledge নিশ্চিত করতে পারেনি, তাই এটিকে থামানো হয়নি। সার্ভারটি যেখানে চালু করা হয়েছিল সেখানে থামান, অথবা কম্পিউটার পুনরায় চালু করুন, তারপর এই প্রকল্পটি আবার খুলুন।"
+
msgid "Couldn't restart the server. Run {cliCommand} start in this folder to retry."
msgstr "সার্ভার পুনরায় চালু করা যায়নি। আবার চেষ্টা করতে এই ফোল্ডারে {cliCommand} start চালান।"
diff --git a/packages/app/src/locales/en/messages.po b/packages/app/src/locales/en/messages.po
index 4a6ab4a0d..71ec4dc24 100644
--- a/packages/app/src/locales/en/messages.po
+++ b/packages/app/src/locales/en/messages.po
@@ -3163,6 +3163,12 @@ msgstr "Couldn't restart the server — it's running under a different account.
msgid "Couldn't restart the server automatically. Try running `{cliCommand} stop all` in a terminal, then reopen this project — or restart your computer if it persists."
msgstr "Couldn't restart the server automatically. Try running `{cliCommand} stop all` in a terminal, then reopen this project — or restart your computer if it persists."
+msgid "Couldn't restart the server: OpenKnowledge asked the process holding this project's server lock to stop, but could not confirm that it is this project's server, so it did not force it to quit. If the project is still locked, stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "Couldn't restart the server: OpenKnowledge asked the process holding this project's server lock to stop, but could not confirm that it is this project's server, so it did not force it to quit. If the project is still locked, stop the server where it was started, or restart your computer, then reopen this project."
+
+msgid "Couldn't restart the server: OpenKnowledge could not confirm that the process holding this project's server lock is still its server, so it was not stopped. Stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "Couldn't restart the server: OpenKnowledge could not confirm that the process holding this project's server lock is still its server, so it was not stopped. Stop the server where it was started, or restart your computer, then reopen this project."
+
msgid "Couldn't restart the server. Run {cliCommand} start in this folder to retry."
msgstr "Couldn't restart the server. Run {cliCommand} start in this folder to retry."
diff --git a/packages/app/src/locales/es/messages.po b/packages/app/src/locales/es/messages.po
index 1b3acc4bb..5bf45326a 100644
--- a/packages/app/src/locales/es/messages.po
+++ b/packages/app/src/locales/es/messages.po
@@ -3163,6 +3163,12 @@ msgstr "No se pudo reiniciar el servidor: se está ejecutando con otra cuenta. R
msgid "Couldn't restart the server automatically. Try running `{cliCommand} stop all` in a terminal, then reopen this project — or restart your computer if it persists."
msgstr "No se pudo reiniciar el servidor automáticamente. Prueba a ejecutar `{cliCommand} stop all` en una terminal y vuelve a abrir este proyecto; si sigue igual, reinicia el equipo."
+msgid "Couldn't restart the server: OpenKnowledge asked the process holding this project's server lock to stop, but could not confirm that it is this project's server, so it did not force it to quit. If the project is still locked, stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "No se pudo reiniciar el servidor: OpenKnowledge pidió al proceso que tiene el bloqueo del servidor de este proyecto que se detuviera, pero no pudo confirmar que sea el servidor de este proyecto, así que no lo forzó a cerrarse. Si el proyecto sigue bloqueado, detén el servidor donde se inició o reinicia el equipo, y vuelve a abrir este proyecto."
+
+msgid "Couldn't restart the server: OpenKnowledge could not confirm that the process holding this project's server lock is still its server, so it was not stopped. Stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "No se pudo reiniciar el servidor: OpenKnowledge no pudo confirmar que el proceso que tiene el bloqueo del servidor de este proyecto siga siendo su servidor, así que no lo detuvo. Detén el servidor donde se inició o reinicia el equipo, y vuelve a abrir este proyecto."
+
msgid "Couldn't restart the server. Run {cliCommand} start in this folder to retry."
msgstr "No se pudo reiniciar el servidor. Ejecuta {cliCommand} start en esta carpeta para reintentar."
diff --git a/packages/app/src/locales/fr/messages.po b/packages/app/src/locales/fr/messages.po
index 55b6f5347..9df56619e 100644
--- a/packages/app/src/locales/fr/messages.po
+++ b/packages/app/src/locales/fr/messages.po
@@ -3163,6 +3163,12 @@ msgstr "Impossible de redémarrer le serveur — il s'exécute sous un autre com
msgid "Couldn't restart the server automatically. Try running `{cliCommand} stop all` in a terminal, then reopen this project — or restart your computer if it persists."
msgstr "Impossible de redémarrer automatiquement le serveur. Essayez d'exécuter `{cliCommand} stop all` dans un terminal, puis rouvrez ce projet — ou redémarrez votre ordinateur si le problème persiste."
+msgid "Couldn't restart the server: OpenKnowledge asked the process holding this project's server lock to stop, but could not confirm that it is this project's server, so it did not force it to quit. If the project is still locked, stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "Impossible de redémarrer le serveur : OpenKnowledge a demandé au processus qui détient le verrou du serveur de ce projet de s'arrêter, mais n'a pas pu confirmer qu'il s'agit du serveur de ce projet ; il ne l'a donc pas forcé à quitter. Si le projet est toujours verrouillé, arrêtez le serveur là où il a été démarré, ou redémarrez votre ordinateur, puis rouvrez ce projet."
+
+msgid "Couldn't restart the server: OpenKnowledge could not confirm that the process holding this project's server lock is still its server, so it was not stopped. Stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "Impossible de redémarrer le serveur : OpenKnowledge n'a pas pu confirmer que le processus qui détient le verrou du serveur de ce projet est toujours son serveur ; il ne l'a donc pas arrêté. Arrêtez le serveur là où il a été démarré, ou redémarrez votre ordinateur, puis rouvrez ce projet."
+
msgid "Couldn't restart the server. Run {cliCommand} start in this folder to retry."
msgstr "Impossible de redémarrer le serveur. Exécutez {cliCommand} start dans ce dossier pour réessayer."
diff --git a/packages/app/src/locales/hi/messages.po b/packages/app/src/locales/hi/messages.po
index 813e97ec7..959b11716 100644
--- a/packages/app/src/locales/hi/messages.po
+++ b/packages/app/src/locales/hi/messages.po
@@ -3159,6 +3159,12 @@ msgstr "सर्वर पुनः आरंभ नहीं किया ज
msgid "Couldn't restart the server automatically. Try running `{cliCommand} stop all` in a terminal, then reopen this project — or restart your computer if it persists."
msgstr "सर्वर अपने आप पुनः आरंभ नहीं किया जा सका। टर्मिनल में `{cliCommand} stop all` चलाकर देखें, फिर यह प्रोजेक्ट दोबारा खोलें — या समस्या बनी रहे तो अपना कंप्यूटर पुनः आरंभ करें।"
+msgid "Couldn't restart the server: OpenKnowledge asked the process holding this project's server lock to stop, but could not confirm that it is this project's server, so it did not force it to quit. If the project is still locked, stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "सर्वर पुनः आरंभ नहीं किया जा सका: OpenKnowledge ने इस प्रोजेक्ट का सर्वर लॉक रखने वाली प्रक्रिया से रुकने को कहा, लेकिन यह पुष्टि नहीं कर सका कि वह इस प्रोजेक्ट का सर्वर है, इसलिए उसे ज़बरदस्ती बंद नहीं किया। अगर प्रोजेक्ट अब भी लॉक है, तो सर्वर को वहीं रोकें जहाँ उसे शुरू किया गया था, या अपना कंप्यूटर पुनः आरंभ करें, फिर यह प्रोजेक्ट दोबारा खोलें।"
+
+msgid "Couldn't restart the server: OpenKnowledge could not confirm that the process holding this project's server lock is still its server, so it was not stopped. Stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "सर्वर पुनः आरंभ नहीं किया जा सका: OpenKnowledge यह पुष्टि नहीं कर सका कि इस प्रोजेक्ट का सर्वर लॉक रखने वाली प्रक्रिया अब भी उसका सर्वर है, इसलिए उसे रोका नहीं गया। सर्वर को वहीं रोकें जहाँ उसे शुरू किया गया था, या अपना कंप्यूटर पुनः आरंभ करें, फिर यह प्रोजेक्ट दोबारा खोलें।"
+
msgid "Couldn't restart the server. Run {cliCommand} start in this folder to retry."
msgstr "सर्वर पुनः आरंभ नहीं किया जा सका। पुनः प्रयास करने के लिए इस फ़ोल्डर में {cliCommand} start चलाएँ।"
diff --git a/packages/app/src/locales/id/messages.po b/packages/app/src/locales/id/messages.po
index 1db534a9c..28bcdfe02 100644
--- a/packages/app/src/locales/id/messages.po
+++ b/packages/app/src/locales/id/messages.po
@@ -3159,6 +3159,12 @@ msgstr "Server tidak bisa dimulai ulang — ia berjalan di bawah akun lain. Mula
msgid "Couldn't restart the server automatically. Try running `{cliCommand} stop all` in a terminal, then reopen this project — or restart your computer if it persists."
msgstr "Server tidak bisa dimulai ulang secara otomatis. Coba jalankan `{cliCommand} stop all` di terminal, lalu buka lagi proyek ini — atau mulai ulang komputer Anda jika masih berlanjut."
+msgid "Couldn't restart the server: OpenKnowledge asked the process holding this project's server lock to stop, but could not confirm that it is this project's server, so it did not force it to quit. If the project is still locked, stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "Server tidak bisa dimulai ulang: OpenKnowledge meminta proses yang memegang kunci server proyek ini untuk berhenti, tetapi tidak dapat memastikan bahwa itu adalah server proyek ini, jadi proses itu tidak dipaksa keluar. Jika proyek masih terkunci, hentikan server di tempat server itu dijalankan, atau mulai ulang komputer Anda, lalu buka lagi proyek ini."
+
+msgid "Couldn't restart the server: OpenKnowledge could not confirm that the process holding this project's server lock is still its server, so it was not stopped. Stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "Server tidak bisa dimulai ulang: OpenKnowledge tidak dapat memastikan bahwa proses yang memegang kunci server proyek ini masih servernya, jadi proses itu tidak dihentikan. Hentikan server di tempat server itu dijalankan, atau mulai ulang komputer Anda, lalu buka lagi proyek ini."
+
msgid "Couldn't restart the server. Run {cliCommand} start in this folder to retry."
msgstr "Tidak dapat memulai ulang server. Jalankan {cliCommand} start di folder ini untuk mencoba lagi."
diff --git a/packages/app/src/locales/ko/messages.po b/packages/app/src/locales/ko/messages.po
index aee9f65d9..101f9447e 100644
--- a/packages/app/src/locales/ko/messages.po
+++ b/packages/app/src/locales/ko/messages.po
@@ -3163,6 +3163,12 @@ msgstr "서버를 다시 시작하지 못했습니다. 다른 계정으로 실
msgid "Couldn't restart the server automatically. Try running `{cliCommand} stop all` in a terminal, then reopen this project — or restart your computer if it persists."
msgstr "서버를 자동으로 다시 시작하지 못했습니다. 터미널에서 `{cliCommand} stop all`을 실행한 다음 이 프로젝트를 다시 여세요. 문제가 계속되면 컴퓨터를 다시 시작하세요."
+msgid "Couldn't restart the server: OpenKnowledge asked the process holding this project's server lock to stop, but could not confirm that it is this project's server, so it did not force it to quit. If the project is still locked, stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "서버를 다시 시작하지 못했습니다. OpenKnowledge가 이 프로젝트의 서버 잠금을 가진 프로세스에 중지를 요청했지만, 이 프로젝트의 서버인지 확인할 수 없어 강제로 종료하지 않았습니다. 프로젝트가 여전히 잠겨 있으면 서버를 시작한 곳에서 중지하거나 컴퓨터를 다시 시작한 다음 이 프로젝트를 다시 여세요."
+
+msgid "Couldn't restart the server: OpenKnowledge could not confirm that the process holding this project's server lock is still its server, so it was not stopped. Stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "서버를 다시 시작하지 못했습니다. OpenKnowledge가 이 프로젝트의 서버 잠금을 가진 프로세스가 여전히 이 프로젝트의 서버인지 확인할 수 없어 중지하지 않았습니다. 서버를 시작한 곳에서 중지하거나 컴퓨터를 다시 시작한 다음 이 프로젝트를 다시 여세요."
+
msgid "Couldn't restart the server. Run {cliCommand} start in this folder to retry."
msgstr "서버를 다시 시작할 수 없습니다. 다시 시도하려면 이 폴더에서 {cliCommand} start를 실행하세요."
diff --git a/packages/app/src/locales/pl/messages.po b/packages/app/src/locales/pl/messages.po
index 6767674bb..f044cf5ea 100644
--- a/packages/app/src/locales/pl/messages.po
+++ b/packages/app/src/locales/pl/messages.po
@@ -3163,6 +3163,12 @@ msgstr "Nie udało się zrestartować serwera — działa na innym koncie. Uruch
msgid "Couldn't restart the server automatically. Try running `{cliCommand} stop all` in a terminal, then reopen this project — or restart your computer if it persists."
msgstr "Nie udało się automatycznie zrestartować serwera. Spróbuj uruchomić `{cliCommand} stop all` w terminalu, potem otwórz ten projekt ponownie — albo zrestartuj komputer, jeśli problem się utrzymuje."
+msgid "Couldn't restart the server: OpenKnowledge asked the process holding this project's server lock to stop, but could not confirm that it is this project's server, so it did not force it to quit. If the project is still locked, stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "Nie udało się zrestartować serwera: OpenKnowledge poprosił proces, który trzyma blokadę serwera tego projektu, o zatrzymanie, ale nie mógł potwierdzić, że jest to serwer tego projektu, więc nie wymusił jego zamknięcia. Jeśli projekt nadal jest zablokowany, zatrzymaj serwer tam, gdzie został uruchomiony, lub uruchom ponownie komputer, a potem otwórz ten projekt ponownie."
+
+msgid "Couldn't restart the server: OpenKnowledge could not confirm that the process holding this project's server lock is still its server, so it was not stopped. Stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "Nie udało się zrestartować serwera: OpenKnowledge nie mógł potwierdzić, że proces, który trzyma blokadę serwera tego projektu, nadal jest jego serwerem, więc go nie zatrzymał. Zatrzymaj serwer tam, gdzie został uruchomiony, lub uruchom ponownie komputer, a potem otwórz ten projekt ponownie."
+
msgid "Couldn't restart the server. Run {cliCommand} start in this folder to retry."
msgstr "Nie udało się zrestartować serwera. Uruchom {cliCommand} start w tym folderze, aby spróbować ponownie."
diff --git a/packages/app/src/locales/pt-BR/messages.po b/packages/app/src/locales/pt-BR/messages.po
index f9065918d..6e6c48c98 100644
--- a/packages/app/src/locales/pt-BR/messages.po
+++ b/packages/app/src/locales/pt-BR/messages.po
@@ -3159,6 +3159,12 @@ msgstr "Não deu para reiniciar o servidor — ele está rodando em outra conta.
msgid "Couldn't restart the server automatically. Try running `{cliCommand} stop all` in a terminal, then reopen this project — or restart your computer if it persists."
msgstr "Não deu para reiniciar o servidor automaticamente. Tente rodar `{cliCommand} stop all` em um terminal e abra este projeto de novo — ou reinicie o computador se persistir."
+msgid "Couldn't restart the server: OpenKnowledge asked the process holding this project's server lock to stop, but could not confirm that it is this project's server, so it did not force it to quit. If the project is still locked, stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "Não deu para reiniciar o servidor: o OpenKnowledge pediu ao processo que detém o bloqueio do servidor deste projeto que parasse, mas não conseguiu confirmar que ele é o servidor deste projeto, então não o forçou a fechar. Se o projeto continuar bloqueado, pare o servidor onde ele foi iniciado ou reinicie o computador e abra este projeto de novo."
+
+msgid "Couldn't restart the server: OpenKnowledge could not confirm that the process holding this project's server lock is still its server, so it was not stopped. Stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "Não deu para reiniciar o servidor: o OpenKnowledge não conseguiu confirmar que o processo que detém o bloqueio do servidor deste projeto ainda é o servidor dele, então não o parou. Pare o servidor onde ele foi iniciado ou reinicie o computador e abra este projeto de novo."
+
msgid "Couldn't restart the server. Run {cliCommand} start in this folder to retry."
msgstr "Não foi possível reiniciar o servidor. Execute {cliCommand} start nesta pasta para tentar novamente."
diff --git a/packages/app/src/locales/ur/messages.po b/packages/app/src/locales/ur/messages.po
index 12d974f63..68e37c032 100644
--- a/packages/app/src/locales/ur/messages.po
+++ b/packages/app/src/locales/ur/messages.po
@@ -3163,6 +3163,12 @@ msgstr "سرور دوبارہ نہیں چل سکا — یہ کسی مختلف ا
msgid "Couldn't restart the server automatically. Try running `{cliCommand} stop all` in a terminal, then reopen this project — or restart your computer if it persists."
msgstr "سرور خودکار طور پر دوبارہ نہیں چل سکا۔ ٹرمینل میں `{cliCommand} stop all` چلا کر دیکھیں، پھر یہ پروجیکٹ دوبارہ کھولیں — یا مسئلہ برقرار رہے تو اپنا کمپیوٹر دوبارہ چالو کریں۔"
+msgid "Couldn't restart the server: OpenKnowledge asked the process holding this project's server lock to stop, but could not confirm that it is this project's server, so it did not force it to quit. If the project is still locked, stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "سرور دوبارہ نہیں چل سکا: OpenKnowledge نے اس پروجیکٹ کا سرور لاک رکھنے والے پروسیس سے رکنے کو کہا، لیکن یہ تصدیق نہیں کر سکا کہ وہ اس پروجیکٹ کا سرور ہے، اس لیے اسے زبردستی بند نہیں کیا۔ اگر پروجیکٹ اب بھی لاک ہے تو سرور کو وہیں روکیں جہاں اسے شروع کیا گیا تھا، یا اپنا کمپیوٹر دوبارہ چالو کریں، پھر یہ پروجیکٹ دوبارہ کھولیں۔"
+
+msgid "Couldn't restart the server: OpenKnowledge could not confirm that the process holding this project's server lock is still its server, so it was not stopped. Stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "سرور دوبارہ نہیں چل سکا: OpenKnowledge یہ تصدیق نہیں کر سکا کہ اس پروجیکٹ کا سرور لاک رکھنے والا پروسیس اب بھی اس کا سرور ہے، اس لیے اسے روکا نہیں گیا۔ سرور کو وہیں روکیں جہاں اسے شروع کیا گیا تھا، یا اپنا کمپیوٹر دوبارہ چالو کریں، پھر یہ پروجیکٹ دوبارہ کھولیں۔"
+
msgid "Couldn't restart the server. Run {cliCommand} start in this folder to retry."
msgstr "سرور دوبارہ شروع نہیں ہو سکا۔ دوبارہ کوشش کرنے کے لیے اس فولڈر میں {cliCommand} start چلائیں۔"
diff --git a/packages/app/src/locales/zh-Hans/messages.po b/packages/app/src/locales/zh-Hans/messages.po
index 81f33c139..42bfde3c0 100644
--- a/packages/app/src/locales/zh-Hans/messages.po
+++ b/packages/app/src/locales/zh-Hans/messages.po
@@ -3163,6 +3163,12 @@ msgstr "无法重启服务器——它正以另一个账户运行。请重启电
msgid "Couldn't restart the server automatically. Try running `{cliCommand} stop all` in a terminal, then reopen this project — or restart your computer if it persists."
msgstr "无法自动重启服务器。请在终端中运行 `{cliCommand} stop all`,然后重新打开这个项目——如果问题依旧,请重启电脑。"
+msgid "Couldn't restart the server: OpenKnowledge asked the process holding this project's server lock to stop, but could not confirm that it is this project's server, so it did not force it to quit. If the project is still locked, stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "无法重启服务器:OpenKnowledge 已请求持有这个项目服务器锁的进程停止,但无法确认它是这个项目的服务器,因此没有强制它退出。如果项目仍被锁定,请在启动该服务器的地方将其停止,或重启电脑,然后重新打开这个项目。"
+
+msgid "Couldn't restart the server: OpenKnowledge could not confirm that the process holding this project's server lock is still its server, so it was not stopped. Stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "无法重启服务器:OpenKnowledge 无法确认持有这个项目服务器锁的进程仍是它的服务器,因此没有将其停止。请在启动该服务器的地方将其停止,或重启电脑,然后重新打开这个项目。"
+
msgid "Couldn't restart the server. Run {cliCommand} start in this folder to retry."
msgstr "无法重启服务器。请在此文件夹中运行 {cliCommand} start 以重试。"
diff --git a/packages/app/src/locales/zh-Hant/messages.po b/packages/app/src/locales/zh-Hant/messages.po
index 16a440e81..032e585ae 100644
--- a/packages/app/src/locales/zh-Hant/messages.po
+++ b/packages/app/src/locales/zh-Hant/messages.po
@@ -3163,6 +3163,12 @@ msgstr "無法重新啟動伺服器——它正以另一個帳戶執行。請重
msgid "Couldn't restart the server automatically. Try running `{cliCommand} stop all` in a terminal, then reopen this project — or restart your computer if it persists."
msgstr "無法自動重新啟動伺服器。請在終端機中執行 `{cliCommand} stop all`,然後重新開啟這個專案——如果狀況持續,請重新啟動電腦。"
+msgid "Couldn't restart the server: OpenKnowledge asked the process holding this project's server lock to stop, but could not confirm that it is this project's server, so it did not force it to quit. If the project is still locked, stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "無法重新啟動伺服器:OpenKnowledge 已要求持有這個專案伺服器鎖定的處理程序停止,但無法確認它是這個專案的伺服器,因此沒有強制它結束。如果專案仍被鎖定,請在啟動該伺服器的地方將它停止,或重新啟動電腦,再重新開啟這個專案。"
+
+msgid "Couldn't restart the server: OpenKnowledge could not confirm that the process holding this project's server lock is still its server, so it was not stopped. Stop the server where it was started, or restart your computer, then reopen this project."
+msgstr "無法重新啟動伺服器:OpenKnowledge 無法確認持有這個專案伺服器鎖定的處理程序仍是它的伺服器,因此沒有將它停止。請在啟動該伺服器的地方將它停止,或重新啟動電腦,再重新開啟這個專案。"
+
msgid "Couldn't restart the server. Run {cliCommand} start in this folder to retry."
msgstr "無法重新啟動伺服器。請在此資料夾中執行 {cliCommand} start 以重試。"
diff --git a/packages/cli/src/cli.test.ts b/packages/cli/src/cli.test.ts
index d803517a3..3ad733cc0 100644
--- a/packages/cli/src/cli.test.ts
+++ b/packages/cli/src/cli.test.ts
@@ -147,6 +147,33 @@ describe('CLI --version notice', () => {
}, 30_000);
});
+describe('the Windows process-start reader', () => {
+ test('the CLI entry registers it for every command', () => {
+ const result = Bun.spawnSync({
+ cmd: [
+ 'node',
+ '--import',
+ 'tsx',
+ '--conditions=development',
+ '-e',
+ `
+ process.on('exit', () => {
+ const reader = globalThis[Symbol.for('@inkeep/open-knowledge/native-process-start-reader')];
+ process.stdout.write('process-start-reader=' + typeof reader + '\\n');
+ });
+ process.argv = [process.execPath, process.cwd() + '/src/cli.ts', '--version'];
+ await import('./src/cli.ts');
+ `,
+ ],
+ cwd: CLI_PACKAGE_ROOT,
+ env: { ...process.env, NO_COLOR: '1' },
+ });
+
+ expect(result.exitCode).toBe(0);
+ expect(result.stdout.toString()).toContain('process-start-reader=function\n');
+ }, 30_000);
+});
+
describe('committed project-local key startup warning', () => {
let projectDir: string;
diff --git a/packages/cli/src/cli.ts b/packages/cli/src/cli.ts
index 20ca70f97..e44c217cb 100644
--- a/packages/cli/src/cli.ts
+++ b/packages/cli/src/cli.ts
@@ -9,6 +9,7 @@ if (process.argv.includes('--no-color')) {
}
trustSystemCertificates();
+registerWindowsProcessStartReader();
import { spawn } from 'node:child_process';
import { resolve } from 'node:path';
@@ -32,6 +33,7 @@ import { deinitCommand } from './commands/deinit.ts';
import { createRealDetectDeps, detectDesktop, launchDesktop } from './commands/desktop-dispatch.ts';
import { diagnoseCommand } from './commands/diagnose.ts';
import { embeddingsCommand } from './commands/embeddings/index.ts';
+import { registerWindowsProcessStartReader } from './commands/holder-process-start.ts';
import { initCommand } from './commands/init.ts';
import { lintCommand } from './commands/lint.ts';
import { mcpCommand } from './commands/mcp.ts';
diff --git a/packages/cli/src/commands/clean.test.ts b/packages/cli/src/commands/clean.test.ts
index 825e60d6d..9557b679c 100644
--- a/packages/cli/src/commands/clean.test.ts
+++ b/packages/cli/src/commands/clean.test.ts
@@ -1,5 +1,13 @@
import type { ChildProcess } from 'node:child_process';
-import { existsSync, mkdirSync, mkdtempSync, realpathSync, rmSync, writeFileSync } from 'node:fs';
+import {
+ existsSync,
+ mkdirSync,
+ mkdtempSync,
+ readFileSync,
+ realpathSync,
+ rmSync,
+ writeFileSync,
+} from 'node:fs';
import { hostname, tmpdir } from 'node:os';
import { join } from 'node:path';
import { afterEach, describe, expect, test } from 'vitest';
@@ -23,21 +31,28 @@ function alive(pid: number, port: number): LockState {
function missing(): LockState {
return { status: 'missing', lockPath: '/tmp/server.lock' };
}
-function corrupt(): LockState {
- return { status: 'corrupt', lockPath: '/tmp/server.lock' };
+function corrupt(): Extract<LockState, { status: 'corrupt' }> {
+ return { status: 'corrupt', lockPath: '/tmp/server.lock', text: '{' };
}
-function dead(pid: number): LockState {
- return {
- status: 'dead-pid',
- lockPath: '/tmp/server.lock',
- lock: {
- pid,
- port: 0,
- hostname: 'host',
- startedAt: '2026-04-16T00:00:00Z',
- worktreeRoot: '/x',
- },
+function dead(pid: number): Extract<LockState, { status: 'dead-pid' }> {
+ const lock = {
+ pid,
+ port: 0,
+ hostname: 'host',
+ startedAt: '2026-04-16T00:00:00Z',
+ worktreeRoot: '/x',
};
+ return { status: 'dead-pid', lockPath: '/tmp/server.lock', lock, text: JSON.stringify(lock) };
+}
+function reused(pid: number): Extract<LockState, { status: 'reused-pid' }> {
+ const lock = {
+ pid,
+ port: 3000,
+ hostname: 'host',
+ startedAt: '2026-04-16T00:00:00Z',
+ worktreeRoot: '/x',
+ };
+ return { status: 'reused-pid', lockPath: '/tmp/server.lock', lock, text: JSON.stringify(lock) };
}
function foreign(): LockState {
return {
@@ -60,16 +75,33 @@ describe('buildCleanPlan', () => {
});
test('dead-pid → pruned', () => {
- const plan = buildCleanPlan(dead(999));
- expect(plan.prune).toEqual([
- { name: 'server', lockPath: '/tmp/server.lock', reason: 'dead-pid' },
+ const checked = dead(999);
+ expect(buildCleanPlan(checked).prune).toEqual([
+ {
+ name: 'server',
+ lockPath: '/tmp/server.lock',
+ reason: 'dead-pid',
+ text: checked.text,
+ },
+ ]);
+ });
+
+ test('reused-pid → pruned', () => {
+ const checked = reused(4242);
+ expect(buildCleanPlan(checked).prune).toEqual([
+ {
+ name: 'server',
+ lockPath: '/tmp/server.lock',
+ reason: 'reused-pid',
+ text: checked.text,
+ },
]);
});
test('corrupt → pruned', () => {
const plan = buildCleanPlan(corrupt());
expect(plan.prune).toEqual([
- { name: 'server', lockPath: '/tmp/server.lock', reason: 'corrupt' },
+ { name: 'server', lockPath: '/tmp/server.lock', reason: 'corrupt', text: '{' },
]);
});
@@ -152,6 +184,21 @@ describe('runClean', () => {
expect(logs.at(0)).toContain('server (dead-pid)');
});
+ test('a lock whose pid now belongs to another process is pruned and labelled reused-pid', () => {
+ const logs: string[] = [];
+ const unlinked: string[] = [];
+ const outcome = runClean({
+ lockDir: '/tmp/x',
+ inspect: () => reused(4242),
+ unlink: (p) => unlinked.push(p),
+ log: (msg) => logs.push(msg),
+ error: () => {},
+ });
+ expect(unlinked).toEqual(['/tmp/server.lock']);
+ expect(outcome.failed).toEqual([]);
+ expect(logs).toEqual(['Pruned 1 stale lock: server (reused-pid)']);
+ });
+
test('unlink failure → reported as failed', () => {
const errors: string[] = [];
const outcome = runClean({
@@ -174,7 +221,12 @@ test('never prunes a malformed foreign-owned lock', () => {
const unlinked: string[] = [];
const outcome = runClean({
lockDir: '/tmp/x',
- inspect: () => ({ status: 'corrupt', lockPath: '/tmp/foreign.lock', foreignHost: true }),
+ inspect: () => ({
+ status: 'corrupt',
+ lockPath: '/tmp/foreign.lock',
+ text: '{',
+ foreignHost: true,
+ }),
unlink: (path) => unlinked.push(path),
log: () => {},
error: () => {},
@@ -183,6 +235,115 @@ test('never prunes a malformed foreign-owned lock', () => {
expect(outcome.failed[0]?.error).toContain('owning machine');
});
+describe('runClean over a lock another server took after it was checked', () => {
+ let root: string | undefined;
+
+ afterEach(() => {
+ if (root !== undefined) rmSync(root, { recursive: true, force: true });
+ root = undefined;
+ });
+
+ const checkedLock = {
+ pid: 4242,
+ hostname: hostname(),
+ port: 7391,
+ startedAt: '2026-04-16T00:00:00Z',
+ worktreeRoot: '/project',
+ };
+ const checkedText = JSON.stringify(checkedLock);
+ const successorUnderSamePid = JSON.stringify({
+ ...checkedLock,
+ port: 7392,
+ startedAt: '2026-04-16T00:05:00Z',
+ });
+
+ function checkedState(
+ status: 'dead-pid' | 'reused-pid' | 'corrupt',
+ lockPath: string,
+ ): Extract<LockState, { text: string }> {
+ return status === 'corrupt'
+ ? { status, lockPath, text: '{ "pid": 4242, "hostname"' }
+ : { status, lockPath, lock: checkedLock, text: checkedText };
+ }
+
+ function lockFile(text: string): string {
+ root = realpathSync(mkdtempSync(join(tmpdir(), 'ok-clean-replaced-')));
+ const lockPath = join(root, 'server.lock');
+ writeFileSync(lockPath, text);
+ return lockPath;
+ }
+
+ const successors = [
+ [
+ 'another pid',
+ JSON.stringify({ ...checkedLock, pid: 4343, startedAt: '2026-04-16T00:05:00Z' }),
+ ],
+ ['the same pid', successorUnderSamePid],
+ ['nothing yet', ''],
+ ['part of its lock', successorUnderSamePid.slice(0, 20)],
+ ] as const;
+ const cases = (['dead-pid', 'reused-pid', 'corrupt'] as const).flatMap((status) =>
+ successors.map(([successor, text]) => [status, successor, text] as const),
+ );
+
+ test.each(cases)(
+ 'leaves the new lock in place when the checked lock was %s and the successor wrote %s',
+ (status, _successor, successorText) => {
+ const lockPath = lockFile(successorText);
+
+ const outcome = runClean({
+ lockDir: root ?? '',
+ inspect: () => checkedState(status, lockPath),
+ log: () => {},
+ error: () => {},
+ });
+
+ expect(readFileSync(lockPath, 'utf-8')).toBe(successorText);
+ expect(outcome.pruned).toEqual([]);
+ expect(outcome.failed.map((failure) => failure.error)).toEqual([
+ 'the lock changed after it was checked, so it was left in place',
+ ]);
+ },
+ );
+
+ test.each(['dead-pid', 'reused-pid', 'corrupt'] as const)(
+ 'removes a %s lock that still holds what was checked',
+ (status) => {
+ const lockPath = lockFile('');
+ const checked = checkedState(status, lockPath);
+ writeFileSync(lockPath, checked.text);
+
+ const outcome = runClean({
+ lockDir: root ?? '',
+ inspect: () => checked,
+ log: () => {},
+ error: () => {},
+ });
+
+ expect(outcome.failed).toEqual([]);
+ expect(outcome.pruned.map((target) => target.reason)).toEqual([status]);
+ expect(existsSync(lockPath)).toBe(false);
+ },
+ );
+
+ test('reports a lock that was removed after it was checked', () => {
+ const lockPath = lockFile(checkedText);
+ rmSync(lockPath);
+
+ const outcome = runClean({
+ lockDir: root ?? '',
+ inspect: () => checkedState('dead-pid', lockPath),
+ log: () => {},
+ error: () => {},
+ });
+
+ expect(outcome.pruned).toEqual([]);
+ expect(outcome.failed.map((failure) => failure.error)).toEqual([
+ 'the lock was removed after it was checked',
+ ]);
+ });
+});
+
describe('runClean over a lock whose recorded process has exited but is unreaped', () => {
const children: ChildProcess[] = [];
let root: string | undefined;
diff --git a/packages/cli/src/commands/clean.ts b/packages/cli/src/commands/clean.ts
index e6d8defd5..7479b5ce5 100644
--- a/packages/cli/src/commands/clean.ts
+++ b/packages/cli/src/commands/clean.ts
@@ -1,12 +1,19 @@
-import { unlinkSync } from 'node:fs';
-import { type Config, isLockProcessRunning, resolveLockDir } from '@inkeep/open-knowledge-server';
+import {
+ type Config,
+ isLockProcessRunning,
+ LOCK_FILE_ACCESS_REMEDY,
+ type LockRemoval,
+ removeLockIfUnchanged,
+ resolveLockDir,
+} from '@inkeep/open-knowledge-server';
import { Command } from 'commander';
import { describeLockOwnershipRefusal, inspectLock, type LockState } from './lock-state.ts';
interface PruneTarget {
name: 'server';
lockPath: string;
- reason: 'dead-pid' | 'corrupt';
+ reason: 'dead-pid' | 'reused-pid' | 'corrupt';
+ text: string;
}
interface CleanPlan {
@@ -14,6 +21,14 @@ interface CleanPlan {
refusal?: { lockPath: string; error: string };
}
+function pruneOnly(server: Extract<LockState, { status: PruneTarget['reason'] }>): CleanPlan {
+ return {
+ prune: [
+ { name: 'server', lockPath: server.lockPath, reason: server.status, text: server.text },
+ ],
+ };
+}
+
export function buildCleanPlan(server: LockState): CleanPlan {
switch (server.status) {
case 'read-error':
@@ -21,7 +36,7 @@ export function buildCleanPlan(server: LockState): CleanPlan {
prune: [],
refusal: {
lockPath: server.lockPath,
- error: `Cannot read the server lock: ${server.error}. Restore file and parent-directory access, then retry.`,
+ error: `Cannot read the server lock: ${server.error}. ${LOCK_FILE_ACCESS_REMEDY}`,
},
};
case 'corrupt':
@@ -34,9 +49,10 @@ export function buildCleanPlan(server: LockState): CleanPlan {
},
};
}
- return { prune: [{ name: 'server', lockPath: server.lockPath, reason: server.status }] };
+ return pruneOnly(server);
case 'dead-pid':
- return { prune: [{ name: 'server', lockPath: server.lockPath, reason: server.status }] };
+ case 'reused-pid':
+ return pruneOnly(server);
case 'unverified-owner':
case 'foreign-host':
return {
@@ -66,13 +82,26 @@ interface CleanOutcome {
failed: Array<{ target: Pick<PruneTarget, 'name' | 'lockPath'>; error: string }>;
}
+const unremovedLockReasons: Record<Exclude<LockRemoval, 'removed'>, string> = {
+ changed: 'the lock changed after it was checked, so it was left in place',
+ missing: 'the lock was removed after it was checked',
+};
+
+function removeUnlessChanged(target: PruneTarget): void {
+ const removal = removeLockIfUnchanged(target.lockPath, target.text);
+ if (removal !== 'removed') throw new Error(unremovedLockReasons[removal]);
+}
+
export function runClean(deps: RunCleanDeps): CleanOutcome {
const inspect =
deps.inspect ?? (() => inspectLock(deps.lockDir, 'server', { isAlive: isLockProcessRunning }));
- const unlink = deps.unlink ?? ((path) => unlinkSync(path));
const log = deps.log ?? ((msg) => console.log(msg));
const error = deps.error ?? ((msg) => console.error(msg));
+ const injectedUnlink = deps.unlink;
+ const unlink = injectedUnlink
+ ? (target: PruneTarget) => injectedUnlink(target.lockPath)
+ : removeUnlessChanged;
const plan = buildCleanPlan(inspect());
if (plan.refusal) {
@@ -93,7 +122,7 @@ export function runClean(deps: RunCleanDeps): CleanOutcome {
const failed: Array<{ target: Pick<PruneTarget, 'name' | 'lockPath'>; error: string }> = [];
for (const target of plan.prune) {
try {
- unlink(target.lockPath);
+ unlink(target);
pruned.push(target);
} catch (err) {
failed.push({ target, error: err instanceof Error ? err.message : String(err) });
diff --git a/packages/cli/src/commands/diagnose-health-checks/server-lock.test.ts b/packages/cli/src/commands/diagnose-health-checks/server-lock.test.ts
index a26ab19a6..1474e273a 100644
--- a/packages/cli/src/commands/diagnose-health-checks/server-lock.test.ts
+++ b/packages/cli/src/commands/diagnose-health-checks/server-lock.test.ts
@@ -1,5 +1,5 @@
import { existsSync, mkdirSync, mkdtempSync, rmSync } from 'node:fs';
-import { tmpdir } from 'node:os';
+import { hostname, tmpdir } from 'node:os';
import { join } from 'node:path';
import type { ProcessLockMetadata } from '@inkeep/open-knowledge-server';
import { afterEach, describe, expect, test, vi } from 'vitest';
@@ -89,6 +89,29 @@ describe('server-lock check', () => {
expect(result.remediation).toContain('owning machine');
});
+ test('a lock with another machine ID from this host is not called a foreign host and names the account and home folder remedies', async () => {
+ const cwd = makeProjectWithLockDir();
+ const def = makeServerLockCheck({
+ inspect: (): LockState => ({
+ status: 'foreign-host',
+ lockPath: '/x/server.lock',
+ lock: makeLockMeta({
+ hostname: hostname(),
+ machineId: '6f1c2d4e-0000-4000-8000-000000000000',
+ }),
+ }),
+ });
+ const result = await def.run({ cwd });
+ expect(result.status).toBe('warn');
+ expect(result.summary).not.toContain('foreign host');
+ expect(result.summary).toContain('records a machine ID different from the one in');
+ expect(result.remediation).not.toMatch(/^Cannot verify shutdown/);
+ expect(result.remediation).toContain(
+ 'If the server was started on this machine from another account, run `ok stop` as that account while it is running, or `ok clean` once it has exited.',
+ );
+ expect(result.remediation).toMatch(/with `(HOME|USERPROFILE)` set to that folder/);
+ });
+
test('warns on dead-pid', async () => {
const cwd = makeProjectWithLockDir();
const def = makeServerLockCheck({
@@ -96,6 +119,7 @@ describe('server-lock check', () => {
status: 'dead-pid',
lockPath: '/x/server.lock',
lock: makeLockMeta({ pid: 999999 }),
+ text: '',
}),
});
const result = await def.run({ cwd });
@@ -104,12 +128,33 @@ describe('server-lock check', () => {
expect(result.remediation).toContain('ok clean');
});
+ test('warns on a lock whose pid now belongs to another process', async () => {
+ const cwd = makeProjectWithLockDir();
+ const def = makeServerLockCheck({
+ inspect: (): LockState => ({
+ status: 'reused-pid',
+ lockPath: '/x/server.lock',
+ lock: makeLockMeta({ pid: 4242 }),
+ text: '',
+ }),
+ });
+ const result = await def.run({ cwd });
+ expect(result).toEqual({
+ name: 'server-lock',
+ status: 'warn',
+ summary: 'stale lock; recorded pid 4242 now belongs to another process',
+ remediation: 'Run `ok clean` to prune.',
+ detail: 'lockPath: /x/server.lock',
+ });
+ });
+
test('warns on corrupt lockfile', async () => {
const cwd = makeProjectWithLockDir();
const def = makeServerLockCheck({
inspect: (): LockState => ({
status: 'corrupt',
lockPath: '/x/server.lock',
+ text: '{',
}),
});
const result = await def.run({ cwd });
@@ -148,6 +193,15 @@ describe('server-lock remediation names the running channel CLI', () => {
status: 'dead-pid',
lockPath: '/x/server.lock',
lock: makeLockMeta({ pid: 999999 }),
+ text: '',
+ }),
+ ).toBe(`Run \`${command} clean\` to prune.`);
+ expect(
+ await remediationFor({
+ status: 'reused-pid',
+ lockPath: '/x/server.lock',
+ lock: makeLockMeta({ pid: 999999 }),
+ text: '',
}),
).toBe(`Run \`${command} clean\` to prune.`);
});
diff --git a/packages/cli/src/commands/diagnose-health-checks/server-lock.ts b/packages/cli/src/commands/diagnose-health-checks/server-lock.ts
index 8dd207a5f..bbe8bae0d 100644
--- a/packages/cli/src/commands/diagnose-health-checks/server-lock.ts
+++ b/packages/cli/src/commands/diagnose-health-checks/server-lock.ts
@@ -1,7 +1,13 @@
import { existsSync } from 'node:fs';
import { currentCliCommandName } from '@inkeep/open-knowledge-core';
-import { resolveLockDir } from '@inkeep/open-knowledge-server';
-import { describeLockOwnershipRefusal, inspectLock, type LockState } from '../lock-state.ts';
+import { LOCK_FILE_ACCESS_REMEDY, resolveLockDir } from '@inkeep/open-knowledge-server';
+import {
+ describeForeignHostRemedy,
+ describeForeignOwner,
+ describeLockOwnershipRefusal,
+ inspectLock,
+ type LockState,
+} from '../lock-state.ts';
import type { CheckContext, CheckDefinition, CheckResult } from './types.ts';
interface ServerLockCheckDeps {
@@ -49,9 +55,8 @@ export function makeServerLockCheck(deps: ServerLockCheckDeps = {}): CheckDefini
return {
name: 'server-lock',
status: 'warn',
- summary: `lock claimed by ${state.lock.hostname} (foreign host)`,
- remediation:
- 'Stop the server on its owning machine and confirm the directory is no longer in use before removing the lock.',
+ summary: `server lock ${describeForeignOwner(state.lock)}`,
+ remediation: describeForeignHostRemedy(state),
detail: `lockPath: ${state.lockPath}; pid: ${state.lock.pid}`,
};
case 'dead-pid':
@@ -62,12 +67,20 @@ export function makeServerLockCheck(deps: ServerLockCheckDeps = {}): CheckDefini
remediation: `Run \`${currentCliCommandName()} clean\` to prune.`,
detail: `lockPath: ${state.lockPath}`,
};
+ case 'reused-pid':
+ return {
+ name: 'server-lock',
+ status: 'warn',
+ summary: `stale lock; recorded pid ${state.lock.pid} now belongs to another process`,
+ remediation: `Run \`${currentCliCommandName()} clean\` to prune.`,
+ detail: `lockPath: ${state.lockPath}`,
+ };
case 'read-error':
return {
name: 'server-lock',
status: 'fail',
summary: 'server.lock could not be read',
- remediation: 'Restore file and parent-directory access, then retry.',
+ remediation: LOCK_FILE_ACCESS_REMEDY,
detail: `${state.lockPath}: ${state.error}`,
};
case 'corrupt':
diff --git a/packages/cli/src/commands/diagnose.test.ts b/packages/cli/src/commands/diagnose.test.ts
index 12a98c8f2..33e985ecf 100644
--- a/packages/cli/src/commands/diagnose.test.ts
+++ b/packages/cli/src/commands/diagnose.test.ts
@@ -131,6 +131,37 @@ describe('runDiagnose', () => {
expect(meta.lockInfo.lock.worktreeRoot).toBe(worktreeRoot);
});
+ test('lockInfo stays empty when the matching lock records a pid that now belongs to another process', async () => {
+ const out = makeTmpDir();
+ tmpDirs.push(out);
+ const pid = process.pid;
+ const worktreeRoot = '/tmp/restarted-content';
+
+ await runDiagnose(
+ { pid, noInspector: true, output: out },
+ {
+ ...makeBaseDeps(out),
+ discover: async () => [`${worktreeRoot}/.ok/local`],
+ inspect: () => ({
+ status: 'reused-pid',
+ lockPath: `${worktreeRoot}/.ok/local/server.lock`,
+ lock: {
+ pid,
+ hostname: 'test-host',
+ port: 5173,
+ startedAt: '2026-05-06T10:00:00.000Z',
+ worktreeRoot,
+ },
+ text: '',
+ }),
+ },
+ );
+
+ const meta = JSON.parse(readFileSync(join(out, 'metadata.json'), 'utf-8'));
+ expect(meta.pid).toBe(pid);
+ expect(meta.lockInfo).toBeNull();
+ });
+
test('output dir defaults to contentDir/.ok/local/diagnostics when lock found', async () => {
const contentDir = makeTmpDir();
tmpDirs.push(contentDir);
diff --git a/packages/cli/src/commands/diagnose.ts b/packages/cli/src/commands/diagnose.ts
index e59ec614a..fec6f4ce4 100644
--- a/packages/cli/src/commands/diagnose.ts
+++ b/packages/cli/src/commands/diagnose.ts
@@ -323,6 +323,7 @@ export async function runDiagnose(
s.status !== 'corrupt' &&
s.status !== 'read-error' &&
s.status !== 'unverified-owner' &&
+ s.status !== 'reused-pid' &&
s.lock.pid === pid
) {
contentDir = s.lock.worktreeRoot;
diff --git a/packages/cli/src/commands/removal-process-start.test.ts b/packages/cli/src/commands/holder-process-start.test.ts
similarity index 56%
rename from packages/cli/src/commands/removal-process-start.test.ts
rename to packages/cli/src/commands/holder-process-start.test.ts
index 24f1e08b9..29f89bf55 100644
--- a/packages/cli/src/commands/removal-process-start.test.ts
+++ b/packages/cli/src/commands/holder-process-start.test.ts
@@ -1,55 +1,44 @@
-import { describe, expect, test, vi } from 'vitest';
-import { readRemovalProcessStart } from './removal-process-start.ts';
+import { readProcessStart as readServerProcessStart } from '@inkeep/open-knowledge-server';
+import pino from 'pino';
+import { afterEach, describe, expect, test, vi } from 'vitest';
+import {
+ readHolderProcessStart,
+ readWindowsNativeProcessStart,
+ registerWindowsProcessStartReader,
+} from './holder-process-start.ts';
-describe('readRemovalProcessStart', () => {
- test.each(['darwin', 'linux'] as const)(
- 'reads %s process start with second precision in UTC regardless of inherited locale',
- (platform) => {
- const run = vi.fn(() => 'Tue Sep 8 20:21:34 2026\n');
- expect(
- readRemovalProcessStart(123, {
- platform,
- env: { LC_ALL: 'fr_FR.UTF-8', TZ: 'America/New_York' },
- run,
- }),
- ).toBe(Date.parse('2026-09-08T20:21:34.000Z'));
- expect(run).toHaveBeenCalledWith(
- 'ps',
- ['-p', '123', '-o', 'lstart='],
- expect.objectContaining({
- timeout: 5000,
- env: expect.objectContaining({ LC_ALL: 'C', TZ: 'UTC0' }),
- }),
- );
- },
- );
+const installedNativeBinding = vi.hoisted(() => ({ current: undefined as unknown }));
+
+vi.mock('../native/load-native-config.ts', async (importOriginal) => {
+ const actual = await importOriginal<typeof import('../native/load-native-config.ts')>();
+ return {
+ ...actual,
+ requireNativeConfigModule: (...args: Parameters<typeof actual.requireNativeConfigModule>) =>
+ installedNativeBinding.current === undefined
+ ? actual.requireNativeConfigModule(...args)
+ : installedNativeBinding.current,
+ };
+});
- test('reads Windows creation time through the native addon without launching a shell', () => {
+describe('readWindowsNativeProcessStart', () => {
+ test('reads Windows creation time through the native addon', () => {
const readProcessStart = vi.fn(() => 1_789_000_000_456);
- const run = vi.fn(() => '');
expect(
- readRemovalProcessStart(123, {
- platform: 'win32',
+ readWindowsNativeProcessStart(123, {
nativeResolver: { requireModule: () => ({ readProcessStart }) },
- run,
}),
).toBe(1_789_000_000_456);
expect(readProcessStart).toHaveBeenCalledWith(123);
- expect(run).not.toHaveBeenCalled();
});
test.each([null, {}, { readProcessStart: 'invalid' }])(
'refuses an unavailable Windows addon %j',
(binding) => {
- const run = vi.fn(() => '');
expect(
- readRemovalProcessStart(123, {
- platform: 'win32',
+ readWindowsNativeProcessStart(123, {
nativeResolver: { requireModule: () => binding },
- run,
}),
).toBeNull();
- expect(run).not.toHaveBeenCalled();
},
);
@@ -57,19 +46,16 @@ describe('readRemovalProcessStart', () => {
'refuses an invalid native process time %j',
(value) => {
expect(
- readRemovalProcessStart(123, {
- platform: 'win32',
+ readWindowsNativeProcessStart(123, {
nativeResolver: { requireModule: () => ({ readProcessStart: () => value }) },
}),
).toBeNull();
},
);
- test('refuses a Windows native query error without a shell fallback', () => {
- const run = vi.fn(() => '');
+ test('refuses a Windows native query error', () => {
expect(
- readRemovalProcessStart(123, {
- platform: 'win32',
+ readWindowsNativeProcessStart(123, {
nativeResolver: {
requireModule: () => ({
readProcessStart() {
@@ -77,53 +63,19 @@ describe('readRemovalProcessStart', () => {
},
}),
},
- run,
}),
).toBeNull();
- expect(run).not.toHaveBeenCalled();
- });
-
- test('refuses a PID outside the 32-bit process-id range before loading native code', () => {
- const requireModule = vi.fn();
- expect(
- readRemovalProcessStart(0x1_0000_0000, {
- platform: 'win32',
- nativeResolver: { requireModule },
- }),
- ).toBeNull();
- expect(requireModule).not.toHaveBeenCalled();
- });
-
- test('refuses a PID outside the 32-bit process-id range before querying ps', () => {
- const run = vi.fn(() => 'Tue Sep 8 20:21:34 2026\n');
- expect(readRemovalProcessStart(0x1_0000_0000, { platform: 'darwin', run })).toBeNull();
- expect(run).not.toHaveBeenCalled();
});
- test.each(['', 'permission denied', 'Tue Sep 8 20:21:34 2026\nTue Sep 8 20:21:34 2026'])(
- 'refuses unusable process start output %j',
- (output) => {
- expect(readRemovalProcessStart(123, { platform: 'darwin', run: () => output })).toBeNull();
+ test.each([0x1_0000_0000, 0, -1, Number.NaN, 12.5])(
+ 'refuses PID %j before loading native code',
+ (pid) => {
+ const requireModule = vi.fn();
+ expect(readWindowsNativeProcessStart(pid, { nativeResolver: { requireModule } })).toBeNull();
+ expect(requireModule).not.toHaveBeenCalled();
},
);
- test('refuses an unavailable, unsuccessful, or timed out probe', () => {
- expect(
- readRemovalProcessStart(123, {
- platform: 'darwin',
- run: () => {
- throw new Error('process query failed');
- },
- }),
- ).toBeNull();
- });
-
- test.each([0, -1, Number.NaN, 12.5])('never queries an invalid PID %j', (pid) => {
- const run = vi.fn(() => '2026-09-08T20:21:34.000Z');
- expect(readRemovalProcessStart(pid, { run })).toBeNull();
- expect(run).not.toHaveBeenCalled();
- });
-
test.each([
['unavailable', () => null, 'did not load or does not export readProcessStart'],
[
@@ -163,8 +115,7 @@ describe('readRemovalProcessStart', () => {
const onNativeFailure = vi.fn();
try {
expect(
- readRemovalProcessStart(123, {
- platform: 'win32',
+ readWindowsNativeProcessStart(123, {
nativeResolver: { requireModule },
onNativeFailure,
}),
@@ -190,8 +141,7 @@ describe('readRemovalProcessStart', () => {
const stderr = vi.spyOn(process.stderr, 'write').mockImplementation(() => true);
try {
expect(
- readRemovalProcessStart(123, {
- platform: 'win32',
+ readWindowsNativeProcessStart(123, {
arch,
nativeResolver: { requireModule: () => null },
onNativeFailure,
@@ -210,8 +160,7 @@ describe('readRemovalProcessStart', () => {
test('describes an undefined binding as absent rather than rendering the value', () => {
const onNativeFailure = vi.fn();
expect(
- readRemovalProcessStart(123, {
- platform: 'win32',
+ readWindowsNativeProcessStart(123, {
arch: 'x64',
nativeResolver: { requireModule: () => undefined },
onNativeFailure,
@@ -245,8 +194,7 @@ describe('readRemovalProcessStart', () => {
);
};
expect(
- readRemovalProcessStart(123, {
- platform: 'win32',
+ readWindowsNativeProcessStart(123, {
arch: 'x64',
nativeResolver: { requireModule },
onNativeFailure,
@@ -284,8 +232,7 @@ describe('readRemovalProcessStart', () => {
] as const)('classifies %s for %s', (kind, _name, requireModule) => {
const onNativeFailure = vi.fn();
expect(
- readRemovalProcessStart(123, {
- platform: 'win32',
+ readWindowsNativeProcessStart(123, {
arch: 'x64',
nativeResolver: { requireModule },
onNativeFailure,
@@ -295,10 +242,107 @@ describe('readRemovalProcessStart', () => {
expect(onNativeFailure.mock.calls.at(-1)?.[0]).toMatchObject({ kind });
});
- test('reads the current process start on the current operating system', () => {
- const startedAt = readRemovalProcessStart(process.pid);
- expect(startedAt).not.toBeNull();
- expect(startedAt).toBeLessThanOrEqual(Date.now());
- expect(startedAt).toBeGreaterThan(Date.now() - process.uptime() * 1000 - 10_000);
+ test.runIf(process.platform === 'win32')(
+ 'reads the current process start through the installed addon',
+ () => {
+ const startedAt = readWindowsNativeProcessStart(process.pid);
+ expect(startedAt).not.toBeNull();
+ expect(startedAt).toBeLessThanOrEqual(Date.now());
+ expect(startedAt).toBeGreaterThan(Date.now() - process.uptime() * 1000 - 10_000);
+ },
+ );
+});
+
+describe('registerWindowsProcessStartReader', () => {
+ afterEach(() => {
+ installedNativeBinding.current = undefined;
});
+
+ test('lets the server read Windows process starts through the installed addon until unregistered', () => {
+ const readProcessStart = vi.fn(() => 1_789_000_000_456);
+ installedNativeBinding.current = { readProcessStart };
+ const unregister = registerWindowsProcessStartReader();
+ try {
+ expect(readServerProcessStart(4242, { platform: 'win32' })).toEqual({
+ startedAtMs: 1_789_000_000_456,
+ clock: 'creation',
+ });
+ expect(readProcessStart).toHaveBeenCalledWith(4242);
+ } finally {
+ unregister();
+ }
+ expect(readServerProcessStart(4242, { platform: 'win32' })).toBeNull();
+ });
+
+ test('records each kind of native failure once, however often the reader is asked', () => {
+ const records: Array<Record<string, unknown>> = [];
+ installedNativeBinding.current = {
+ readProcessStart() {
+ throw new Error('OpenProcess: os error 5');
+ },
+ };
+ const unregister = registerWindowsProcessStartReader(
+ pino({ level: 'warn' }, { write: (line) => records.push(JSON.parse(line)) }),
+ );
+ try {
+ for (let read = 0; read < 3; read += 1) {
+ expect(readServerProcessStart(4242, { platform: 'win32' })).toBeNull();
+ }
+ installedNativeBinding.current = null;
+ for (let read = 0; read < 3; read += 1) {
+ expect(readServerProcessStart(4242, { platform: 'win32' })).toBeNull();
+ }
+ } finally {
+ unregister();
+ }
+ expect(records).toEqual([
+ expect.objectContaining({
+ event: 'process-start-native-failure',
+ pid: process.pid,
+ lockPid: 4242,
+ kind: 'query-failed',
+ reasons: [expect.stringContaining('OpenProcess: os error 5')],
+ }),
+ expect.objectContaining({
+ event: 'process-start-native-failure',
+ pid: process.pid,
+ lockPid: 4242,
+ kind: 'unavailable',
+ }),
+ ]);
+ });
+});
+
+describe('readHolderProcessStart', () => {
+ afterEach(() => {
+ installedNativeBinding.current = undefined;
+ });
+
+ test('reads a Windows holder through the installed addon as a creation time', () => {
+ const readProcessStart = vi.fn(() => 1_789_000_000_456);
+ installedNativeBinding.current = { readProcessStart };
+ expect(readHolderProcessStart(4242, { platform: 'win32' })).toEqual({
+ startedAtMs: 1_789_000_000_456,
+ clock: 'creation',
+ });
+ expect(readProcessStart).toHaveBeenCalledWith(4242);
+ });
+
+ test('reports why the Windows addon could not read the holder', () => {
+ installedNativeBinding.current = {
+ readProcessStart() {
+ throw new Error('OpenProcess: os error 5');
+ },
+ };
+ const onNativeFailure = vi.fn();
+ expect(readHolderProcessStart(4242, { platform: 'win32', onNativeFailure })).toBeNull();
+ expect(onNativeFailure).toHaveBeenCalledWith(expect.objectContaining({ kind: 'query-failed' }));
+ });
+
+ test.skipIf(process.platform === 'win32')(
+ 'reads a holder on this host the way the server records it in a lock',
+ () => {
+ expect(readHolderProcessStart(process.pid)).toEqual(readServerProcessStart(process.pid));
+ },
+ );
});
diff --git a/packages/cli/src/commands/holder-process-start.ts b/packages/cli/src/commands/holder-process-start.ts
new file mode 100644
index 000000000..7aa5b8953
--- /dev/null
+++ b/packages/cli/src/commands/holder-process-start.ts
@@ -0,0 +1,176 @@
+import { okUserHomeDisplayPath } from '@inkeep/open-knowledge-core';
+import {
+ type ProcessStart,
+ readProcessStart,
+ registerNativeProcessStartReader,
+} from '@inkeep/open-knowledge-server';
+import { getCliLogger } from '../cli-logger.ts';
+import {
+ debugNativeLoadFailure,
+ describeNativeFailure,
+ describeValue,
+ type NativeConfigResolver,
+ requireNativeConfigModule,
+} from '../native/load-native-config.ts';
+
+const MAX_PROCESS_ID = 0xffff_ffff;
+
+type NativeFailureKind = 'unavailable' | 'query-failed';
+
+export interface NativeFailure {
+ kind: NativeFailureKind;
+ reason: string;
+}
+
+export interface WindowsNativeProcessStartOptions {
+ arch?: NodeJS.Architecture;
+ nativeResolver?: Partial<NativeConfigResolver>;
+ onNativeFailure?: (failure: NativeFailure) => void;
+}
+
+export interface HolderProcessStartOptions {
+ platform?: NodeJS.Platform;
+ onNativeFailure?: (failure: NativeFailure) => void;
+}
+
+type NativeFailureLogRecord = {
+ event: 'process-start-native-failure';
+ lockPid: number;
+ kind: NativeFailureKind;
+ reasons: string[];
+};
+
+interface NativeFailureLog {
+ warn(data: NativeFailureLogRecord, message: string): void;
+}
+
+function describeBinding(binding: unknown): string {
+ if (!binding || typeof binding !== 'object') return String(binding);
+ const exported = typeof (binding as Record<string, unknown>).readProcessStart;
+ return `${describeValue(binding)} readProcessStart=${exported}`;
+}
+
+export function readWindowsNativeProcessStart(
+ pid: number,
+ options: WindowsNativeProcessStartOptions = {},
+): number | null {
+ if (!Number.isSafeInteger(pid) || pid <= 0 || pid > MAX_PROCESS_ID) return null;
+ const arch = options.arch ?? process.arch;
+ const reportNativeFailure = (kind: NativeFailureKind, context: string, detail: unknown): void => {
+ debugNativeLoadFailure(context, detail);
+ options.onNativeFailure?.({ kind, reason: describeNativeFailure(context, detail) });
+ };
+ try {
+ let loadFailureReported = false;
+ const binding = requireNativeConfigModule(options.nativeResolver, (context, detail) => {
+ loadFailureReported = true;
+ reportNativeFailure('unavailable', context, detail);
+ });
+ if (
+ !binding ||
+ typeof binding !== 'object' ||
+ !('readProcessStart' in binding) ||
+ typeof binding.readProcessStart !== 'function'
+ ) {
+ let detail: string;
+ if (binding !== null && binding !== undefined) detail = describeBinding(binding);
+ else if (loadFailureReported) detail = 'no usable binding was returned';
+ else detail = 'the addon was not found';
+ reportNativeFailure(
+ 'unavailable',
+ `the Windows native addon (native-config.win32-${arch}-msvc.node) did not load or does not export readProcessStart`,
+ detail,
+ );
+ return null;
+ }
+ const startedAt: unknown = binding.readProcessStart(pid);
+ if (typeof startedAt === 'number' && Number.isSafeInteger(startedAt) && startedAt >= 0)
+ return startedAt;
+ reportNativeFailure(
+ 'unavailable',
+ 'readProcessStart returned an invalid value',
+ describeValue(startedAt),
+ );
+ return null;
+ } catch (err) {
+ reportNativeFailure('query-failed', 'readProcessStart failed', err);
+ return null;
+ }
+}
+
+export function readHolderProcessStart(
+ pid: number,
+ options: HolderProcessStartOptions = {},
+): ProcessStart | null {
+ return readProcessStart(pid, {
+ platform: options.platform,
+ readNative: (nativePid) =>
+ readWindowsNativeProcessStart(nativePid, { onNativeFailure: options.onNativeFailure }),
+ });
+}
+
+export function describeHolderStartReadFailure(args: {
+ platform: NodeJS.Platform;
+ pid: number;
+ nativeFailures: readonly NativeFailure[];
+ causeRecorded: boolean;
+}): string {
+ const { platform, pid, nativeFailures, causeRecorded } = args;
+ const componentUnavailable = nativeFailures.some((failure) => {
+ switch (failure.kind) {
+ case 'unavailable':
+ return true;
+ case 'query-failed':
+ return false;
+ default: {
+ const exhaustive: never = failure.kind;
+ return exhaustive;
+ }
+ }
+ });
+ const windowsRecovery =
+ nativeFailures.length === 0
+ ? ''
+ : componentUnavailable
+ ? 'Reinstall OpenKnowledge to restore the Windows component that verifies process identity. '
+ : 'The Windows component loaded but the operating system refused the query, so reinstalling will not help; retry from an account that can inspect that process. ';
+ const probeRecovery =
+ platform === 'linux'
+ ? `Linux reads the process start from /proc/${pid}/stat and /proc/stat; restore read access to /proc and retry. `
+ : platform === 'darwin'
+ ? 'macOS reads the process start with ps, and that read failed, as it does inside some sandboxes; retry from a shell where ps can run. '
+ : platform === 'win32'
+ ? windowsRecovery
+ : '';
+ const causeRecord =
+ nativeFailures.length > 0 && causeRecorded
+ ? `The failure detail is recorded in the CLI log under ${okUserHomeDisplayPath('logs')}. `
+ : '';
+ return probeRecovery + causeRecord;
+}
+
+export function registerWindowsProcessStartReader(log?: NativeFailureLog): () => void {
+ const recordedKinds = new Set<NativeFailureKind>();
+ return registerNativeProcessStartReader((pid) => {
+ const failures: NativeFailure[] = [];
+ const startedAt = readWindowsNativeProcessStart(pid, {
+ onNativeFailure: (failure) => {
+ failures.push(failure);
+ },
+ });
+ for (const kind of new Set(failures.map((failure) => failure.kind))) {
+ if (recordedKinds.has(kind)) continue;
+ recordedKinds.add(kind);
+ (log ?? getCliLogger())?.warn(
+ {
+ event: 'process-start-native-failure',
+ lockPid: pid,
+ kind,
+ reasons: failures.filter((failure) => failure.kind === kind).map((f) => f.reason),
+ },
+ 'the Windows process-start query failed, so lock holders cannot be verified; later failures of this kind are not recorded again',
+ );
+ }
+ return startedAt;
+ });
+}
diff --git a/packages/cli/src/commands/lock-pid-reuse.test.ts b/packages/cli/src/commands/lock-pid-reuse.test.ts
new file mode 100644
index 000000000..a26083027
--- /dev/null
+++ b/packages/cli/src/commands/lock-pid-reuse.test.ts
@@ -0,0 +1,283 @@
+import { randomUUID } from 'node:crypto';
+import { existsSync, mkdirSync, mkdtempSync, realpathSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import {
+ getMachineId,
+ lockFilePath,
+ type ProcessLockMetadata,
+ resolveLockDir,
+} from '@inkeep/open-knowledge-server';
+import { afterEach, describe, expect, test } from 'vitest';
+import {
+ installSignalBoundary,
+ type SignalBoundary,
+} from '../../../../test-support/held-signal-boundary.test-helper.ts';
+import {
+ legacyServerLock,
+ lockWrittenBeforeProcessStarted,
+ lockWrittenByRunningProcess,
+ type OwnedProcess,
+ ownedProcessHasExited,
+ startOwnedProcess,
+ stopOwnedProcess,
+} from '../../../../test-support/owned-process.test-helper.ts';
+import { runClean } from './clean.ts';
+import { makeServerLockCheck } from './diagnose-health-checks/server-lock.ts';
+import { inspectLock } from './lock-state.ts';
+import { runPs } from './ps.ts';
+import { runStatus } from './status.ts';
+import { runStop } from './stop.ts';
+
+const owned: OwnedProcess[] = [];
+const roots: string[] = [];
+let boundary: SignalBoundary | undefined;
+
+afterEach(async () => {
+ boundary?.restore();
+ boundary = undefined;
+ for (const holder of owned.splice(0)) await stopOwnedProcess(holder);
+ for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true });
+});
+
+async function seedServerLock(
+ startedAt: (holder: OwnedProcess) => string,
+ lockOverrides: Record<string, unknown> = {},
+): Promise<{
+ root: string;
+ lockDir: string;
+ lockPath: string;
+ lock: ProcessLockMetadata;
+ holder: OwnedProcess;
+}> {
+ const holder = await startOwnedProcess();
+ owned.push(holder);
+ const root = realpathSync(mkdtempSync(join(tmpdir(), 'ok-lock-pid-reuse-')));
+ roots.push(root);
+ const lockDir = join(root, '.ok', 'local');
+ mkdirSync(lockDir, { recursive: true });
+ const lockPath = lockFilePath(lockDir, 'server');
+ const serialized = JSON.stringify({
+ ...legacyServerLock({
+ pid: holder.pid,
+ startedAt: startedAt(holder),
+ machineId: getMachineId(),
+ worktreeRoot: root,
+ }),
+ ...lockOverrides,
+ });
+ writeFileSync(lockPath, serialized, { mode: 0o600 });
+ return { root, lockDir, lockPath, lock: JSON.parse(serialized), holder };
+}
+
+async function psEntries(lockDir: string, holderPid: number): Promise<unknown> {
+ const printed: string[] = [];
+ await runPs({
+ discover: async () => [lockDir],
+ resolveCommand: (pid) => (pid === holderPid ? `owned-process ${pid}` : null),
+ resolveUsage: (pid) => (pid === holderPid ? { cpuPercent: 1, memPercent: 2 } : null),
+ json: true,
+ log: (msg) => printed.push(msg),
+ });
+ return JSON.parse(printed.join('\n'));
+}
+
+function signalsSentTo(pid: number): Array<{ target: number; signal: unknown }> {
+ if (boundary === undefined) throw new Error('no signal boundary installed');
+ return [...boundary.delivered, ...boundary.refused].filter((record) => record.target === pid);
+}
+
+describe('a server lock whose pid now belongs to a process started after the lock was written', () => {
+ test('inspectLock does not report the server alive', async () => {
+ const { lockDir } = await seedServerLock(lockWrittenBeforeProcessStarted);
+
+ expect(inspectLock(lockDir, 'server').status).not.toBe('alive');
+ });
+
+ test('ok status does not report the server alive', async () => {
+ const { lockDir } = await seedServerLock(lockWrittenBeforeProcessStarted);
+
+ const report = runStatus({ lockDir, log: () => {} });
+
+ expect(report.server.state).not.toBe('alive');
+ expect(report.server.alive).toBe(false);
+ });
+
+ test('ok clean prunes the lock', async () => {
+ const { lockDir, lockPath, holder } = await seedServerLock(lockWrittenBeforeProcessStarted);
+
+ const outcome = runClean({ lockDir, log: () => {}, error: () => {} });
+
+ expect(outcome.failed).toEqual([]);
+ expect(outcome.pruned.map((target) => target.lockPath)).toEqual([lockPath]);
+ expect(existsSync(lockPath)).toBe(false);
+ expect(ownedProcessHasExited(holder)).toBe(false);
+ });
+
+ test('ok stop signals nothing to the process now holding the pid', async () => {
+ const { lockDir, holder } = await seedServerLock(lockWrittenBeforeProcessStarted);
+ boundary = installSignalBoundary({ deliverToHeldChildren: false });
+
+ const outcome = await runStop({
+ lockDir,
+ log: () => {},
+ error: () => {},
+ probeClients: async () => null,
+ });
+
+ expect(signalsSentTo(holder.pid).filter((record) => record.signal !== 0)).toEqual([]);
+ expect(outcome.stopped).toEqual([]);
+ expect(ownedProcessHasExited(holder)).toBe(false);
+ });
+
+ test('ok ps reports the lock stale and attributes no process to it', async () => {
+ const { lockDir, lockPath, holder } = await seedServerLock(lockWrittenBeforeProcessStarted);
+
+ expect(await psEntries(lockDir, holder.pid)).toEqual([
+ expect.objectContaining({
+ lockPath,
+ displayStatus: 'stale',
+ command: null,
+ ui: null,
+ server: expect.objectContaining({ usage: null }),
+ }),
+ ]);
+ });
+
+ test('ok diagnose warns about a stale lock and points at ok clean', async () => {
+ const { root, lockDir } = await seedServerLock(lockWrittenBeforeProcessStarted);
+ expect(resolveLockDir(root)).toBe(lockDir);
+
+ const result = await makeServerLockCheck().run({ cwd: root });
+
+ expect(result).toMatchObject({ name: 'server-lock', status: 'warn' });
+ expect(result.remediation).toContain('ok clean');
+ });
+
+ test('ok stop signals nothing when the reading it was handed still showed the lock live', async () => {
+ const { lockDir, lockPath, lock, holder } = await seedServerLock(
+ lockWrittenBeforeProcessStarted,
+ );
+ boundary = installSignalBoundary({ deliverToHeldChildren: false });
+
+ const outcome = await runStop({
+ lockDir,
+ inspect: () => ({ status: 'alive', lockPath, lock }),
+ force: true,
+ log: () => {},
+ error: () => {},
+ });
+
+ expect(signalsSentTo(holder.pid).filter((record) => record.signal !== 0)).toEqual([]);
+ expect(outcome.stopped).toEqual([]);
+ expect(ownedProcessHasExited(holder)).toBe(false);
+ });
+});
+
+describe('a server lock from another machine whose pid is a local process started after the lock was written', () => {
+ test('ok stop signals nothing to that local process', async () => {
+ const { lockDir, holder } = await seedServerLock(lockWrittenBeforeProcessStarted, {
+ machineId: randomUUID(),
+ hostname: 'another-machine.example',
+ });
+ boundary = installSignalBoundary({ deliverToHeldChildren: false });
+
+ const outcome = await runStop({
+ lockDir,
+ log: () => {},
+ error: () => {},
+ probeClients: async () => null,
+ });
+
+ expect(signalsSentTo(holder.pid).filter((record) => record.signal !== 0)).toEqual([]);
+ expect(outcome.stopped).toEqual([]);
+ expect(ownedProcessHasExited(holder)).toBe(false);
+ });
+});
+
+describe('a server lock held by the live process that wrote it', () => {
+ test('inspectLock reports the server alive', async () => {
+ const { lockDir, holder } = await seedServerLock(lockWrittenByRunningProcess);
+
+ expect(inspectLock(lockDir, 'server')).toMatchObject({
+ status: 'alive',
+ lock: { pid: holder.pid },
+ });
+ });
+
+ test('ok status reports the server alive', async () => {
+ const { lockDir, holder } = await seedServerLock(lockWrittenByRunningProcess);
+
+ const report = runStatus({ lockDir, log: () => {} });
+
+ expect(report.server).toMatchObject({ state: 'alive', alive: true, pid: holder.pid });
+ });
+
+ test('ok clean leaves the lock in place', async () => {
+ const { lockDir, lockPath } = await seedServerLock(lockWrittenByRunningProcess);
+
+ const outcome = runClean({ lockDir, log: () => {}, error: () => {} });
+
+ expect(outcome).toEqual({ pruned: [], failed: [] });
+ expect(existsSync(lockPath)).toBe(true);
+ });
+
+ test('ok stop sends SIGTERM to the writer', async () => {
+ const { lockDir, holder } = await seedServerLock(lockWrittenByRunningProcess);
+ boundary = installSignalBoundary({ deliverToHeldChildren: true });
+ boundary.hold(holder.child);
+
+ const outcome = await runStop({
+ lockDir,
+ log: () => {},
+ error: () => {},
+ probeClients: async () => null,
+ });
+
+ expect(boundary.delivered).toContainEqual({ target: holder.pid, signal: 'SIGTERM' });
+ expect(outcome.stopped).toEqual([expect.objectContaining({ pid: holder.pid })]);
+ });
+
+ test('ok ps reports the writer running', async () => {
+ const { lockDir, lockPath, holder } = await seedServerLock(lockWrittenByRunningProcess);
+
+ expect(await psEntries(lockDir, holder.pid)).toEqual([
+ expect.objectContaining({
+ lockPath,
+ displayStatus: 'running',
+ command: `owned-process ${holder.pid}`,
+ server: expect.objectContaining({
+ pid: holder.pid,
+ usage: { cpuPercent: 1, memPercent: 2 },
+ }),
+ }),
+ ]);
+ });
+
+ test('ok diagnose reports the lock held by the writer', async () => {
+ const { root, lockDir, holder } = await seedServerLock(lockWrittenByRunningProcess);
+ expect(resolveLockDir(root)).toBe(lockDir);
+
+ const result = await makeServerLockCheck().run({ cwd: root });
+
+ expect(result).toMatchObject({ name: 'server-lock', status: 'fail' });
+ expect(result.summary).toContain(`pid ${holder.pid}`);
+ });
+
+ test('ok stop sends SIGTERM to the writer when handed a reading of its lock', async () => {
+ const { lockDir, lockPath, lock, holder } = await seedServerLock(lockWrittenByRunningProcess);
+ boundary = installSignalBoundary({ deliverToHeldChildren: true });
+ boundary.hold(holder.child);
+
+ const outcome = await runStop({
+ lockDir,
+ inspect: () => ({ status: 'alive', lockPath, lock }),
+ force: true,
+ log: () => {},
+ error: () => {},
+ });
+
+ expect(boundary.delivered).toContainEqual({ target: holder.pid, signal: 'SIGTERM' });
+ expect(outcome.stopped).toEqual([expect.objectContaining({ pid: holder.pid })]);
+ });
+});
diff --git a/packages/cli/src/commands/lock-state.test.ts b/packages/cli/src/commands/lock-state.test.ts
index 7f880ff8f..0586b39c1 100644
--- a/packages/cli/src/commands/lock-state.test.ts
+++ b/packages/cli/src/commands/lock-state.test.ts
@@ -1,19 +1,20 @@
-import { chmodSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
-import { hostname, tmpdir } from 'node:os';
+import { chmodSync, writeFileSync } from 'node:fs';
+import { hostname } from 'node:os';
import { join } from 'node:path';
-import { afterEach, describe, expect, test } from 'vitest';
-import { inspectLock } from './lock-state.ts';
+import { machineIdFilePath, type ProcessStart } from '@inkeep/open-knowledge-server';
+import { afterEach, describe, expect, test, vi } from 'vitest';
+import { createTempDirFactory } from '../../../../test-support/temp-dir.test-helper.ts';
+import { describeLockOwnershipRefusal, inspectLock } from './lock-state.ts';
-const fixtures: string[] = [];
-afterEach(() => {
- for (const dir of fixtures.splice(0)) rmSync(dir, { recursive: true, force: true });
+const startedBeforeFixtureLocks = (): ProcessStart => ({
+ startedAtMs: Date.parse('2026-04-15T00:00:00Z'),
+ clock: 'creation',
});
+const tempDir = createTempDirFactory(afterEach);
+
function freshLockDir(): string {
- const dir = mkdtempSync(join(tmpdir(), 'ok-lock-state-'));
- fixtures.push(dir);
- mkdirSync(dir, { recursive: true });
- return dir;
+ return tempDir('ok-lock-state-');
}
describe('inspectLock', () => {
@@ -52,7 +53,10 @@ describe('inspectLock', () => {
test('recovers a dead PID-only lock without inventing foreign ownership', () => {
const dir = freshLockDir();
writeFileSync(join(dir, 'server.lock'), JSON.stringify({ pid: 4242 }));
- expect(inspectLock(dir, 'server', { isAlive: () => false }).status).toBe('dead-pid');
+ expect(inspectLock(dir, 'server', { isAlive: () => false })).toMatchObject({
+ status: 'dead-pid',
+ text: JSON.stringify({ pid: 4242 }),
+ });
});
test('missing lock file', () => {
@@ -65,16 +69,21 @@ describe('inspectLock', () => {
test('corrupt JSON', () => {
const dir = freshLockDir();
writeFileSync(join(dir, 'server.lock'), 'not-json{{{', 'utf-8');
- const result = inspectLock(dir, 'server');
- expect(result.status).toBe('corrupt');
- expect(result.lockPath.endsWith('/server.lock')).toBe(true);
+ expect(inspectLock(dir, 'server')).toEqual({
+ status: 'corrupt',
+ lockPath: join(dir, 'server.lock'),
+ text: 'not-json{{{',
+ });
});
test('valid JSON but missing pid is treated as corrupt', () => {
const dir = freshLockDir();
writeFileSync(join(dir, 'server.lock'), JSON.stringify({ port: 3000 }), 'utf-8');
- const result = inspectLock(dir, 'server');
- expect(result.status).toBe('corrupt');
+ expect(inspectLock(dir, 'server')).toEqual({
+ status: 'corrupt',
+ lockPath: join(dir, 'server.lock'),
+ text: JSON.stringify({ port: 3000 }),
+ });
});
test('foreign host with locally-live PID classifies foreign-host', () => {
@@ -139,6 +148,7 @@ describe('inspectLock', () => {
machineId: 'machine-A',
host: 'current-hostname',
isAlive: () => true,
+ readProcessStart: startedBeforeFixtureLocks,
});
expect(result.status).toBe('alive');
});
@@ -198,7 +208,10 @@ describe('inspectLock', () => {
}),
'utf-8',
);
- const result = inspectLock(dir, 'server', { isAlive: (pid) => pid === 4242 });
+ const result = inspectLock(dir, 'server', {
+ isAlive: (pid) => pid === 4242,
+ readProcessStart: startedBeforeFixtureLocks,
+ });
expect(result.status).toBe('alive');
if (result.status === 'alive') {
expect(result.lock.pid).toBe(4242);
@@ -206,6 +219,87 @@ describe('inspectLock', () => {
}
});
+ function writeSameHostLock(dir: string, overrides: Record<string, unknown> = {}) {
+ const lock = {
+ pid: 4242,
+ hostname: hostname(),
+ port: 52831,
+ startedAt: '2026-04-16T00:00:00Z',
+ worktreeRoot: '/x',
+ ...overrides,
+ };
+ writeFileSync(join(dir, 'server.lock'), JSON.stringify(lock), 'utf-8');
+ return lock;
+ }
+
+ test('a live pid whose holder started well after the lock was written is a reused pid', () => {
+ const dir = freshLockDir();
+ const lock = writeSameHostLock(dir);
+ expect(
+ inspectLock(dir, 'server', {
+ isAlive: () => true,
+ readProcessStart: () => ({
+ startedAtMs: Date.parse('2026-04-17T12:00:00Z'),
+ clock: 'creation',
+ }),
+ }),
+ ).toEqual({
+ status: 'reused-pid',
+ lockPath: join(dir, 'server.lock'),
+ lock,
+ text: JSON.stringify(lock),
+ });
+ });
+
+ test.each([
+ ['started before the lock was written', startedBeforeFixtureLocks],
+ ['cannot be identified', () => null],
+ ] as const)('a live pid whose holder %s stays alive', (_case, readProcessStart) => {
+ const dir = freshLockDir();
+ const lock = writeSameHostLock(dir);
+ expect(inspectLock(dir, 'server', { isAlive: () => true, readProcessStart })).toEqual({
+ status: 'alive',
+ lockPath: join(dir, 'server.lock'),
+ lock,
+ });
+ });
+
+ test('a recorded process start naming another process decides over the lock start time', () => {
+ const dir = freshLockDir();
+ const lock = writeSameHostLock(dir, {
+ processStart: { startedAtMs: Date.parse('2026-04-15T00:00:00Z') },
+ });
+ expect(
+ inspectLock(dir, 'server', {
+ isAlive: () => true,
+ readProcessStart: () => ({
+ startedAtMs: Date.parse('2026-04-16T00:00:01Z'),
+ clock: 'creation',
+ }),
+ }),
+ ).toEqual({
+ status: 'reused-pid',
+ lockPath: join(dir, 'server.lock'),
+ lock,
+ text: JSON.stringify(lock),
+ });
+ });
+
+ test('a holder that exits while its start is read is a dead pid', () => {
+ const dir = freshLockDir();
+ writeFileSync(
+ join(dir, 'server.lock'),
+ JSON.stringify({ pid: 4242, hostname: hostname(), startedAt: '2026-04-16T00:00:00Z' }),
+ );
+ const liveness = [true, false];
+ expect(
+ inspectLock(dir, 'server', {
+ isAlive: () => liveness.shift() ?? false,
+ readProcessStart: () => null,
+ }).status,
+ ).toBe('dead-pid');
+ });
+
test('peeks do not mutate filesystem (dead pid lock remains for ok clean)', () => {
const dir = freshLockDir();
writeFileSync(
@@ -243,3 +337,110 @@ test.skipIf(process.platform === 'win32' || process.getuid?.() === 0)(
}
},
);
+
+describe('describeLockOwnershipRefusal', () => {
+ const lockPath = '/tmp/x/server.lock';
+ const lock = {
+ pid: 4242,
+ port: 3000,
+ startedAt: '2026-04-16T00:00:00Z',
+ worktreeRoot: '/tmp/x',
+ };
+ const keepTheLockUntilConfirmed =
+ 'Confirm that no OpenKnowledge server is using this directory on the owning machine or any other machine sharing it. ' +
+ `Only then remove the stale lock file at ${lockPath} and retry cleanup.`;
+
+ test('a lock with another machine ID names the account and home folder remedies before removing the lock', () => {
+ expect(
+ describeLockOwnershipRefusal(
+ {
+ status: 'foreign-host',
+ lockPath,
+ lock: {
+ ...lock,
+ hostname: hostname(),
+ machineId: '6f1c2d4e-0000-4000-8000-000000000000',
+ },
+ },
+ 'darwin',
+ ),
+ ).toBe(
+ `Cannot verify shutdown: the server lock at ${lockPath} records a machine ID different from the one in ${machineIdFilePath()}. ` +
+ 'If the server was started on this machine from another account, run `ok stop` as that account while it is running, or `ok clean` once it has exited. ' +
+ 'If it was started from this account with a different home folder, run the same command with `HOME` set to that folder. ' +
+ 'Otherwise run the same command on the machine that owns the lock. ' +
+ 'This lock records no process start; a server started by this release records one, which lets `ok stop` confirm it from any home folder. ' +
+ keepTheLockUntilConfirmed,
+ );
+ });
+
+ test('a lock naming a host with control characters reaches the refusal escaped', () => {
+ const text = describeLockOwnershipRefusal(
+ {
+ status: 'foreign-host',
+ lockPath,
+ lock: { ...lock, hostname: 'evil\nRun: ok clean --all\u001b]52;c;Zm9v\u0007' },
+ },
+ 'darwin',
+ );
+
+ expect(text).toContain(
+ 'records only the host evil\\u000aRun: ok clean --all\\u001b]52;c;Zm9v\\u0007,',
+ );
+ expect(text).toContain(
+ 'If evil\\u000aRun: ok clean --all\\u001b]52;c;Zm9v\\u0007 is another machine',
+ );
+ expect(text).not.toContain('\n');
+ expect(text).not.toContain('\u001b');
+ expect(text).not.toContain('\u0007');
+ });
+
+ test('a Beta CLI names its own stop command in the account and home folder remedies', () => {
+ vi.stubEnv('OK_CHANNEL', 'beta');
+ try {
+ const text = describeLockOwnershipRefusal(
+ {
+ status: 'foreign-host',
+ lockPath,
+ lock: {
+ ...lock,
+ hostname: hostname(),
+ machineId: '6f1c2d4e-0000-4000-8000-000000000000',
+ },
+ },
+ 'darwin',
+ );
+ expect(text).toContain(
+ 'run `ok-beta stop` as that account while it is running, or `ok-beta clean` once it has exited',
+ );
+ expect(text).toContain('which lets `ok-beta stop` confirm it');
+ expect(text).not.toMatch(/`ok (stop|clean)/);
+ } finally {
+ vi.unstubAllEnvs();
+ }
+ });
+
+ test('a lock with only another host name allows for a renamed machine', () => {
+ expect(
+ describeLockOwnershipRefusal({
+ status: 'foreign-host',
+ lockPath,
+ lock: { ...lock, hostname: 'other-host' },
+ }),
+ ).toBe(
+ `Cannot verify shutdown: the server lock at ${lockPath} records only the host other-host, not this machine's ${hostname()}. ` +
+ 'If other-host is another machine, stop OpenKnowledge there; if it is an earlier name of this machine, stop OpenKnowledge here if it is still running. ' +
+ keepTheLockUntilConfirmed,
+ );
+ });
+
+ test('a malformed lock naming another owner does not claim the owner is another machine', () => {
+ expect(
+ describeLockOwnershipRefusal({ status: 'corrupt', lockPath, text: '{', foreignHost: true }),
+ ).toBe(
+ `Cannot verify shutdown: the server lock at ${lockPath} records a machine ID or host name that is not this machine's, so its server may run on another machine, or on this one from another account, with a different home folder, or under an earlier host name. ` +
+ 'Stop OpenKnowledge wherever it was started. ' +
+ keepTheLockUntilConfirmed,
+ );
+ });
+});
diff --git a/packages/cli/src/commands/lock-state.ts b/packages/cli/src/commands/lock-state.ts
index 3dac3c692..71caaac60 100644
--- a/packages/cli/src/commands/lock-state.ts
+++ b/packages/cli/src/commands/lock-state.ts
@@ -1,17 +1,24 @@
import { readFileSync } from 'node:fs';
import { hostname } from 'node:os';
+import { currentCliCommandName } from '@inkeep/open-knowledge-core';
import {
- getMachineId,
- isProcessAlive,
+ assessLockHolder,
+ isLockFromThisMachine,
isValidLockPid,
type LockName,
+ type LockOwnerFields,
lockFilePath,
+ machineIdFilePath,
type ProcessLockMetadata,
+ type ProcessStartReader,
+ recordsLockOwner,
+ toProcessLockMetadata,
} from '@inkeep/open-knowledge-server';
+import { escapeDisplayPath } from '../utils/escape-display-path.ts';
export type LockState =
| { status: 'missing'; lockPath: string }
- | { status: 'corrupt'; lockPath: string; foreignHost?: boolean }
+ | { status: 'corrupt'; lockPath: string; text: string; foreignHost?: boolean }
| { status: 'read-error'; lockPath: string; error: string }
| {
status: 'foreign-host';
@@ -19,11 +26,13 @@ export type LockState =
lock: ProcessLockMetadata;
}
| { status: 'unverified-owner'; lockPath: string; pid: number }
- | { status: 'dead-pid'; lockPath: string; lock: ProcessLockMetadata }
+ | { status: 'dead-pid'; lockPath: string; lock: ProcessLockMetadata; text: string }
+ | { status: 'reused-pid'; lockPath: string; lock: ProcessLockMetadata; text: string }
| { status: 'alive'; lockPath: string; lock: ProcessLockMetadata };
interface InspectLockOptions {
isAlive?: (pid: number) => boolean;
+ readProcessStart?: ProcessStartReader;
host?: string;
machineId?: string;
}
@@ -51,46 +60,127 @@ export function inspectLock(
try {
parsed = JSON.parse(raw);
} catch {
- return { status: 'corrupt', lockPath };
+ return { status: 'corrupt', lockPath, text: raw };
}
- const metadata = parsed && typeof parsed === 'object' ? parsed : {};
- const foreignHost =
- 'machineId' in metadata && typeof metadata.machineId === 'string'
- ? metadata.machineId !== (opts.machineId ?? getMachineId())
- : 'hostname' in metadata &&
- typeof metadata.hostname === 'string' &&
- metadata.hostname !== (opts.host ?? hostname());
+ const metadata: LockOwnerFields = parsed && typeof parsed === 'object' ? parsed : {};
+ const hasOwner = recordsLockOwner(metadata);
+ const foreignHost = hasOwner && !isLockFromThisMachine(metadata, opts);
if (!parsed || typeof parsed !== 'object' || !isValidLockPid((parsed as { pid?: unknown }).pid)) {
- return { status: 'corrupt', lockPath, ...(foreignHost ? { foreignHost: true } : {}) };
+ return {
+ status: 'corrupt',
+ lockPath,
+ text: raw,
+ ...(foreignHost ? { foreignHost: true } : {}),
+ };
}
- const pid = (parsed as { pid: number }).pid;
- const hasOwner =
- ('machineId' in metadata && typeof metadata.machineId === 'string') ||
- ('hostname' in metadata && typeof metadata.hostname === 'string');
+ const lock = toProcessLockMetadata(parsed as { pid: number });
+ const { pid } = lock;
if (foreignHost) {
- return { status: 'foreign-host', lockPath, lock: parsed as ProcessLockMetadata };
+ return { status: 'foreign-host', lockPath, lock };
}
- if (!(opts.isAlive ?? isProcessAlive)(pid)) {
- return { status: 'dead-pid', lockPath, lock: parsed as ProcessLockMetadata };
+ const assessment = assessLockHolder(lock, {
+ isAlive: opts.isAlive,
+ readProcessStart: opts.readProcessStart,
+ sameMachine: hasOwner,
+ });
+ switch (assessment.verdict) {
+ case 'gone':
+ return { status: 'dead-pid', lockPath, lock, text: raw };
+ case 'replaced':
+ return { status: 'reused-pid', lockPath, lock, text: raw };
+ case 'unverified':
+ return assessment.cause === 'unverified-owner'
+ ? { status: 'unverified-owner', lockPath, pid }
+ : { status: 'alive', lockPath, lock };
+ case 'writer':
+ return { status: 'alive', lockPath, lock };
+ default: {
+ const exhaustive: never = assessment;
+ return exhaustive;
+ }
}
- if (!hasOwner) return { status: 'unverified-owner', lockPath, pid };
- return { status: 'alive', lockPath, lock: parsed as ProcessLockMetadata };
+}
+
+export function describeUnrecordedOwner(pid: number): string {
+ return `does not record its owning machine, and its recorded process ${pid} is still running on this machine`;
+}
+
+export function describeForeignOwner(lock: ProcessLockMetadata): string {
+ return typeof lock.machineId === 'string'
+ ? `records a machine ID different from the one in ${machineIdFilePath()}`
+ : `records only the host ${escapeDisplayPath(lock.hostname)}, not this machine's ${hostname()}`;
+}
+
+function homeFolderVariable(platform: NodeJS.Platform): string {
+ return platform === 'win32' ? 'USERPROFILE' : 'HOME';
+}
+
+function describeMissingProcessStart(lock: ProcessLockMetadata): string {
+ return lock.processStart === undefined
+ ? ` This lock records no process start; a server started by this release records one, which lets \`${currentCliCommandName()} stop\` confirm it from any home folder.`
+ : '';
+}
+
+export function describeOtherHomeFolderRemedy(
+ lock: ProcessLockMetadata,
+ platform: NodeJS.Platform,
+): string {
+ const stop = `\`${currentCliCommandName()} stop\``;
+ return (
+ `If the server was started on this machine from another account, run ${stop} as that account. ` +
+ `If it was started from this account with a different home folder, run ${stop} with \`${homeFolderVariable(platform)}\` set to that folder. ` +
+ `Otherwise stop it from the machine that owns the lock.${describeMissingProcessStart(lock)}`
+ );
+}
+
+function describeOtherHomeFolderCleanupRemedy(
+ lock: ProcessLockMetadata,
+ platform: NodeJS.Platform,
+): string {
+ const command = currentCliCommandName();
+ return (
+ `If the server was started on this machine from another account, run \`${command} stop\` as that account while it is running, or \`${command} clean\` once it has exited. ` +
+ `If it was started from this account with a different home folder, run the same command with \`${homeFolderVariable(platform)}\` set to that folder. ` +
+ `Otherwise run the same command on the machine that owns the lock.${describeMissingProcessStart(lock)}`
+ );
+}
+
+function keepTheLockUntilConfirmed(lockPath: string): string {
+ return (
+ 'Confirm that no OpenKnowledge server is using this directory on the owning machine or any other machine sharing it. ' +
+ `Only then remove the stale lock file at ${lockPath} and retry cleanup.`
+ );
+}
+
+export function describeForeignHostRemedy(
+ state: Extract<LockState, { status: 'foreign-host' }>,
+ platform: NodeJS.Platform = process.platform,
+): string {
+ const { lock, lockPath } = state;
+ const remedy =
+ typeof lock.machineId === 'string'
+ ? describeOtherHomeFolderCleanupRemedy(lock, platform)
+ : `If ${escapeDisplayPath(lock.hostname)} is another machine, stop OpenKnowledge there; if it is an earlier name of this machine, stop OpenKnowledge here if it is still running.`;
+ return `${remedy} ${keepTheLockUntilConfirmed(lockPath)}`;
}
export function describeLockOwnershipRefusal(
state: Extract<LockState, { status: 'foreign-host' | 'corrupt' | 'unverified-owner' }>,
+ platform: NodeJS.Platform = process.platform,
): string {
+ const { lockPath } = state;
if (state.status === 'unverified-owner') {
return (
- `Cannot verify shutdown: the server lock at ${state.lockPath} does not record its owning machine, ` +
- `and its recorded process ${state.pid} is still running on this machine. ` +
+ `Cannot verify shutdown: the server lock at ${lockPath} ${describeUnrecordedOwner(state.pid)}. ` +
`Check whether process ${state.pid} is an OpenKnowledge server. If it is, stop it and retry. ` +
'Do not remove the lock file while that process is running.'
);
}
- return (
- `Cannot verify shutdown: the server lock at ${state.lockPath} belongs to another machine. ` +
- 'Stop OpenKnowledge on the owning machine. Confirm that no OpenKnowledge server is using this directory on the owning machine or any other machine sharing it. ' +
- `Only then remove the stale lock file at ${state.lockPath} and retry cleanup.`
- );
+ if (state.status === 'corrupt') {
+ return (
+ `Cannot verify shutdown: the server lock at ${lockPath} records a machine ID or host name that is not this machine's, so its server may run on another machine, or on this one from another account, with a different home folder, or under an earlier host name. ` +
+ `Stop OpenKnowledge wherever it was started. ${keepTheLockUntilConfirmed(lockPath)}`
+ );
+ }
+ return `Cannot verify shutdown: the server lock at ${lockPath} ${describeForeignOwner(state.lock)}. ${describeForeignHostRemedy(state, platform)}`;
}
diff --git a/packages/cli/src/commands/ps.test.ts b/packages/cli/src/commands/ps.test.ts
index a67d32bd6..5af3b2047 100644
--- a/packages/cli/src/commands/ps.test.ts
+++ b/packages/cli/src/commands/ps.test.ts
@@ -44,6 +44,7 @@ function makeDeadServer(overrides?: {
startedAt: overrides?.startedAt ?? '2026-05-01T00:00:00.000Z',
worktreeRoot: overrides?.worktreeRoot ?? '/tmp/old-project',
},
+ text: '',
};
}
@@ -74,6 +75,7 @@ const missingLock: LockState = {
const corruptLock: LockState = {
status: 'corrupt',
lockPath: '/tmp/notes/.ok/server.lock',
+ text: '{',
};
describe('startedCell', () => {
@@ -495,6 +497,74 @@ describe('PORTS column', () => {
});
});
+describe('a lock whose pid now belongs to another process', () => {
+ const reusedPidLock: LockState = {
+ status: 'reused-pid',
+ lockPath: '/tmp/restarted/.ok/local/server.lock',
+ lock: {
+ pid: 55555,
+ hostname: 'test-host',
+ port: 5173,
+ startedAt: '2026-05-01T00:00:00.000Z',
+ worktreeRoot: '/tmp/restarted',
+ capabilities: ['http', 'ws', 'ui'],
+ },
+ text: '',
+ };
+
+ test('json reports it stale with no command, usage, binary or ui attributed to that process', async () => {
+ const lines: string[] = [];
+ await runPs({
+ discover: async () => ['/tmp/restarted/.ok/local'],
+ inspect: () => reusedPidLock,
+ resolveCommand: () => ELECTRON_UTILITY_COMMAND,
+ resolveUsage: () => ({ cpuPercent: 9.9, memPercent: 8.8 }),
+ json: true,
+ log: (msg) => lines.push(msg),
+ });
+
+ expect(JSON.parse(lines.join('\n'))).toEqual([
+ expect.objectContaining({
+ directory: '/tmp/restarted',
+ lockPath: '/tmp/restarted/.ok/local/server.lock',
+ displayStatus: 'stale',
+ command: null,
+ binary: null,
+ isDesktop: false,
+ ui: null,
+ server: expect.objectContaining({ status: 'reused-pid', pid: 55555, usage: null }),
+ }),
+ ]);
+ });
+
+ test('the table hides it by default and lists it as stale with all', async () => {
+ const hidden: string[] = [];
+ await runPs({
+ discover: async () => ['/tmp/restarted/.ok/local'],
+ inspect: () => reusedPidLock,
+ resolveCommand: () => ELECTRON_UTILITY_COMMAND,
+ resolveUsage: () => null,
+ log: (msg) => hidden.push(msg),
+ });
+ const shown: string[] = [];
+ await runPs({
+ discover: async () => ['/tmp/restarted/.ok/local'],
+ inspect: () => reusedPidLock,
+ resolveCommand: () => ELECTRON_UTILITY_COMMAND,
+ resolveUsage: () => null,
+ all: true,
+ log: (msg) => shown.push(msg),
+ });
+
+ expect(hidden.join('\n')).toBe('No open-knowledge servers found.');
+ const row = shown.join('\n').split('\n')[1] ?? '';
+ expect(row).toContain('/tmp/restarted');
+ expect(row).toContain('stale');
+ expect(row).toContain('55555');
+ expect(row).not.toContain('desktop');
+ });
+});
+
describe('server lock missing/corrupt discards entry', () => {
test('missing server lock: entry discarded', async () => {
const lines: string[] = [];
diff --git a/packages/cli/src/commands/ps.ts b/packages/cli/src/commands/ps.ts
index 37bcf9831..297966f88 100644
--- a/packages/cli/src/commands/ps.ts
+++ b/packages/cli/src/commands/ps.ts
@@ -137,6 +137,7 @@ function displayStatus(entry: PsEntry): DisplayStatus {
case 'foreign-host':
return entry.isDesktop ? 'desktop' : 'foreign';
case 'dead-pid':
+ case 'reused-pid':
return 'stale';
default: {
const exhaustive: never = serverStatus;
@@ -271,14 +272,16 @@ export async function runPs(deps: RunPsDeps = {}): Promise<void> {
serverState.status === 'missing' ||
serverState.status === 'corrupt' ||
serverState.status === 'read-error' ||
- serverState.status === 'unverified-owner'
+ serverState.status === 'unverified-owner' ||
+ serverState.status === 'reused-pid'
? null
: resolveCommand(serverState.lock.pid);
const serverUsage =
serverState.status === 'missing' ||
serverState.status === 'corrupt' ||
serverState.status === 'read-error' ||
- serverState.status === 'unverified-owner'
+ serverState.status === 'unverified-owner' ||
+ serverState.status === 'reused-pid'
? null
: resolveUsage(serverState.lock.pid);
const entry = buildEntry(lockDir, serverState, command, serverUsage);
@@ -303,7 +306,7 @@ export async function runPs(deps: RunPsDeps = {}): Promise<void> {
export function psCommand(): Command {
return new Command('ps')
.description('List all running open-knowledge servers')
- .argument('[modifier]', '"all" to include stale (dead-pid) entries')
+ .argument('[modifier]', '"all" to include stale entries')
.option('--all', 'Include stale entries (foreign and unverified entries already show)')
.option('--json', 'Emit structured JSON (always includes all statuses)')
.action(async (modifier: string | undefined, opts: { all?: boolean; json?: boolean }) => {
diff --git a/packages/cli/src/commands/removal-process-start.ts b/packages/cli/src/commands/removal-process-start.ts
deleted file mode 100644
index 1499e7a9c..000000000
--- a/packages/cli/src/commands/removal-process-start.ts
+++ /dev/null
@@ -1,100 +0,0 @@
-import { type ExecFileSyncOptionsWithStringEncoding, execFileSync } from 'node:child_process';
-import {
- debugNativeLoadFailure,
- describeNativeFailure,
- describeValue,
- type NativeConfigResolver,
- requireNativeConfigModule,
-} from '../native/load-native-config.ts';
-
-const PS_PROCESS_START_QUERY_TIMEOUT_MS = 5000;
-const MAX_PROCESS_ID = 0xffff_ffff;
-
-type NativeFailureKind = 'unavailable' | 'query-failed';
-
-export interface NativeFailure {
- kind: NativeFailureKind;
- reason: string;
-}
-
-export interface ProcessStartOptions {
- platform?: NodeJS.Platform;
- arch?: NodeJS.Architecture;
- nativeResolver?: Partial<NativeConfigResolver>;
- env?: NodeJS.ProcessEnv;
- run?: (command: string, args: string[], options: ExecFileSyncOptionsWithStringEncoding) => string;
- onNativeFailure?: (failure: NativeFailure) => void;
-}
-
-function describeBinding(binding: unknown): string {
- if (!binding || typeof binding !== 'object') return String(binding);
- const exported = typeof (binding as Record<string, unknown>).readProcessStart;
- return `${describeValue(binding)} readProcessStart=${exported}`;
-}
-
-export function readRemovalProcessStart(
- pid: number,
- options: ProcessStartOptions = {},
-): number | null {
- if (!Number.isSafeInteger(pid) || pid <= 0 || pid > MAX_PROCESS_ID) return null;
- const platform = options.platform ?? process.platform;
- const arch = options.arch ?? process.arch;
- const reportNativeFailure = (kind: NativeFailureKind, context: string, detail: unknown): void => {
- debugNativeLoadFailure(context, detail);
- options.onNativeFailure?.({ kind, reason: describeNativeFailure(context, detail) });
- };
- try {
- if (platform === 'win32') {
- let loadFailureReported = false;
- const binding = requireNativeConfigModule(options.nativeResolver, (context, detail) => {
- loadFailureReported = true;
- reportNativeFailure('unavailable', context, detail);
- });
- if (
- !binding ||
- typeof binding !== 'object' ||
- !('readProcessStart' in binding) ||
- typeof binding.readProcessStart !== 'function'
- ) {
- let detail: string;
- if (binding !== null && binding !== undefined) detail = describeBinding(binding);
- else if (loadFailureReported) detail = 'no usable binding was returned';
- else detail = 'the addon was not found';
- reportNativeFailure(
- 'unavailable',
- `the Windows native addon (native-config.win32-${arch}-msvc.node) did not load or does not export readProcessStart`,
- detail,
- );
- return null;
- }
- const startedAt: unknown = binding.readProcessStart(pid);
- if (typeof startedAt === 'number' && Number.isSafeInteger(startedAt) && startedAt >= 0)
- return startedAt;
- reportNativeFailure(
- 'unavailable',
- 'readProcessStart returned an invalid value',
- describeValue(startedAt),
- );
- return null;
- } else if (platform === 'darwin' || platform === 'linux') {
- const env = options.env ?? process.env;
- const run = options.run ?? execFileSync;
- const commandOptions: ExecFileSyncOptionsWithStringEncoding = {
- encoding: 'utf8',
- stdio: ['ignore', 'pipe', 'pipe'],
- timeout: PS_PROCESS_START_QUERY_TIMEOUT_MS,
- windowsHide: true,
- env: { ...env, LC_ALL: 'C', TZ: 'UTC0' },
- };
- const raw = run('ps', ['-p', String(pid), '-o', 'lstart='], commandOptions).trim();
- if (!/^\w{3}\s+\w{3}\s+\d{1,2}\s+\d{2}:\d{2}:\d{2}\s+\d{4}$/.test(raw)) return null;
- const startedAt = Date.parse(`${raw} UTC`);
- return Number.isFinite(startedAt) ? startedAt : null;
- } else {
- return null;
- }
- } catch (err) {
- if (platform === 'win32') reportNativeFailure('query-failed', 'readProcessStart failed', err);
- return null;
- }
-}
diff --git a/packages/cli/src/commands/start.test.ts b/packages/cli/src/commands/start.test.ts
index 6b1060730..7ae7fc7ab 100644
--- a/packages/cli/src/commands/start.test.ts
+++ b/packages/cli/src/commands/start.test.ts
@@ -1,5 +1,6 @@
import type { spawn as NativeSpawn } from 'node:child_process';
import {
+ chmodSync,
existsSync,
mkdirSync,
mkdtempSync,
@@ -1317,6 +1318,40 @@ describe('tryDescribeLockCollision', () => {
expect(tryDescribeLockCollision(err, '/other', serverModule)).toBe(err.message);
expect(err.message).toContain('Close that session');
});
+
+ test.runIf(process.platform !== 'win32' && process.getuid?.() !== 0)(
+ 'a lock acquisition cannot read has its remedy as the message, without a stack',
+ async () => {
+ const serverModule = await import('@inkeep/open-knowledge-server');
+ const projectDir = await mkdtemp(join(tmpdir(), 'ok-start-unreadable-lock-'));
+ const lockDir = join(projectDir, '.ok', 'local');
+ const lockPath = join(lockDir, 'server.lock');
+ try {
+ mkdirSync(lockDir, { recursive: true });
+ writeFileSync(lockPath, '{}');
+ chmodSync(lockPath, 0o000);
+ const err = (() => {
+ try {
+ serverModule.acquireProcessLock({
+ lockName: 'server',
+ lockDir,
+ metadata: { port: 0, worktreeRoot: projectDir },
+ });
+ } catch (thrown) {
+ return thrown;
+ }
+ return undefined;
+ })();
+
+ expect(tryDescribeLockCollision(err, projectDir, serverModule)).toBe(
+ `[server-lock] Could not read the lock at ${lockPath}: EACCES: permission denied, open '${lockPath}'. Restore file and parent-directory access, then retry.`,
+ );
+ } finally {
+ chmodSync(lockPath, 0o600);
+ await rm(projectDir, { recursive: true, force: true });
+ }
+ },
+ );
function fakeServerModule(opts: {
meta?: { kind?: string; pid?: number; port?: number; hostname?: string } | null;
throwOnRead?: boolean;
diff --git a/packages/cli/src/commands/start.ts b/packages/cli/src/commands/start.ts
index fc7ec5ad2..130183893 100644
--- a/packages/cli/src/commands/start.ts
+++ b/packages/cli/src/commands/start.ts
@@ -958,6 +958,8 @@ export function tryDescribeLockCollision(
if (authorityErr !== undefined && err instanceof authorityErr) return err.message;
const registryErr = serverModule.ServerAuthorityRegistryError;
if (registryErr !== undefined && err instanceof registryErr) return err.message;
+ const lockFileErr = serverModule.ProcessLockFileError;
+ if (lockFileErr !== undefined && err instanceof lockFileErr) return err.message;
const lockErr = serverModule.ServerLockCollisionError;
if (lockErr === undefined || !(err instanceof lockErr)) return null;
diff --git a/packages/cli/src/commands/status.test.ts b/packages/cli/src/commands/status.test.ts
index b0763ed57..dfe159860 100644
--- a/packages/cli/src/commands/status.test.ts
+++ b/packages/cli/src/commands/status.test.ts
@@ -30,10 +30,25 @@ function dead(pid: number): LockState {
startedAt: '2026-04-16T00:00:00Z',
worktreeRoot: '/x',
},
+ text: '',
+ };
+}
+function reused(pid: number): LockState {
+ return {
+ status: 'reused-pid',
+ lockPath: '/tmp/r.lock',
+ lock: {
+ pid,
+ port: 3001,
+ hostname: 'host',
+ startedAt: '2026-04-16T00:00:00Z',
+ worktreeRoot: '/x',
+ },
+ text: '',
};
}
function corrupt(): LockState {
- return { status: 'corrupt', lockPath: '/tmp/c.lock' };
+ return { status: 'corrupt', lockPath: '/tmp/c.lock', text: '{' };
}
function foreign(): LockState {
return {
@@ -78,6 +93,19 @@ describe('buildStatusReport', () => {
expect(r.server.pid).toBe(999);
});
+ test('reused-pid → alive: false, reports the recorded lock', () => {
+ const r = buildStatusReport(reused(4242));
+ expect(r.server).toEqual({
+ name: 'server',
+ state: 'reused-pid',
+ pid: 4242,
+ port: 3001,
+ startedAt: '2026-04-16T00:00:00Z',
+ host: 'host',
+ alive: false,
+ });
+ });
+
test('corrupt → alive: false, no pid surfaced', () => {
const r = buildStatusReport(corrupt());
expect(r.server.state).toBe('corrupt');
@@ -107,6 +135,11 @@ describe('buildStatusReport', () => {
});
describe('renderStatusText', () => {
+ test('reused-pid renders as stale and points at ok clean', () => {
+ const out = renderStatusText(buildStatusReport(reused(4242)));
+ expect(out).toContain('stale (pid=4242 now belongs to another process); run `ok clean`');
+ });
+
test('alive entries include pid + port + startedAt', () => {
const out = renderStatusText(buildStatusReport(alive(100, 3001, 'host', ['http', 'ws', 'ui'])));
expect(out).toContain('pid=100 port=3001');
@@ -201,5 +234,8 @@ describe('status hints name the running channel CLI', () => {
expect(renderStatusText(buildStatusReport(corrupt()))).toContain(
`server lock file corrupt — run \`${command} clean\``,
);
+ expect(renderStatusText(buildStatusReport(reused(999)))).toContain(
+ `server stale (pid=999 now belongs to another process); run \`${command} clean\``,
+ );
});
});
diff --git a/packages/cli/src/commands/status.ts b/packages/cli/src/commands/status.ts
index ebe114a5a..c91bfb737 100644
--- a/packages/cli/src/commands/status.ts
+++ b/packages/cli/src/commands/status.ts
@@ -58,9 +58,10 @@ function summarize(name: 'server' | 'ui', state: LockState): StatusEntry {
alive: 'unknown',
};
case 'dead-pid':
+ case 'reused-pid':
return {
name,
- state: 'dead-pid',
+ state: state.status,
pid: state.lock.pid,
port: state.lock.port,
startedAt: state.lock.startedAt,
@@ -104,6 +105,9 @@ function renderEntry(entry: StatusEntry): string {
if (entry.state === 'dead-pid') {
return `${label} stale (dead pid=${entry.pid}) — run \`${currentCliCommandName()} clean\``;
}
+ if (entry.state === 'reused-pid') {
+ return `${label} stale (pid=${entry.pid} now belongs to another process); run \`${currentCliCommandName()} clean\``;
+ }
if (entry.servedByServer === true) {
return `${label} served by server pid=${entry.pid} port=${entry.port}`;
}
diff --git a/packages/cli/src/commands/stop-for-removal.test.ts b/packages/cli/src/commands/stop-for-removal.test.ts
index 6c113d5e3..055754bcb 100644
--- a/packages/cli/src/commands/stop-for-removal.test.ts
+++ b/packages/cli/src/commands/stop-for-removal.test.ts
@@ -1,4 +1,4 @@
-import { type ChildProcess, execFileSync, type SpawnSyncReturns, spawn } from 'node:child_process';
+import { type ChildProcess, execFileSync, spawn } from 'node:child_process';
import { once } from 'node:events';
import {
chmodSync,
@@ -12,12 +12,23 @@ import {
} from 'node:fs';
import { hostname, tmpdir } from 'node:os';
import { join } from 'node:path';
-import { isProcessAlive, lockFilePath, scanLockProcesses } from '@inkeep/open-knowledge-server';
+import {
+ getMachineId,
+ isProcessAlive,
+ lockFilePath,
+ type ProcessStart,
+ scanLockProcesses,
+} from '@inkeep/open-knowledge-server';
import pino from 'pino';
import { afterEach, beforeEach, describe, expect, test, vi } from 'vitest';
+import {
+ processStartTokenFromOs,
+ processStartTokenOffsetFromOs,
+ readProcessStartFromOs,
+} from '../../../../test-support/os-process-start.test-helper.ts';
import * as cliLogger from '../cli-logger.ts';
+import { type HolderProcessStartOptions, readHolderProcessStart } from './holder-process-start.ts';
import { runRemoval } from './removal-plan.ts';
-import { readRemovalProcessStart } from './removal-process-start.ts';
import { stopServerForRemoval } from './stop-for-removal.ts';
describe('stopServerForRemoval', () => {
@@ -66,7 +77,13 @@ describe('stopServerForRemoval', () => {
if (pid === undefined) throw new Error('Server fixture did not start');
writeFileSync(
lockFilePath(dir, 'server'),
- JSON.stringify({ pid, hostname: hostname(), port: 0, startedAt: new Date().toISOString() }),
+ JSON.stringify({
+ pid,
+ hostname: hostname(),
+ machineId: getMachineId(),
+ port: 0,
+ startedAt: new Date().toISOString(),
+ }),
);
return pid;
}
@@ -119,7 +136,13 @@ describe('stopServerForRemoval', () => {
});
writeFileSync(
lockFilePath(dir, 'server'),
- JSON.stringify({ pid, hostname: hostname(), port: 0, startedAt: new Date().toISOString() }),
+ JSON.stringify({
+ pid,
+ hostname: hostname(),
+ machineId: getMachineId(),
+ port: 0,
+ startedAt: new Date().toISOString(),
+ }),
);
return pid;
}
@@ -327,7 +350,7 @@ describe('stopServerForRemoval', () => {
},
);
- test('does not signal an unrelated process whose PID appears in an older lock', async () => {
+ test('skips a lock whose PID now belongs to a process started after the lock was written, without signalling it', async () => {
const pid = await startServer(`
process.send('ready');
setInterval(() => {}, 1000);
@@ -336,59 +359,94 @@ describe('stopServerForRemoval', () => {
lockFilePath(dir, 'server'),
JSON.stringify({ pid, hostname: hostname(), port: 0, startedAt: '2000-01-01T00:00:00.000Z' }),
);
- const probe: {
- elapsedMs?: number;
- code?: string | null;
- status?: number | null;
- signal?: NodeJS.Signals | null;
- output?: string;
- } = {};
- const removal = stopServerForRemoval(dir, {
- readProcessStart: (processId) =>
- readRemovalProcessStart(processId, {
- run: (command, args, options) => {
- const startedAt = performance.now();
- try {
- const output = execFileSync(command, args, options);
- probe.output = output.trim().slice(0, 80);
- return output;
- } catch (error) {
- const failure = error as NodeJS.ErrnoException &
- Pick<SpawnSyncReturns<string>, 'status' | 'signal'>;
- probe.code = failure.code ?? null;
- probe.status = failure.status ?? null;
- probe.signal = failure.signal ?? null;
- throw error;
- } finally {
- probe.elapsedMs = Math.round(performance.now() - startedAt);
- }
- },
- }),
+ expect(await stopServerForRemoval(dir)).toEqual({
+ stopped: 0,
+ failed: [],
+ skipped: `Skipped stale lock at ${lockFilePath(dir, 'server')}; its recorded process ${pid} has exited and that process ID now belongs to another program, which was not signalled. No server was stopped.`,
});
- await expect(removal, `OS process-start query: ${JSON.stringify(probe)}`).rejects.toThrow(
- 'started after the server lock',
- );
expect(isProcessAlive(pid)).toBe(true);
- expectRefusalLogged('alive', 'started after the server lock');
+ expect(warnings).toEqual([]);
});
+ test.skipIf(process.platform === 'win32')(
+ 'judges a lock by its recorded process start even when its startedAt is later than the holder started',
+ async () => {
+ const pid = await startServer(`
+ process.send('ready');
+ setInterval(() => {}, 1000);
+ `);
+ writeFileSync(
+ lockFilePath(dir, 'server'),
+ JSON.stringify({
+ pid,
+ hostname: hostname(),
+ port: 0,
+ startedAt: new Date().toISOString(),
+ processStart: processStartTokenOffsetFromOs(pid, -5),
+ }),
+ );
+ expect(await stopServerForRemoval(dir)).toMatchObject({
+ stopped: 0,
+ failed: [],
+ skipped: expect.stringContaining('now belongs to another program'),
+ });
+ expect(isProcessAlive(pid)).toBe(true);
+ },
+ );
+
test('does not signal a process when its lock has no acquisition timestamp', async () => {
const pid = await startServer(`
process.send('ready');
setInterval(() => {}, 1000);
`);
- writeFileSync(
- lockFilePath(dir, 'server'),
- JSON.stringify({ pid, hostname: hostname(), port: 0 }),
+ const lockPath = lockFilePath(dir, 'server');
+ writeFileSync(lockPath, JSON.stringify({ pid, hostname: hostname(), port: 0 }));
+ const error = await stopServerForRemoval(dir).then(
+ () => undefined,
+ (err: unknown) => err as Error,
+ );
+ expect(error?.message).toBe(
+ `The start of process ${pid} cannot be compared with the start recorded in ${lockPath}, so it cannot be confirmed as the server that wrote the lock; ` +
+ 'it was not signalled. ' +
+ `Before ending process ${pid} yourself, confirm that it is an OpenKnowledge server. ` +
+ `Once you have confirmed no OpenKnowledge server is running for this project, remove ${lockPath} and retry cleanup.`,
);
- await expect(stopServerForRemoval(dir)).rejects.toThrow('valid start time');
expect(isProcessAlive(pid)).toBe(true);
- expectRefusalLogged('alive', 'valid start time');
+ expect(warnings).toContainEqual(
+ expect.objectContaining({
+ pid: process.pid,
+ lockPid: pid,
+ cause: 'process-start-incomparable',
+ msg: 'stop declined: lock holder identity unverified',
+ }),
+ );
});
+ test.skipIf(process.platform === 'win32')(
+ 'stops a holder matching the recorded process start even when its lock has no acquisition timestamp',
+ async () => {
+ const pid = await startServer(`
+ process.send('ready');
+ setInterval(() => {}, 1000);
+ `);
+ writeFileSync(
+ lockFilePath(dir, 'server'),
+ JSON.stringify({
+ pid,
+ hostname: hostname(),
+ port: 0,
+ processStart: processStartTokenFromOs(pid),
+ }),
+ );
+ expect(await stopServerForRemoval(dir)).toEqual({ stopped: 1, failed: [] });
+ const exited = await fixtureExit(pid);
+ expect(exited.signalCode ?? exited.exitCode).not.toBe(0);
+ },
+ );
+
test.each([
- ['linux', 'ps supporting -p and -o lstart='],
- ['darwin', 'Stop the server manually.'],
+ ['linux', '/proc'],
+ ['darwin', 'macOS reads the process start with ps, and that read failed'],
] as const)(
'explains the %s probe recovery when the process start probe is unavailable',
async (platform, expected) => {
@@ -396,42 +454,84 @@ describe('stopServerForRemoval', () => {
process.send('ready');
setInterval(() => {}, 1000);
`);
- await expect(
- stopServerForRemoval(dir, { platform, readProcessStart: () => null }),
- ).rejects.toThrow('Confirm the OpenKnowledge server has exited before retrying cleanup');
+ const error = await stopServerForRemoval(dir, {
+ platform,
+ readProcessStart: () => null,
+ }).then(
+ () => undefined,
+ (err: unknown) => err as Error,
+ );
+ expect(error?.message).toContain(expected);
+ expect(error?.message).toContain(`remove ${lockFilePath(dir, 'server')} and retry cleanup.`);
+ expect(error?.message).not.toContain('--force');
expect(isProcessAlive(pid)).toBe(true);
- expectRefusalLogged('alive', 'Cannot verify the identity');
- expect(warnings[0]).toMatchObject({ lockPid: pid });
- expect(warnings[0]?.msg).toContain(expected);
+ expect(warnings).toEqual([
+ expect.objectContaining({
+ level: 40,
+ lockDir: dir,
+ lockPath: lockFilePath(dir, 'server'),
+ pid: process.pid,
+ lockPid: pid,
+ platform,
+ cause: 'process-start-unreadable',
+ msg: 'stop declined: lock holder identity unverified',
+ }),
+ ]);
},
);
+ test('without a CLI logger, as the desktop removes a project, the refusal points at no log', async () => {
+ vi.spyOn(cliLogger, 'getCliLogger').mockReturnValue(undefined);
+ await startServer(`
+ process.send('ready');
+ setInterval(() => {}, 1000);
+ `);
+ const error = await stopServerForRemoval(dir, {
+ readProcessStart: (_pid, options) => {
+ options?.onNativeFailure?.({
+ kind: 'query-failed',
+ reason: 'readProcessStart failed: OpenProcess: os error 5',
+ });
+ return null;
+ },
+ }).then(
+ () => undefined,
+ (err: unknown) => err as Error,
+ );
+ expect(error?.message).toContain('Could not read when process');
+ expect(error?.message).not.toContain('recorded in');
+ expect(error?.message).not.toContain('~/.ok/logs');
+ });
+
test('records the native identity failure cause in the CLI log, never in the refusal a user reads', async () => {
const pid = await startServer(`
process.send('ready');
setInterval(() => {}, 1000);
`);
- await expect(
- stopServerForRemoval(dir, {
- readProcessStart: (_pid, options) => {
- options?.onNativeFailure?.({
- kind: 'query-failed',
- reason: 'readProcessStart failed: OpenProcess: os error 5',
- });
- return null;
- },
- }),
- ).rejects.toThrow('Cannot verify the identity');
+ const error = await stopServerForRemoval(dir, {
+ readProcessStart: (_pid, options) => {
+ options?.onNativeFailure?.({
+ kind: 'query-failed',
+ reason: 'readProcessStart failed: OpenProcess: os error 5',
+ });
+ return null;
+ },
+ }).then(
+ () => undefined,
+ (err: unknown) => err as Error,
+ );
+ expect(error?.message).toContain('Could not read when process');
expect(isProcessAlive(pid)).toBe(true);
expect(warnings).toHaveLength(1);
expect(warnings[0]).toMatchObject({
+ cause: 'process-start-unreadable',
nativeFailures: [
{ kind: 'query-failed', reason: 'readProcessStart failed: OpenProcess: os error 5' },
],
});
- expect(warnings[0]?.msg).toContain(lockFilePath(dir, 'server'));
- expect(warnings[0]?.msg).not.toContain('OpenProcess: os error 5');
- expect(warnings[0]?.msg).toContain('recorded in the CLI log under ~/.ok/logs');
+ expect(error?.message).toContain(lockFilePath(dir, 'server'));
+ expect(error?.message).not.toContain('OpenProcess: os error 5');
+ expect(error?.message).toContain('recorded in the CLI log under ~/.ok/logs');
});
test.each([
@@ -452,17 +552,19 @@ describe('stopServerForRemoval', () => {
process.send('ready');
setInterval(() => {}, 1000);
`);
- await expect(
- stopServerForRemoval(dir, {
- platform: 'win32',
- readProcessStart: (_pid, options) => {
- options?.onNativeFailure?.({ kind, reason: 'probe detail' });
- return null;
- },
- }),
- ).rejects.toThrow(expected);
+ const error = await stopServerForRemoval(dir, {
+ platform: 'win32',
+ readProcessStart: (_pid, options) => {
+ options?.onNativeFailure?.({ kind, reason: 'probe detail' });
+ return null;
+ },
+ }).then(
+ () => undefined,
+ (err: unknown) => err as Error,
+ );
+ expect(error?.message).toContain(expected);
expect(isProcessAlive(pid)).toBe(true);
- expect(warnings[0]?.msg).not.toContain(absent);
+ expect(error?.message).not.toContain(absent);
},
);
@@ -471,12 +573,19 @@ describe('stopServerForRemoval', () => {
process.send('ready');
setInterval(() => {}, 1000);
`);
- await expect(
- stopServerForRemoval(dir, { platform: 'win32', readProcessStart: () => null }),
- ).rejects.toThrow('Stop the server manually.');
+ const error = await stopServerForRemoval(dir, {
+ platform: 'win32',
+ readProcessStart: () => null,
+ }).then(
+ () => undefined,
+ (err: unknown) => err as Error,
+ );
+ expect(error?.message).toContain(
+ `it was not signalled. Before ending process ${pid} yourself, confirm that it is an OpenKnowledge server.`,
+ );
expect(isProcessAlive(pid)).toBe(true);
- expect(warnings[0]?.msg).not.toContain('Reinstall OpenKnowledge');
- expect(warnings[0]?.msg).not.toContain('refused the query');
+ expect(error?.message).not.toContain('Reinstall OpenKnowledge');
+ expect(error?.message).not.toContain('refused the query');
});
test('does not prescribe a reinstall when the component loaded and the OS refused the query', async () => {
@@ -494,7 +603,7 @@ describe('stopServerForRemoval', () => {
return null;
},
}),
- ).rejects.toThrow('Cannot verify the identity');
+ ).rejects.toThrow('Could not read when process');
expect(isProcessAlive(pid)).toBe(true);
expect(warnings[0]).toMatchObject({
nativeFailures: [{ kind: 'query-failed' }],
@@ -518,7 +627,7 @@ describe('stopServerForRemoval', () => {
(err: unknown) => err as Error,
);
expect(isProcessAlive(pid)).toBe(true);
- expect(error?.message).toContain('Cannot verify the identity');
+ expect(error?.message).toContain('Could not read when process');
for (const upstreamOnly of ['package-lock.json', 'node_modules', 'npm i', 'npm/cli/issues']) {
expect(error?.message).not.toContain(upstreamOnly);
}
@@ -546,15 +655,16 @@ describe('stopServerForRemoval', () => {
return null;
},
}),
- ).rejects.toThrow('Cannot verify the identity');
+ ).rejects.toThrow('Could not read when process');
expect(isProcessAlive(pid)).toBe(true);
expect(warnings).toHaveLength(1);
expect(warnings[0]).toMatchObject({
level: 40,
lockDir: dir,
lockPath: lockFilePath(dir, 'server'),
+ pid: process.pid,
lockPid: pid,
- status: 'alive',
+ cause: 'process-start-unreadable',
nativeFailures: [
{ kind: 'unavailable', reason: 'bundled loader failed to load: invalid ELF header' },
{ kind: 'unavailable', reason: 'the Windows native addon did not load' },
@@ -567,38 +677,51 @@ describe('stopServerForRemoval', () => {
process.send('ready');
setInterval(() => {}, 1000);
`);
- await expect(stopServerForRemoval(dir, { readProcessStart: () => null })).rejects.toThrow(
- 'Cannot verify the identity',
+ const error = await stopServerForRemoval(dir, { readProcessStart: () => null }).then(
+ () => undefined,
+ (err: unknown) => err as Error,
);
+ expect(error?.message).toContain('Could not read when process');
expect(isProcessAlive(pid)).toBe(true);
- expect(warnings[0]?.msg).not.toContain('~/.ok/logs');
+ expect(error?.message).not.toContain('~/.ok/logs');
expect(warnings[0]).not.toHaveProperty('nativeFailures');
});
- test('does not log a native failure when the liveness recheck accepts an exited process', async () => {
+ test('does not log a native failure when the holder exits between inspection and the stop signal', async () => {
const pid = await startServer(`
process.send('ready');
setInterval(() => {}, 1000);
`);
- const isAlive = vi.fn().mockReturnValueOnce(true).mockReturnValueOnce(false);
- expect(
- await stopServerForRemoval(dir, {
- readProcessStart: (_pid, options) => {
- options?.onNativeFailure?.({
- kind: 'query-failed',
- reason: 'readProcessStart failed: OpenProcess: os error 87',
- });
- return null;
- },
- isAlive,
- }),
- ).toEqual({ stopped: 0, failed: [] });
+ const start = readHolderProcessStart(pid);
+ if (start === null) throw new Error('Could not read fixture process start time');
+ const readProcessStart = vi
+ .fn<(pid: number, options?: HolderProcessStartOptions) => ProcessStart | null>()
+ .mockReturnValueOnce(start)
+ .mockImplementation((_pid, options) => {
+ options?.onNativeFailure?.({
+ kind: 'query-failed',
+ reason: 'readProcessStart failed: OpenProcess: os error 87',
+ });
+ return null;
+ });
+ const isAlive = vi
+ .fn()
+ .mockReturnValueOnce(true)
+ .mockReturnValueOnce(true)
+ .mockReturnValueOnce(false);
+
+ expect(await stopServerForRemoval(dir, { readProcessStart, isAlive })).toEqual({
+ stopped: 0,
+ failed: [],
+ });
+ expect(readProcessStart).toHaveBeenCalledTimes(2);
+ expect(readProcessStart.mock.calls[1]?.[1]?.onNativeFailure).toBeTypeOf('function');
expect(warnings).toEqual([]);
- expect(isAlive).toHaveBeenCalledTimes(2);
+ expect(isAlive).toHaveBeenCalledTimes(3);
expect(isProcessAlive(pid)).toBe(true);
});
- test('requires a liveness recheck to accept an exited process after an unavailable identity probe', async () => {
+ test('skips the lock when its holder exits while inspection reads its start', async () => {
const pid = await startServer(`
process.send('ready');
setInterval(() => {}, 1000);
@@ -607,6 +730,7 @@ describe('stopServerForRemoval', () => {
expect(await stopServerForRemoval(dir, { readProcessStart: () => null, isAlive })).toEqual({
stopped: 0,
failed: [],
+ skipped: `Skipped stale lock at ${lockFilePath(dir, 'server')}; its recorded local process ${pid} has exited. No server was stopped.`,
});
expect(isAlive).toHaveBeenCalledTimes(2);
expect(isProcessAlive(pid)).toBe(true);
@@ -617,19 +741,24 @@ describe('stopServerForRemoval', () => {
process.send('ready');
setInterval(() => {}, 1000);
`);
- const startedAt = readRemovalProcessStart(pid);
- if (startedAt === null) throw new Error('Could not read fixture process start time');
- const startSecond = Math.floor(startedAt / 1000) * 1000;
+ const start = readHolderProcessStart(pid);
+ if (start === null) throw new Error('Could not read fixture process start time');
+ const startSecond = Math.floor(start.startedAtMs / 1000) * 1000;
writeFileSync(
lockFilePath(dir, 'server'),
JSON.stringify({
pid,
hostname: hostname(),
+ machineId: getMachineId(),
port: 0,
startedAt: new Date(startSecond + 999).toISOString(),
}),
);
- expect(await stopServerForRemoval(dir, { readProcessStart: () => startSecond })).toEqual({
+ expect(
+ await stopServerForRemoval(dir, {
+ readProcessStart: () => ({ ...start, startedAtMs: startSecond }),
+ }),
+ ).toEqual({
stopped: 1,
failed: [],
});
@@ -637,6 +766,175 @@ describe('stopServerForRemoval', () => {
expect(exited.signalCode ?? exited.exitCode).not.toBe(0);
});
+ describe('a same-machine lock that records no process start', () => {
+ const MINUTE_MS = 60_000;
+
+ async function startSignalRecordingServer(): Promise<{
+ pid: number;
+ receivedSigterm: () => boolean;
+ }> {
+ const marker = join(dir, 'sigterm-received');
+ const pid = await startServer(`
+ const fs = require('node:fs');
+ process.on('SIGTERM', () => {
+ fs.writeFileSync(${JSON.stringify(marker)}, 'SIGTERM');
+ process.exit(0);
+ });
+ process.send('ready');
+ setInterval(() => {}, 1000);
+ `);
+ return { pid, receivedSigterm: () => existsSync(marker) };
+ }
+
+ function writeLockWithoutProcessStart(pid: number, startedAtMs: number): void {
+ writeFileSync(
+ lockFilePath(dir, 'server'),
+ JSON.stringify({
+ pid,
+ hostname: hostname(),
+ machineId: getMachineId(),
+ port: 0,
+ startedAt: new Date(startedAtMs).toISOString(),
+ }),
+ );
+ }
+
+ function holderStart(clock: ProcessStart['clock'], startedAtMs: number): ProcessStart {
+ if (clock === 'creation') return { startedAtMs, clock };
+ const bootedAtMs = startedAtMs - 60 * MINUTE_MS;
+ return {
+ startedAtMs,
+ clock,
+ startTicks: (startedAtMs - bootedAtMs) / 10,
+ bootId: 'fixture-boot-id',
+ };
+ }
+
+ function removalReport(
+ options?: Parameters<typeof stopServerForRemoval>[1],
+ ): Promise<{ stopped: number; failed: unknown[]; explanation: string | undefined }> {
+ return stopServerForRemoval(dir, options).then(
+ ({ stopped, failed, skipped }) => ({ stopped, failed, explanation: skipped }),
+ (err: unknown) => ({
+ stopped: 0,
+ failed: [],
+ explanation: err instanceof Error ? err.message : String(err),
+ }),
+ );
+ }
+
+ test.each([
+ { clock: 'creation', platform: 'darwin', gapMs: 5_000, gap: '5 s' },
+ { clock: 'boot-derived', platform: 'linux', gapMs: MINUTE_MS, gap: '60 s' },
+ ] as const)(
+ 'does not signal a holder whose $clock-clock start is $gap after the lock was written, and reports it was not signalled',
+ async ({ clock, platform, gapMs }) => {
+ const { pid, receivedSigterm } = await startSignalRecordingServer();
+ const lockWrittenAtMs = Date.now() - 10 * MINUTE_MS;
+ writeLockWithoutProcessStart(pid, lockWrittenAtMs);
+
+ const report = await removalReport({
+ platform,
+ readProcessStart: () => holderStart(clock, lockWrittenAtMs + gapMs),
+ });
+
+ expect(receivedSigterm(), 'the removal sent SIGTERM to the holder').toBe(false);
+ expect(isProcessAlive(pid)).toBe(true);
+ expect(report).toEqual({
+ stopped: 0,
+ failed: [],
+ explanation: expect.stringMatching(new RegExp(`\\b${pid}\\b.*not signalled`)),
+ });
+ },
+ );
+
+ test.each([
+ { clock: 'creation', platform: 'darwin', gapMs: 5_000, gap: '5 s' },
+ { clock: 'boot-derived', platform: 'linux', gapMs: MINUTE_MS, gap: '60 s' },
+ ] as const)(
+ 'still stops a holder whose $clock-clock start is $gap before the lock was written',
+ async ({ clock, platform, gapMs }) => {
+ const { pid, receivedSigterm } = await startSignalRecordingServer();
+ const lockWrittenAtMs = Date.now() - 10 * MINUTE_MS;
+ writeLockWithoutProcessStart(pid, lockWrittenAtMs);
+
+ expect(
+ await stopServerForRemoval(dir, {
+ platform,
+ readProcessStart: () => holderStart(clock, lockWrittenAtMs - gapMs),
+ }),
+ ).toEqual({ stopped: 1, failed: [] });
+ const exited = await fixtureExit(pid);
+ if (process.platform !== 'win32') {
+ expect(receivedSigterm()).toBe(true);
+ expect(exited.exitCode).toBe(0);
+ }
+ },
+ );
+
+ test.skipIf(process.platform === 'win32')(
+ 'does not signal a live process that started seconds after the lock was written, read from the OS',
+ async () => {
+ const { pid, receivedSigterm } = await startSignalRecordingServer();
+ writeLockWithoutProcessStart(pid, readProcessStartFromOs(pid).startedAtMs - 5_000);
+
+ const report = await removalReport();
+
+ expect(receivedSigterm(), 'the removal sent SIGTERM to the holder').toBe(false);
+ expect(isProcessAlive(pid)).toBe(true);
+ expect(report).toEqual({
+ stopped: 0,
+ failed: [],
+ explanation: expect.stringMatching(new RegExp(`\\b${pid}\\b.*not signalled`)),
+ });
+ },
+ );
+ });
+
+ test('reads the recorded process start again at the signal and leaves a replaced holder running', async () => {
+ const pid = await startServer(`
+ process.send('ready');
+ setInterval(() => {}, 1000);
+ `);
+ const start = readHolderProcessStart(pid);
+ if (start === null) throw new Error('Could not read fixture process start time');
+ const readProcessStart = vi
+ .fn()
+ .mockReturnValueOnce(start)
+ .mockReturnValue({ ...start, startedAtMs: start.startedAtMs + 36 * 60 * 60 * 1000 });
+
+ expect(await stopServerForRemoval(dir, { readProcessStart })).toEqual({
+ stopped: 0,
+ failed: [],
+ });
+ expect(isProcessAlive(pid)).toBe(true);
+ });
+
+ test('refuses a lock from before machine IDs with the cleanup remedy and no restart advice', async () => {
+ const pid = await startServer(`
+ process.send('ready');
+ setInterval(() => {}, 1000);
+ `);
+ const lockPath = lockFilePath(dir, 'server');
+ writeFileSync(
+ lockPath,
+ JSON.stringify({ pid, hostname: hostname(), port: 0, startedAt: new Date().toISOString() }),
+ );
+
+ const error = await stopServerForRemoval(dir).then(
+ () => undefined,
+ (err: unknown) => err as Error,
+ );
+
+ expect(error?.message).toBe(
+ `The server lock at ${lockPath} records neither a machine ID nor a process start, as locks written by version 0.25.0 and earlier do, so process ${pid} cannot be confirmed as the server that wrote it; ` +
+ 'it was not signalled. ' +
+ `Before ending process ${pid} yourself, confirm that it is an OpenKnowledge server. ` +
+ `Once you have confirmed no OpenKnowledge server is running for this project, remove ${lockPath} and retry cleanup.`,
+ );
+ expect(isProcessAlive(pid)).toBe(true);
+ });
+
test('does not signal a local process named by a foreign-host lock', async () => {
const pid = await startServer(`
process.send('ready');
@@ -689,7 +987,7 @@ describe('stopServerForRemoval', () => {
const stateReads = readFileSync(psCalls.path, 'utf8')
.split('\n')
.filter((call) => call.includes('stat=') && call.includes(String(pid)));
- expect(stateReads.length).toBeLessThanOrEqual(2);
+ expect(stateReads.length).toBeLessThanOrEqual(3);
},
);
diff --git a/packages/cli/src/commands/stop-for-removal.ts b/packages/cli/src/commands/stop-for-removal.ts
index e70be4fac..9b72696c7 100644
--- a/packages/cli/src/commands/stop-for-removal.ts
+++ b/packages/cli/src/commands/stop-for-removal.ts
@@ -1,23 +1,19 @@
import { realpath } from 'node:fs/promises';
import { resolve } from 'node:path';
import { setTimeout } from 'node:timers/promises';
-import { okUserHomeDisplayPath } from '@inkeep/open-knowledge-core';
import {
createProbeFailureReporter,
isLockProcessRunning,
isProcessAlive,
type LockProcessScan,
type ProcessProbeOptions,
+ type ProcessStart,
scanLockProcesses,
} from '@inkeep/open-knowledge-server';
import { getCliLogger } from '../cli-logger.ts';
+import { type HolderProcessStartOptions, readHolderProcessStart } from './holder-process-start.ts';
import { describeLockOwnershipRefusal, inspectLock } from './lock-state.ts';
-import {
- type NativeFailure,
- type ProcessStartOptions,
- readRemovalProcessStart,
-} from './removal-process-start.ts';
-import { runStop } from './stop.ts';
+import { describeUnconfirmedHolderStart, runStop } from './stop.ts';
interface StopForRemovalOptions {
preserveProjectState?: boolean;
@@ -25,10 +21,7 @@ interface StopForRemovalOptions {
scanProcesses?: () => Promise<LockProcessScan>;
timeoutMs?: number;
pollIntervalMs?: number;
- readProcessStart?: (
- pid: number,
- options?: Pick<ProcessStartOptions, 'platform' | 'onNativeFailure'>,
- ) => number | null;
+ readProcessStart?: (pid: number, options?: HolderProcessStartOptions) => ProcessStart | null;
isAlive?: (pid: number) => boolean;
platform?: NodeJS.Platform;
}
@@ -41,7 +34,13 @@ export async function stopServerForRemoval(
const isAlive =
options.isAlive ?? ((pid: number) => isLockProcessRunning(pid, probeOptions(pid)));
const hasProcessEntry = options.isAlive ?? isProcessAlive;
- const state = inspectLock(lockDir, 'server', { isAlive });
+ const platform = options.platform ?? process.platform;
+ const readHolderStart = (
+ pid: number,
+ onNativeFailure?: HolderProcessStartOptions['onNativeFailure'],
+ ): ProcessStart | null =>
+ (options.readProcessStart ?? readHolderProcessStart)(pid, { platform, onNativeFailure });
+ const state = inspectLock(lockDir, 'server', { isAlive, readProcessStart: readHolderStart });
const lockRecovery =
'Quit OpenKnowledge and stop any OpenKnowledge server processes. ' +
`Once you have confirmed they have exited, remove the stale lock file at ${state.lockPath} and retry cleanup.`;
@@ -49,6 +48,7 @@ export async function stopServerForRemoval(
switch (state.status) {
case 'alive':
case 'dead-pid':
+ case 'reused-pid':
case 'foreign-host':
return state.lock.pid;
case 'unverified-owner':
@@ -72,11 +72,8 @@ export async function stopServerForRemoval(
status: state.status,
};
};
- const refusal = (message: string, nativeFailures: NativeFailure[] = []): Error => {
- getCliLogger()?.warn(
- nativeFailures.length > 0 ? { ...logContext(), nativeFailures } : logContext(),
- message,
- );
+ const refusal = (message: string): Error => {
+ getCliLogger()?.warn(logContext(), message);
return new Error(message);
};
if (state.status === 'read-error') {
@@ -101,6 +98,13 @@ export async function stopServerForRemoval(
skipped: `Skipped stale lock at ${state.lockPath}; its recorded local process ${state.lock.pid} has exited. No server was stopped.`,
};
}
+ if (state.status === 'reused-pid') {
+ return {
+ stopped: 0,
+ failed: [],
+ skipped: `Skipped stale lock at ${state.lockPath}; its recorded process ${state.lock.pid} has exited and that process ID now belongs to another program, which was not signalled. No server was stopped.`,
+ };
+ }
if (
state.status === 'corrupt' ||
state.status === 'foreign-host' ||
@@ -137,71 +141,47 @@ export async function stopServerForRemoval(
: `Skipped malformed lock at ${state.lockPath}; process and listener inspection found no live server candidate for this directory. No server was stopped.`,
};
}
- if (state.status === 'alive') {
- const lockStartedAt =
- typeof state.lock.startedAt === 'string' ? Date.parse(state.lock.startedAt) : Number.NaN;
- if (!Number.isFinite(lockStartedAt)) {
- throw refusal(`The server lock has no valid start time. ${lockRecovery}`);
- }
- const nativeFailures: NativeFailure[] = [];
- const platform = options.platform ?? process.platform;
- const processStartedAt = (options.readProcessStart ?? readRemovalProcessStart)(state.lock.pid, {
- platform,
- onNativeFailure: (failure) => {
- nativeFailures.push(failure);
- },
- });
- if (processStartedAt === null) {
- if (!isAlive(state.lock.pid)) return { stopped: 0, failed: [] };
- const componentUnavailable = nativeFailures.some((failure) => {
- switch (failure.kind) {
- case 'unavailable':
- return true;
- case 'query-failed':
- return false;
- default: {
- const exhaustive: never = failure.kind;
- return exhaustive;
- }
- }
- });
- const windowsRecovery =
- nativeFailures.length === 0
- ? 'Stop the server manually. '
- : componentUnavailable
- ? 'Reinstall OpenKnowledge to restore the Windows component that verifies process identity, or stop the server manually. '
- : 'The Windows component loaded but the operating system refused the query, so reinstalling will not help; retry from an account that can inspect that process, or stop the server manually. ';
- const probeRecovery =
- platform === 'linux'
- ? 'Linux requires ps supporting -p and -o lstart= (such as procps). Install a compatible ps and retry, or stop the server manually. '
- : platform === 'win32'
- ? windowsRecovery
- : 'Stop the server manually. ';
- const causeRecord =
- nativeFailures.length > 0
- ? `The failure detail is recorded in the CLI log under ${okUserHomeDisplayPath('logs')}. `
- : '';
- throw refusal(
- `Cannot verify the identity of process ${state.lock.pid} recorded in the server lock at ${state.lockPath}; the OS process-start query failed or is unavailable, so it was not signalled. ` +
- probeRecovery +
- causeRecord +
- 'Confirm the OpenKnowledge server has exited before retrying cleanup after a manual stop.',
- nativeFailures,
- );
- }
- if (processStartedAt > lockStartedAt) {
- throw refusal(
- `Process ${state.lock.pid} started after the server lock was acquired; it was not signalled. ${lockRecovery}`,
- );
- }
- }
const outcome = await runStop({
lockDir,
inspect: () => state,
+ isAlive,
+ readProcessStart: (pid, readOptions) => readHolderStart(pid, readOptions?.onNativeFailure),
+ platform,
force: true,
+ refuseHolderStartedAfterLock: true,
log: () => {},
error: () => {},
});
+ const unverified = outcome.failed.find((failure) => failure.unverified !== undefined)?.unverified;
+ if (unverified !== undefined) {
+ switch (unverified.cause) {
+ case 'foreign-host':
+ throw new Error(
+ describeLockOwnershipRefusal({
+ status: 'foreign-host',
+ lockPath: unverified.lockPath,
+ lock: unverified.lock,
+ }),
+ );
+ case 'unverified-owner':
+ throw new Error(
+ describeLockOwnershipRefusal({
+ status: 'unverified-owner',
+ lockPath: unverified.lockPath,
+ pid: unverified.pid,
+ }),
+ );
+ default:
+ throw new Error(
+ describeUnconfirmedHolderStart(
+ unverified,
+ 'it was not signalled.',
+ `Once you have confirmed no OpenKnowledge server is running for this project, remove ${unverified.lockPath} and retry cleanup.`,
+ getCliLogger() !== undefined,
+ ),
+ );
+ }
+ }
const failed = outcome.failed.map((failure) => ({
pid: failure.target.pid,
error: failure.error,
diff --git a/packages/cli/src/commands/stop.test.ts b/packages/cli/src/commands/stop.test.ts
index 7653f956b..26e50a08a 100644
--- a/packages/cli/src/commands/stop.test.ts
+++ b/packages/cli/src/commands/stop.test.ts
@@ -1,10 +1,25 @@
+import { type ChildProcess, spawn } from 'node:child_process';
+import { once } from 'node:events';
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
import { hostname, tmpdir } from 'node:os';
import { join } from 'node:path';
-import { lockFilePath } from '@inkeep/open-knowledge-server';
+import { okUserHomeDisplayPath } from '@inkeep/open-knowledge-core';
+import {
+ type Config,
+ getMachineId,
+ isProcessAlive,
+ lockFilePath,
+ machineIdFilePath,
+} from '@inkeep/open-knowledge-server';
import { afterEach, describe, expect, test, vi } from 'vitest';
import type { LockState } from './lock-state.ts';
-import { buildStopPlan, formatNoTargetMessage, probeCollabClients, runStop } from './stop.ts';
+import {
+ buildStopPlan,
+ formatNoTargetMessage,
+ probeCollabClients,
+ runStop,
+ stopCommand,
+} from './stop.ts';
function aliveLock(pid: number, port: number): LockState {
return {
@@ -13,12 +28,18 @@ function aliveLock(pid: number, port: number): LockState {
lock: {
pid,
port,
- hostname: 'host',
+ hostname: hostname(),
+ machineId: getMachineId(),
startedAt: '2026-04-16T00:00:00Z',
worktreeRoot: '/x',
},
};
}
+const startedBeforeFixtureLocks = () => ({
+ startedAtMs: Date.parse('2026-01-01T00:00:00Z'),
+ clock: 'creation' as const,
+});
+const verifiedWriter = { isAlive: () => true, readProcessStart: startedBeforeFixtureLocks };
function missing(): LockState {
return { status: 'missing', lockPath: '/tmp/missing.lock' };
}
@@ -33,6 +54,7 @@ function dead(pid: number): LockState {
startedAt: '2026-04-16T00:00:00Z',
worktreeRoot: '/x',
},
+ text: '',
};
}
function foreign(pid: number, port: number): LockState {
@@ -88,6 +110,7 @@ describe('runStop', () => {
const outcome = await runStop({
lockDir: '/tmp/x',
inspect: () => aliveLock(100, 3001),
+ ...verifiedWriter,
kill: (pid, sig) => killed.push([pid, sig]),
log: (msg) => logs.push(msg),
error: () => {},
@@ -103,6 +126,7 @@ describe('runStop', () => {
const outcome = await runStop({
lockDir: '/tmp/x',
inspect: () => aliveLock(100, 3001),
+ ...verifiedWriter,
kill: () => {
throw new Error('EPERM');
},
@@ -143,18 +167,19 @@ describe('buildStopPlan with foreign-host states', () => {
});
describe('runStop with foreign-host states', () => {
- test('foreign-host + locally-live → SIGTERM sent', async () => {
+ test('foreign-host + locally-live → refused without a signal', async () => {
const killed: Array<[number, string]> = [];
const outcome = await runStop({
lockDir: '/tmp/x',
inspect: () => foreign(100, 3001),
kill: (pid, sig) => killed.push([pid, sig]),
- isAlive: () => true,
+ ...verifiedWriter,
log: () => {},
error: () => {},
});
- expect(killed).toEqual([[100, 'SIGTERM']]);
- expect(outcome.stopped.map((t) => t.pid)).toEqual([100]);
+ expect(killed).toEqual([]);
+ expect(outcome.stopped).toEqual([]);
+ expect(outcome.failed.map((failure) => failure.unverified?.cause)).toEqual(['foreign-host']);
});
test('foreign-host + dead PID → no targets, no SIGTERM', async () => {
@@ -190,6 +215,7 @@ describe('runStop channel guard', () => {
const outcome = runStop({
lockDir: '/tmp/channel',
inspect: () => lockOf(channel),
+ ...verifiedWriter,
selfChannel: 'beta',
kill: (pid, sig) => killed.push([pid, sig]),
log: () => {},
@@ -293,6 +319,7 @@ describe('runStop in-use guard', () => {
const oneAlive = () => ({
lockDir: '/tmp/guard',
inspect: () => aliveLock(100, 3001),
+ ...verifiedWriter,
});
test('declines when the target reports live collaboration clients', async () => {
@@ -379,6 +406,7 @@ describe('runStop durable record', () => {
await runStop({
lockDir: '/tmp/target/.ok/local',
inspect: () => aliveLock(100, 3001),
+ ...verifiedWriter,
kill: () => {},
log: () => {},
error: () => {},
@@ -411,6 +439,436 @@ describe('runStop durable record', () => {
});
});
+describe('runStop identity check at the signal', () => {
+ const lockDir = '/tmp/gate/.ok/local';
+ const lockPath = `${lockDir}/server.lock`;
+ const lockWrittenAt = '2026-04-16T00:00:00Z';
+ const startedLongAfterTheLock = () => ({
+ startedAtMs: Date.parse('2026-04-17T12:00:00Z'),
+ clock: 'creation' as const,
+ });
+
+ function lockState(status: 'alive' | 'foreign-host', owner: Record<string, string>): LockState {
+ return {
+ status,
+ lockPath,
+ lock: { pid: 100, port: 3001, startedAt: lockWrittenAt, worktreeRoot: '/x', ...owner },
+ };
+ }
+ const thisMachine = () => lockState('alive', { hostname: hostname(), machineId: getMachineId() });
+ const anotherMachine = () =>
+ lockState('foreign-host', {
+ hostname: 'another-machine.example',
+ machineId: '6f1c2d4e-0000-4000-8000-000000000000',
+ });
+
+ function run(state: LockState, overrides: Partial<Parameters<typeof runStop>[0]>) {
+ const killed: Array<[number, string]> = [];
+ const logs: string[] = [];
+ const errors: string[] = [];
+ const warnings: Array<{ obj: Record<string, unknown>; msg: string }> = [];
+ const logger = {
+ info: () => {},
+ warn: (obj: Record<string, unknown>, msg: string) => warnings.push({ obj, msg }),
+ } as unknown as Parameters<typeof runStop>[0]['logger'];
+ const outcome = runStop({
+ lockDir,
+ inspect: () => state,
+ kill: (pid, sig) => killed.push([pid, sig]),
+ log: (msg) => logs.push(msg),
+ error: (msg) => errors.push(msg),
+ probeClients: async () => null,
+ logger,
+ ...overrides,
+ });
+ return { outcome, killed, logs, errors, warnings };
+ }
+
+ test('a pid now held by a process started after the lock is reported stale and not signalled', async () => {
+ const { outcome, killed, logs, errors } = run(thisMachine(), {
+ isAlive: () => true,
+ readProcessStart: startedLongAfterTheLock,
+ });
+
+ expect(await outcome).toEqual({ stopped: [], failed: [], hadTargets: true });
+ expect(killed).toEqual([]);
+ expect(logs).toEqual([]);
+ expect(errors).toEqual([
+ `Process 100 is no longer the server that wrote ${lockPath}; it was not signalled. Run \`ok clean\` to remove the stale lock.`,
+ ]);
+ });
+
+ test('a lock naming a host with control characters reaches the refusal escaped', async () => {
+ const { outcome, killed, errors } = run(
+ lockState('foreign-host', { hostname: 'evil\nRun: ok clean --all\u001b]52;c;Zm9v\u0007' }),
+ { isAlive: () => true, readProcessStart: startedLongAfterTheLock },
+ );
+
+ expect((await outcome).failed).toHaveLength(1);
+ expect(killed).toEqual([]);
+ const text = errors.join('');
+ expect(text).toContain(
+ 'If evil\\u000aRun: ok clean --all\\u001b]52;c;Zm9v\\u0007 is another machine',
+ );
+ expect(text).not.toContain('\n');
+ expect(text).not.toContain('\u001b');
+ expect(text).not.toContain('\u0007');
+ });
+
+ test('a Beta CLI names its own clean command for a reused pid', async () => {
+ vi.stubEnv('OK_CHANNEL', 'beta');
+ const { outcome, errors } = run(thisMachine(), {
+ isAlive: () => true,
+ readProcessStart: startedLongAfterTheLock,
+ });
+ try {
+ await outcome;
+ expect(errors.at(-1)).toContain('Run `ok-beta clean` to remove the stale lock.');
+ } finally {
+ vi.unstubAllEnvs();
+ }
+ });
+
+ test('a server that exited before the signal is reported, not signalled and not a failure', async () => {
+ const { outcome, killed, logs, errors } = run(thisMachine(), {
+ isAlive: () => false,
+ readProcessStart: startedLongAfterTheLock,
+ });
+
+ expect(await outcome).toEqual({ stopped: [], failed: [], hadTargets: true });
+ expect(killed).toEqual([]);
+ expect(logs).toEqual(['server (pid=100) had already exited']);
+ expect(errors).toEqual([]);
+ });
+
+ test('a holder whose start cannot be read fails without a signal, names the cause and says --force does not help', async () => {
+ const { outcome, killed, errors, warnings } = run(thisMachine(), {
+ isAlive: () => true,
+ readProcessStart: () => null,
+ force: true,
+ platform: 'darwin',
+ });
+ const refusal =
+ `Could not read when process 100 started, so it cannot be confirmed as the server that wrote ${lockPath}; ` +
+ 'it was not signalled, and --force does not override this check. ' +
+ 'macOS reads the process start with ps, and that read failed, as it does inside some sandboxes; retry from a shell where ps can run. ' +
+ 'Before ending process 100 yourself, confirm that it is an OpenKnowledge server. ' +
+ `Once you have confirmed no OpenKnowledge server is running for this project, remove ${lockPath}.`;
+
+ expect(await outcome).toEqual({
+ stopped: [],
+ failed: [
+ {
+ target: { name: 'server', pid: 100, port: 3001 },
+ error: refusal,
+ unverified: expect.objectContaining({
+ cause: 'process-start-unreadable',
+ pid: 100,
+ lockPath,
+ platform: 'darwin',
+ nativeFailures: [],
+ }),
+ },
+ ],
+ hadTargets: true,
+ });
+ expect(killed).toEqual([]);
+ expect(errors).toEqual([`Failed to stop: server (pid=100): ${refusal}`]);
+ expect(warnings).toContainEqual({
+ obj: {
+ lockDir,
+ lockPath,
+ lockPid: 100,
+ platform: 'darwin',
+ cause: 'process-start-unreadable',
+ },
+ msg: 'stop declined: lock holder identity unverified',
+ });
+ });
+
+ test('on Linux an unreadable start points at the /proc files it reads', async () => {
+ const { outcome } = run(thisMachine(), {
+ isAlive: () => true,
+ readProcessStart: () => null,
+ platform: 'linux',
+ });
+
+ expect((await outcome).failed.map((failure) => failure.error)).toEqual([
+ `Could not read when process 100 started, so it cannot be confirmed as the server that wrote ${lockPath}; ` +
+ 'it was not signalled, and --force does not override this check. ' +
+ 'Linux reads the process start from /proc/100/stat and /proc/stat; restore read access to /proc and retry. ' +
+ 'Before ending process 100 yourself, confirm that it is an OpenKnowledge server. ' +
+ `Once you have confirmed no OpenKnowledge server is running for this project, remove ${lockPath}.`,
+ ]);
+ });
+
+ test.each([
+ [
+ 'unavailable',
+ 'the Windows native addon (native-config.win32-x64-msvc.node) did not load or does not export readProcessStart: the addon was not found',
+ 'Reinstall OpenKnowledge to restore the Windows component that verifies process identity. ',
+ ],
+ [
+ 'query-failed',
+ 'readProcessStart failed: OpenProcess: os error 5',
+ 'The Windows component loaded but the operating system refused the query, so reinstalling will not help; retry from an account that can inspect that process. ',
+ ],
+ ] as const)(
+ 'on Windows a %s native read reaches the refusal and the record',
+ async (kind, reason, recovery) => {
+ const { outcome, warnings } = run(thisMachine(), {
+ isAlive: () => true,
+ readProcessStart: (_pid, options) => {
+ options?.onNativeFailure?.({ kind, reason });
+ return null;
+ },
+ platform: 'win32',
+ });
+
+ expect((await outcome).failed.map((failure) => failure.error)).toEqual([
+ `Could not read when process 100 started, so it cannot be confirmed as the server that wrote ${lockPath}; ` +
+ 'it was not signalled, and --force does not override this check. ' +
+ recovery +
+ `The failure detail is recorded in the CLI log under ${okUserHomeDisplayPath('logs')}. ` +
+ 'Before ending process 100 yourself, confirm that it is an OpenKnowledge server. ' +
+ `Once you have confirmed no OpenKnowledge server is running for this project, remove ${lockPath}.`,
+ ]);
+ expect(warnings).toContainEqual({
+ obj: {
+ lockDir,
+ lockPath,
+ lockPid: 100,
+ platform: 'win32',
+ cause: 'process-start-unreadable',
+ nativeFailures: [{ kind, reason }],
+ },
+ msg: 'stop declined: lock holder identity unverified',
+ });
+ },
+ );
+
+ test('on Windows without a logger, as the desktop runs it, a native read failure points at no log', async () => {
+ const { outcome } = run(thisMachine(), {
+ isAlive: () => true,
+ readProcessStart: (_pid, options) => {
+ options?.onNativeFailure?.({
+ kind: 'query-failed',
+ reason: 'readProcessStart failed: OpenProcess: os error 5',
+ });
+ return null;
+ },
+ platform: 'win32',
+ logger: undefined,
+ });
+
+ expect((await outcome).failed.map((failure) => failure.error)).toEqual([
+ `Could not read when process 100 started, so it cannot be confirmed as the server that wrote ${lockPath}; ` +
+ 'it was not signalled, and --force does not override this check. ' +
+ 'The Windows component loaded but the operating system refused the query, so reinstalling will not help; retry from an account that can inspect that process. ' +
+ 'Before ending process 100 yourself, confirm that it is an OpenKnowledge server. ' +
+ `Once you have confirmed no OpenKnowledge server is running for this project, remove ${lockPath}.`,
+ ]);
+ });
+
+ test('a holder whose start cannot be compared with the recorded start fails without a signal', async () => {
+ const state = lockState('alive', { hostname: hostname(), machineId: getMachineId() });
+ if (state.status !== 'alive') throw new Error('expected an alive lock state');
+ state.lock.processStart = { bootId: 'boot-a', startTicks: 5_000 };
+ const { outcome, killed, warnings } = run(state, {
+ isAlive: () => true,
+ readProcessStart: startedLongAfterTheLock,
+ });
+
+ expect((await outcome).failed.map((failure) => failure.error)).toEqual([
+ `The start of process 100 cannot be compared with the start recorded in ${lockPath}, so it cannot be confirmed as the server that wrote the lock; ` +
+ 'it was not signalled, and --force does not override this check. ' +
+ 'Before ending process 100 yourself, confirm that it is an OpenKnowledge server. ' +
+ `Once you have confirmed no OpenKnowledge server is running for this project, remove ${lockPath}.`,
+ ]);
+ expect(killed).toEqual([]);
+ expect(warnings.map((w) => w.obj.cause)).toEqual(['process-start-incomparable']);
+ });
+
+ test('a lock from this machine recording only its host name fails without a signal', async () => {
+ const { outcome, killed, warnings } = run(lockState('alive', { hostname: hostname() }), {
+ isAlive: () => true,
+ readProcessStart: () => ({
+ startedAtMs: Date.parse(lockWrittenAt) - 60_000,
+ clock: 'creation' as const,
+ }),
+ });
+
+ expect((await outcome).failed.map((failure) => failure.error)).toEqual([
+ `The server lock at ${lockPath} records neither a machine ID nor a process start, as locks written by version 0.25.0 and earlier do, so process 100 cannot be confirmed as the server that wrote it; ` +
+ 'it was not signalled, and --force does not override this check. ' +
+ 'Before ending process 100 yourself, confirm that it is an OpenKnowledge server. ' +
+ `Once you have confirmed no OpenKnowledge server is running for this project, remove ${lockPath}. ` +
+ 'A server you start again with this release records its machine ID, so `ok stop` can confirm it.',
+ ]);
+ expect(killed).toEqual([]);
+ expect(warnings.map((w) => w.obj.cause)).toEqual(['writer-identity-unrecorded']);
+ });
+
+ test('a lock whose startedAt is ahead of the clock fails without a signal, naming that', async () => {
+ const { outcome, killed, warnings } = run(
+ lockState('alive', {
+ hostname: hostname(),
+ machineId: getMachineId(),
+ startedAt: '2099-01-01T00:00:00Z',
+ }),
+ {
+ isAlive: () => true,
+ readProcessStart: () => ({ startedAtMs: Date.now() - 60_000, clock: 'creation' as const }),
+ },
+ );
+
+ expect((await outcome).failed.map((failure) => failure.error)).toEqual([
+ `The server lock at ${lockPath} records a start later than the current time on this machine, so process 100 cannot be confirmed as the server that wrote it; ` +
+ 'it was not signalled, and --force does not override this check. ' +
+ 'Before ending process 100 yourself, confirm that it is an OpenKnowledge server. ' +
+ `Once you have confirmed no OpenKnowledge server is running for this project, remove ${lockPath}.`,
+ ]);
+ expect(killed).toEqual([]);
+ expect(warnings.map((w) => w.obj.cause)).toEqual(['recorded-start-in-future']);
+ });
+
+ test('a holder whose start contradicts the recorded start fails without a signal', async () => {
+ const state = lockState('alive', {
+ hostname: hostname(),
+ machineId: getMachineId(),
+ });
+ if (state.status !== 'alive') throw new Error('expected an alive lock state');
+ state.lock.processStart = { startedAtMs: Date.parse('2026-04-17T12:00:00Z') + 60_000 };
+ const { outcome, killed, warnings } = run(state, {
+ isAlive: () => true,
+ readProcessStart: startedLongAfterTheLock,
+ });
+ const refusal =
+ `Process 100 started before the start recorded in ${lockPath}, so it cannot be confirmed as the server that wrote the lock; ` +
+ 'it was not signalled, and --force does not override this check. ' +
+ 'Before ending process 100 yourself, confirm that it is an OpenKnowledge server. ' +
+ `Once you have confirmed no OpenKnowledge server is running for this project, remove ${lockPath}.`;
+
+ expect(await outcome).toEqual({
+ stopped: [],
+ failed: [
+ {
+ target: { name: 'server', pid: 100, port: 3001 },
+ error: refusal,
+ unverified: expect.objectContaining({ cause: 'process-start-mismatch', pid: 100 }),
+ },
+ ],
+ hadTargets: true,
+ });
+ expect(killed).toEqual([]);
+ expect(warnings.map((w) => w.obj.cause)).toEqual(['process-start-mismatch']);
+ });
+
+ test("a local process named by another machine's lock fails without a signal and points at the owning machine", async () => {
+ const { outcome, killed, errors, warnings } = run(anotherMachine(), {
+ isAlive: () => true,
+ readProcessStart: startedLongAfterTheLock,
+ platform: 'darwin',
+ });
+ const refusal =
+ `The server lock at ${lockPath} records a machine ID different from the one in ${machineIdFilePath()}, so process 100, which is running on this machine, cannot be confirmed as the server that wrote it; ` +
+ 'it was not signalled, and --force does not override this check. ' +
+ 'If the server was started on this machine from another account, run `ok stop` as that account. ' +
+ 'If it was started from this account with a different home folder, run `ok stop` with `HOME` set to that folder. ' +
+ 'Otherwise stop it from the machine that owns the lock. ' +
+ 'This lock records no process start; a server started by this release records one, which lets `ok stop` confirm it from any home folder.';
+
+ expect(await outcome).toEqual({
+ stopped: [],
+ failed: [
+ {
+ target: { name: 'server', pid: 100, port: 3001 },
+ error: refusal,
+ unverified: expect.objectContaining({ cause: 'foreign-host', pid: 100 }),
+ },
+ ],
+ hadTargets: true,
+ });
+ expect(killed).toEqual([]);
+ expect(errors).toEqual([`Failed to stop: server (pid=100): ${refusal}`]);
+ expect(warnings.map((w) => w.obj.cause)).toEqual(['foreign-host']);
+ });
+
+ test("another machine's lock keeps the owning-machine guidance when the local start cannot be read", async () => {
+ const { outcome, warnings } = run(anotherMachine(), {
+ isAlive: () => true,
+ readProcessStart: () => null,
+ platform: 'darwin',
+ });
+
+ expect((await outcome).failed.map((failure) => failure.error)).toEqual([
+ `The server lock at ${lockPath} records a machine ID different from the one in ${machineIdFilePath()}, so process 100, which is running on this machine, cannot be confirmed as the server that wrote it; ` +
+ 'it was not signalled, and --force does not override this check. ' +
+ 'If the server was started on this machine from another account, run `ok stop` as that account. ' +
+ 'If it was started from this account with a different home folder, run `ok stop` with `HOME` set to that folder. ' +
+ 'Otherwise stop it from the machine that owns the lock. ' +
+ 'This lock records no process start; a server started by this release records one, which lets `ok stop` confirm it from any home folder.',
+ ]);
+ expect(warnings.map((w) => w.obj.cause)).toEqual(['foreign-host']);
+ });
+
+ test.each([
+ ['darwin', 'HOME'],
+ ['win32', 'USERPROFILE'],
+ ] as const)(
+ "another machine's lock that records a start is refused on %s naming %s and without the restart advice",
+ async (platform, homeVariable) => {
+ const state = anotherMachine();
+ if (state.status !== 'foreign-host') throw new Error('expected a foreign-host lock state');
+ state.lock.processStart = { startedAtMs: Date.parse('2026-04-17T12:00:00Z') + 60_000 };
+ const { outcome, killed, warnings } = run(state, {
+ isAlive: () => true,
+ readProcessStart: startedLongAfterTheLock,
+ platform,
+ });
+
+ expect((await outcome).failed.map((failure) => failure.error)).toEqual([
+ `The server lock at ${lockPath} records a machine ID different from the one in ${machineIdFilePath()}, so process 100, which is running on this machine, cannot be confirmed as the server that wrote it; ` +
+ 'it was not signalled, and --force does not override this check. ' +
+ 'If the server was started on this machine from another account, run `ok stop` as that account. ' +
+ `If it was started from this account with a different home folder, run \`ok stop\` with \`${homeVariable}\` set to that folder. ` +
+ 'Otherwise stop it from the machine that owns the lock.',
+ ]);
+ expect(killed).toEqual([]);
+ expect(warnings.map((w) => w.obj.cause)).toEqual(['foreign-host']);
+ },
+ );
+
+ test('a lock naming only another host is refused with both host names and the advice for a renamed host', async () => {
+ const state = lockState('foreign-host', { hostname: 'another-machine.example' });
+ if (state.status !== 'foreign-host') throw new Error('expected a foreign-host lock state');
+ state.lock.processStart = { startedAtMs: Date.parse('2026-04-17T12:00:00Z') + 60_000 };
+ const { outcome, killed, warnings } = run(state, {
+ isAlive: () => true,
+ readProcessStart: startedLongAfterTheLock,
+ });
+
+ expect((await outcome).failed.map((failure) => failure.error)).toEqual([
+ `The server lock at ${lockPath} records only the host another-machine.example, not this machine's ${hostname()}, and the start of process 100, which is running on this machine, does not confirm it as the server that wrote the lock; ` +
+ 'it was not signalled, and --force does not override this check. ' +
+ 'If another-machine.example is another machine, stop the server there. ' +
+ `If it is an earlier name of this machine, confirm that process 100 is an OpenKnowledge server before ending it yourself, and once no OpenKnowledge server is running for this project, remove ${lockPath}.`,
+ ]);
+ expect(killed).toEqual([]);
+ expect(warnings.map((w) => w.obj.cause)).toEqual(['foreign-host']);
+ });
+
+ test("an earlier-started local process named by another machine's lock is not signalled", async () => {
+ const { outcome, killed } = run(anotherMachine(), {
+ isAlive: () => true,
+ readProcessStart: startedBeforeFixtureLocks,
+ });
+
+ expect((await outcome).stopped).toEqual([]);
+ expect(killed).toEqual([]);
+ });
+});
+
describe('formatNoTargetMessage', () => {
test('nothing here AND nothing anywhere says both', () => {
const msg = formatNoTargetMessage('/proj', 0);
@@ -470,7 +928,7 @@ describe('probeCollabClients failure record', () => {
pid: process.pid,
port: 24999,
hostname: hostname(),
- startedAt: '2026-08-18T00:00:00Z',
+ startedAt: new Date().toISOString(),
worktreeRoot: dir,
}),
'utf-8',
@@ -537,3 +995,134 @@ describe('probeCollabClients failure record', () => {
expect(records).toEqual([]);
});
});
+
+describe('runStop on a live lock that records no owning machine', () => {
+ let lockDir: string | undefined;
+
+ afterEach(() => {
+ if (lockDir !== undefined) rmSync(lockDir, { recursive: true, force: true });
+ lockDir = undefined;
+ });
+
+ test('refuses it without a signal, even with --force, and says not to remove the lock', async () => {
+ lockDir = mkdtempSync(join(tmpdir(), 'ok-stop-ownerless-'));
+ const lockPath = lockFilePath(lockDir, 'server');
+ writeFileSync(
+ lockPath,
+ JSON.stringify({
+ pid: process.pid,
+ port: 24998,
+ startedAt: new Date().toISOString(),
+ worktreeRoot: lockDir,
+ }),
+ 'utf-8',
+ );
+ const killed: Array<[number, string]> = [];
+ const errors: string[] = [];
+ const warnings: Array<{ obj: Record<string, unknown>; msg: string }> = [];
+ const logger = {
+ info: () => {},
+ warn: (obj: Record<string, unknown>, msg: string) => warnings.push({ obj, msg }),
+ } as unknown as Parameters<typeof runStop>[0]['logger'];
+
+ const outcome = await runStop({
+ lockDir,
+ force: true,
+ kill: (pid, sig) => killed.push([pid, sig]),
+ log: () => {},
+ error: (msg) => errors.push(msg),
+ probeClients: async () => null,
+ logger,
+ });
+
+ const refusal =
+ `The server lock at ${lockPath} does not record its owning machine, and its recorded process ${process.pid} is still running on this machine, so it cannot be confirmed as the server that wrote the lock; ` +
+ 'it was not signalled, and --force does not override this check. ' +
+ `Check whether process ${process.pid} is an OpenKnowledge server, and if it is, stop it yourself. ` +
+ 'Do not remove the lock file while that process is running.';
+
+ expect(outcome).toEqual({
+ stopped: [],
+ failed: [
+ {
+ target: { name: 'server', pid: process.pid },
+ error: refusal,
+ unverified: { cause: 'unverified-owner', pid: process.pid, lockPath },
+ },
+ ],
+ hadTargets: true,
+ });
+ expect(killed).toEqual([]);
+ expect(errors).toEqual([`Failed to stop: server (pid=${process.pid}): ${refusal}`]);
+ expect(warnings).toContainEqual({
+ obj: {
+ lockDir,
+ lockPath,
+ lockPid: process.pid,
+ platform: process.platform,
+ cause: 'unverified-owner',
+ },
+ msg: 'stop declined: lock holder identity unverified',
+ });
+ });
+});
+
+describe('ok stop on a project whose live lock records no owning machine', () => {
+ let project: string | undefined;
+ let holder: ChildProcess | undefined;
+ const savedExitCode = process.exitCode;
+
+ afterEach(async () => {
+ process.exitCode = savedExitCode;
+ vi.restoreAllMocks();
+ if (holder !== undefined && holder.exitCode === null && holder.signalCode === null) {
+ const exited = once(holder, 'exit');
+ holder.kill('SIGKILL');
+ await exited;
+ }
+ holder = undefined;
+ if (project !== undefined) rmSync(project, { recursive: true, force: true });
+ project = undefined;
+ });
+
+ test.each([
+ ['directory', []],
+ ['directory', ['--force']],
+ ['no-argument', []],
+ ['no-argument', ['--force']],
+ ] as const)('exits 1 and leaves the process running (%s form, flags %j)', async (form, flags) => {
+ project = mkdtempSync(join(tmpdir(), 'ok-stop-command-ownerless-'));
+ holder = spawn(process.execPath, ['-e', 'setInterval(() => {}, 1000)'], {
+ stdio: 'ignore',
+ });
+ await once(holder, 'spawn');
+ const pid = holder.pid;
+ if (pid === undefined) throw new Error('holder did not start');
+ const lockDir = join(project, '.ok', 'local');
+ mkdirSync(lockDir, { recursive: true });
+ writeFileSync(lockFilePath(lockDir, 'server'), JSON.stringify({ pid }), 'utf-8');
+ const errors: string[] = [];
+ vi.spyOn(console, 'error').mockImplementation((msg: unknown) => {
+ errors.push(String(msg));
+ });
+ vi.spyOn(console, 'log').mockImplementation(() => {});
+ process.exitCode = undefined;
+
+ if (form === 'directory') {
+ await stopCommand(() => {
+ throw new Error('the directory form reads no config');
+ }).parseAsync([project, ...flags], { from: 'user' });
+ } else {
+ vi.spyOn(process, 'cwd').mockReturnValue(project);
+ await stopCommand(() => ({}) as Config).parseAsync([...flags], { from: 'user' });
+ }
+
+ expect(process.exitCode).toBe(1);
+ expect(isProcessAlive(pid)).toBe(true);
+ expect(errors).toEqual([
+ expect.stringContaining(
+ `process ${pid} is still running on this machine, so it cannot be confirmed as the server that wrote the lock; it was not signalled`,
+ ),
+ ]);
+ });
+});
diff --git a/packages/cli/src/commands/stop.ts b/packages/cli/src/commands/stop.ts
index 63e2253a5..3c37be67c 100644
--- a/packages/cli/src/commands/stop.ts
+++ b/packages/cli/src/commands/stop.ts
@@ -7,17 +7,35 @@ import {
resolveDesktopProductName,
} from '@inkeep/open-knowledge-core';
import {
+ assessLockHolder,
type Config,
isProcessAlive,
lockBaseUrl,
+ type ProcessLockMetadata,
+ type ProcessStart,
+ type ProcessStartCause,
resolveLockDir,
+ type UnverifiedLockHolderCause,
} from '@inkeep/open-knowledge-server';
import { Command } from 'commander';
import type { Logger as PinoLoggerInstance } from 'pino';
import { getCliLogger } from '../cli-logger.ts';
import { getInvocationCwd } from '../project-anchor.ts';
+import { escapeDisplayPath } from '../utils/escape-display-path.ts';
import { discoverLockDirs } from '../utils/process-scan.ts';
-import { inspectLock, type LockState } from './lock-state.ts';
+import {
+ describeHolderStartReadFailure,
+ type HolderProcessStartOptions,
+ type NativeFailure,
+ readHolderProcessStart,
+} from './holder-process-start.ts';
+import {
+ describeForeignOwner,
+ describeOtherHomeFolderRemedy,
+ describeUnrecordedOwner,
+ inspectLock,
+ type LockState,
+} from './lock-state.ts';
import { runPs } from './ps.ts';
const CLIENT_PROBE_TIMEOUT_MS = 1500;
@@ -64,14 +82,18 @@ export async function probeCollabClients(
}
}
-interface StopTargetPlan {
+interface StopTarget {
name: 'server';
pid: number;
+}
+
+interface StopTargetPlan extends StopTarget {
port: number;
}
interface StopPlan {
targets: StopTargetPlan[];
+ lock?: ProcessLockMetadata;
}
interface BuildStopPlanDeps {
@@ -80,10 +102,10 @@ interface BuildStopPlanDeps {
export function buildStopPlan(server: LockState, deps: BuildStopPlanDeps = {}): StopPlan {
const isAlive = deps.isAlive ?? isProcessAlive;
+ if (!isStoppableState(server, isAlive)) return { targets: [] };
return {
- targets: isStoppableState(server, isAlive)
- ? [{ name: 'server', pid: server.lock.pid, port: server.lock.port }]
- : [],
+ targets: [{ name: 'server', pid: server.lock.pid, port: server.lock.port }],
+ lock: server.lock,
};
}
@@ -93,16 +115,44 @@ interface RunStopDeps {
inspect?: () => LockState;
kill?: (pid: number, signal: NodeJS.Signals) => void;
isAlive?: (pid: number) => boolean;
+ readProcessStart?: (pid: number, options?: HolderProcessStartOptions) => ProcessStart | null;
log?: (msg: string) => void;
error?: (msg: string) => void;
probeClients?: (lockDir: string, logger?: PinoLoggerInstance) => Promise<number | null>;
logger?: PinoLoggerInstance;
selfChannel?: 'stable' | 'beta' | 'dev';
+ platform?: NodeJS.Platform;
+ refuseHolderStartedAfterLock?: boolean;
+}
+
+interface UnconfirmedStartHolder {
+ readonly cause: ProcessStartCause;
+ readonly pid: number;
+ readonly lockPath: string;
+ readonly platform: NodeJS.Platform;
+ readonly nativeFailures: readonly NativeFailure[];
+}
+
+type UnverifiedHolder =
+ | UnconfirmedStartHolder
+ | {
+ readonly cause: 'foreign-host';
+ readonly pid: number;
+ readonly lockPath: string;
+ readonly lock: ProcessLockMetadata;
+ readonly platform: NodeJS.Platform;
+ }
+ | { readonly cause: 'unverified-owner'; readonly pid: number; readonly lockPath: string };
+
+interface StopFailure {
+ target: StopTarget;
+ error: string;
+ unverified?: UnverifiedHolder;
}
interface StopOutcome {
stopped: StopTargetPlan[];
- failed: Array<{ target: StopTargetPlan; error: string }>;
+ failed: StopFailure[];
hadTargets: boolean;
declined?: { clients: number } | { otherChannel: string };
}
@@ -126,11 +176,63 @@ export async function runStop(deps: RunStopDeps): Promise<StopOutcome> {
const error = deps.error ?? ((msg) => console.error(msg));
const probeClients = deps.probeClients ?? probeCollabClients;
const logger = deps.logger ?? getCliLogger();
+ const platform = deps.platform ?? process.platform;
const serverState = inspect();
const plan = buildStopPlan(serverState, { isAlive: deps.isAlive });
+ const { lock } = plan;
+ const stopped: StopTargetPlan[] = [];
+ const failed: StopFailure[] = [];
+ const refuse = (target: StopTarget, holder: UnverifiedHolder): void => {
+ logger?.warn(
+ {
+ lockDir: deps.lockDir,
+ lockPath: holder.lockPath,
+ lockPid: holder.pid,
+ platform,
+ cause: holder.cause,
+ ...('nativeFailures' in holder && holder.nativeFailures.length > 0
+ ? { nativeFailures: holder.nativeFailures }
+ : {}),
+ },
+ 'stop declined: lock holder identity unverified',
+ );
+ failed.push({
+ target,
+ error: describeUnverifiedHolderForStop(holder, logger !== undefined),
+ unverified: holder,
+ });
+ };
+ const report = (): StopOutcome => {
+ if (stopped.length > 0) {
+ const rendered = stopped.map((t) => `${t.name} (pid=${t.pid}, port=${t.port})`).join(', ');
+ log(`Stopped: ${rendered}`);
+ }
+ logger?.info(
+ {
+ lockDir: deps.lockDir,
+ signalled: stopped.map((t) => ({ name: t.name, pid: t.pid, port: t.port })),
+ failed: failed.map(({ target, error: msg }) => ({ pid: target.pid, error: msg })),
+ forced: deps.force === true,
+ },
+ 'stop signalled processes',
+ );
+ if (failed.length > 0) {
+ const rendered = failed
+ .map(({ target, error: msg }) => `${target.name} (pid=${target.pid}): ${msg}`)
+ .join('; ');
+ error(`Failed to stop: ${rendered}`);
+ }
+ return { stopped, failed, hadTargets: true };
+ };
+
+ if (serverState.status === 'unverified-owner') {
+ const { pid, lockPath } = serverState;
+ refuse({ name: 'server', pid }, { cause: 'unverified-owner', pid, lockPath });
+ return report();
+ }
- if (plan.targets.length === 0) {
+ if (lock === undefined) {
log('No running open-knowledge processes.');
logger?.info({ lockDir: deps.lockDir, targets: 0 }, 'stop found nothing to signal');
return { stopped: [], failed: [], hadTargets: false };
@@ -178,38 +280,170 @@ export async function runStop(deps: RunStopDeps): Promise<StopOutcome> {
}
}
- const stopped: StopTargetPlan[] = [];
- const failed: Array<{ target: StopTargetPlan; error: string }> = [];
for (const target of plan.targets) {
- try {
- kill(target.pid, 'SIGTERM');
- stopped.push(target);
- } catch (err) {
- failed.push({ target, error: err instanceof Error ? err.message : String(err) });
+ const nativeFailures: NativeFailure[] = [];
+ const holderStart = (deps.readProcessStart ?? readHolderProcessStart)(target.pid, {
+ platform,
+ onNativeFailure: (failure) => {
+ nativeFailures.push(failure);
+ },
+ });
+ const assessment = assessLockHolder(lock, {
+ isAlive: deps.isAlive ?? isProcessAlive,
+ readProcessStart: () => holderStart,
+ refuseHolderStartedAfterLock: deps.refuseHolderStartedAfterLock,
+ });
+ switch (assessment.verdict) {
+ case 'writer':
+ try {
+ kill(target.pid, 'SIGTERM');
+ stopped.push(target);
+ } catch (err) {
+ failed.push({ target, error: err instanceof Error ? err.message : String(err) });
+ }
+ break;
+ case 'gone':
+ log(`${target.name} (pid=${target.pid}) had already exited`);
+ break;
+ case 'replaced':
+ error(
+ `Process ${target.pid} is no longer the server that wrote ${serverState.lockPath}; it was not signalled. ` +
+ `Run \`${currentCliCommandName()} clean\` to remove the stale lock.`,
+ );
+ break;
+ case 'unverified':
+ refuse(
+ target,
+ unverifiedHolder(assessment.cause, {
+ pid: target.pid,
+ lockPath: serverState.lockPath,
+ lock,
+ platform,
+ nativeFailures,
+ }),
+ );
+ break;
+ default: {
+ const exhaustive: never = assessment;
+ return exhaustive;
+ }
}
}
- if (stopped.length > 0) {
- const rendered = stopped.map((t) => `${t.name} (pid=${t.pid}, port=${t.port})`).join(', ');
- log(`Stopped: ${rendered}`);
+ return report();
+}
+
+function unverifiedHolder(
+ cause: UnverifiedLockHolderCause,
+ found: {
+ pid: number;
+ lockPath: string;
+ lock: ProcessLockMetadata;
+ platform: NodeJS.Platform;
+ nativeFailures: readonly NativeFailure[];
+ },
+): UnverifiedHolder {
+ const { pid, lockPath } = found;
+ switch (cause) {
+ case 'foreign-host':
+ return { cause, pid, lockPath, lock: found.lock, platform: found.platform };
+ case 'unverified-owner':
+ return { cause, pid, lockPath };
+ default:
+ return {
+ cause,
+ pid,
+ lockPath,
+ platform: found.platform,
+ nativeFailures: found.nativeFailures,
+ };
}
- logger?.info(
- {
- lockDir: deps.lockDir,
- signalled: stopped.map((t) => ({ name: t.name, pid: t.pid, port: t.port })),
- failed: failed.map(({ target, error: msg }) => ({ pid: target.pid, error: msg })),
- forced: deps.force === true,
- },
- 'stop signalled processes',
- );
- if (failed.length > 0) {
- const rendered = failed
- .map(({ target, error: msg }) => `${target.name} (pid=${target.pid}): ${msg}`)
- .join('; ');
- error(`Failed to stop: ${rendered}`);
+}
+
+const STOP_NOT_SIGNALLED = 'it was not signalled, and --force does not override this check.';
+
+function describeUnverifiedHolderForStop(holder: UnverifiedHolder, causeRecorded: boolean): string {
+ const { pid, lockPath } = holder;
+ switch (holder.cause) {
+ case 'foreign-host': {
+ const { lock } = holder;
+ if (typeof lock.machineId !== 'string') {
+ return (
+ `The server lock at ${lockPath} ${describeForeignOwner(lock)}, and the start of process ${pid}, which is running on this machine, does not confirm it as the server that wrote the lock; ` +
+ `${STOP_NOT_SIGNALLED} If ${escapeDisplayPath(lock.hostname)} is another machine, stop the server there. ` +
+ `If it is an earlier name of this machine, confirm that process ${pid} is an OpenKnowledge server before ending it yourself, and once no OpenKnowledge server is running for this project, remove ${lockPath}.`
+ );
+ }
+ return (
+ `The server lock at ${lockPath} ${describeForeignOwner(lock)}, so process ${pid}, which is running on this machine, cannot be confirmed as the server that wrote it; ` +
+ `${STOP_NOT_SIGNALLED} ${describeOtherHomeFolderRemedy(lock, holder.platform)}`
+ );
+ }
+ case 'unverified-owner':
+ return (
+ `The server lock at ${lockPath} ${describeUnrecordedOwner(pid)}, so it cannot be confirmed as the server that wrote the lock; ` +
+ `${STOP_NOT_SIGNALLED} Check whether process ${pid} is an OpenKnowledge server, and if it is, stop it yourself. ` +
+ 'Do not remove the lock file while that process is running.'
+ );
+ default: {
+ const refusal = describeUnconfirmedHolderStart(
+ holder,
+ STOP_NOT_SIGNALLED,
+ `Once you have confirmed no OpenKnowledge server is running for this project, remove ${lockPath}.`,
+ causeRecorded,
+ );
+ if (holder.cause !== 'writer-identity-unrecorded') return refusal;
+ return `${refusal} A server you start again with this release records its machine ID, so \`${currentCliCommandName()} stop\` can confirm it.`;
+ }
}
+}
- return { stopped, failed, hadTargets: true };
+export function describeUnconfirmedHolderStart(
+ holder: UnconfirmedStartHolder,
+ notSignalled: string,
+ afterManualStop: string,
+ causeRecorded: boolean,
+): string {
+ const { pid, lockPath, platform, nativeFailures } = holder;
+ const confirmFirst = `Before ending process ${pid} yourself, confirm that it is an OpenKnowledge server.`;
+ switch (holder.cause) {
+ case 'process-start-unreadable':
+ return (
+ `Could not read when process ${pid} started, so it cannot be confirmed as the server that wrote ${lockPath}; ` +
+ `${notSignalled} ` +
+ describeHolderStartReadFailure({ platform, pid, nativeFailures, causeRecorded }) +
+ `${confirmFirst} ${afterManualStop}`
+ );
+ case 'process-start-incomparable':
+ return (
+ `The start of process ${pid} cannot be compared with the start recorded in ${lockPath}, so it cannot be confirmed as the server that wrote the lock; ` +
+ `${notSignalled} ${confirmFirst} ${afterManualStop}`
+ );
+ case 'process-start-mismatch':
+ return (
+ `Process ${pid} started before the start recorded in ${lockPath}, so it cannot be confirmed as the server that wrote the lock; ` +
+ `${notSignalled} ${confirmFirst} ${afterManualStop}`
+ );
+ case 'process-started-after-lock':
+ return (
+ `Process ${pid} started after the server lock at ${lockPath} was written, so it cannot be confirmed as the server that wrote the lock; ` +
+ `${notSignalled} ${confirmFirst} ${afterManualStop}`
+ );
+ case 'recorded-start-in-future':
+ return (
+ `The server lock at ${lockPath} records a start later than the current time on this machine, so process ${pid} cannot be confirmed as the server that wrote it; ` +
+ `${notSignalled} ${confirmFirst} ${afterManualStop}`
+ );
+ case 'writer-identity-unrecorded':
+ return (
+ `The server lock at ${lockPath} records neither a machine ID nor a process start, as locks written by version 0.25.0 and earlier do, so process ${pid} cannot be confirmed as the server that wrote it; ` +
+ `${notSignalled} ${confirmFirst} ${afterManualStop}`
+ );
+ default: {
+ const exhaustive: never = holder.cause;
+ return exhaustive;
+ }
+ }
}
function isStoppableState(
@@ -236,9 +470,13 @@ async function findLockDirByNumber(
return pidMatch;
}
+function setExitCodeForStop(outcome: StopOutcome): void {
+ if (outcome.failed.length > 0 || outcome.declined !== undefined) process.exitCode = 1;
+}
+
async function executeStop(lockDir: string, force: boolean): Promise<StopOutcome> {
const outcome = await runStop({ lockDir, force });
- if (outcome.failed.length > 0 || outcome.declined !== undefined) process.exitCode = 1;
+ setExitCodeForStop(outcome);
return outcome;
}
@@ -291,15 +529,11 @@ export function stopCommand(getConfig: () => Config): Command {
if (target === undefined) {
getConfig();
const lockDir = resolveLockDir(process.cwd());
- const outcome = await runStop({ lockDir, force, log: () => {} });
+ const buffered: string[] = [];
+ const outcome = await runStop({ lockDir, force, log: (msg) => buffered.push(msg) });
if (outcome.hadTargets) {
- if (outcome.stopped.length > 0) {
- const rendered = outcome.stopped
- .map((t) => `${t.name} (pid=${t.pid}, port=${t.port})`)
- .join(', ');
- console.log(`Stopped: ${rendered}`);
- }
- if (outcome.failed.length > 0 || outcome.declined !== undefined) process.exitCode = 1;
+ for (const line of buffered) console.log(line);
+ setExitCodeForStop(outcome);
} else {
const others = await countOtherRunningServers(lockDir);
console.log(formatNoTargetMessage(process.cwd(), others, { listingFollows: others > 0 }));
@@ -339,7 +573,7 @@ export function stopCommand(getConfig: () => Config): Command {
const lockDir = resolveLockDir(targetDir);
const buffered: string[] = [];
const outcome = await runStop({ lockDir, force, log: (msg) => buffered.push(msg) });
- if (outcome.failed.length > 0 || outcome.declined !== undefined) process.exitCode = 1;
+ setExitCodeForStop(outcome);
if (outcome.hadTargets) {
for (const line of buffered) console.log(line);
} else {
diff --git a/packages/cli/src/commands/uninstall-lock-recovery.test.ts b/packages/cli/src/commands/uninstall-lock-recovery.test.ts
index f055ddf3a..9b32c46f3 100644
--- a/packages/cli/src/commands/uninstall-lock-recovery.test.ts
+++ b/packages/cli/src/commands/uninstall-lock-recovery.test.ts
@@ -12,6 +12,7 @@ import {
import { hostname, tmpdir } from 'node:os';
import { dirname, join } from 'node:path';
import {
+ getMachineId,
isProcessAlive,
type LockProcessScan,
scanLockProcesses,
@@ -109,7 +110,7 @@ describe.skipIf(process.platform === 'win32')('uninstall lock recovery in an iso
} else livePid = child.pid;
raw = JSON.stringify({
pid: child.pid,
- ...(kind === 'pid-only' ? {} : { hostname: hostname() }),
+ ...(kind === 'pid-only' ? {} : { hostname: hostname(), machineId: getMachineId() }),
port: 0,
startedAt: new Date().toISOString(),
});
diff --git a/packages/cli/src/index.ts b/packages/cli/src/index.ts
index f85985546..3979b68f0 100644
--- a/packages/cli/src/index.ts
+++ b/packages/cli/src/index.ts
@@ -36,6 +36,7 @@ export {
isOwnManagedEntry,
type McpInstallOptions,
} from './commands/editors.ts';
+export { registerWindowsProcessStartReader } from './commands/holder-process-start.ts';
export {
classifyExistingMcpEntry,
detectInstalledEditors,
diff --git a/packages/cli/src/project-anchor.test.ts b/packages/cli/src/project-anchor.test.ts
index af08ad627..5deedc062 100644
--- a/packages/cli/src/project-anchor.test.ts
+++ b/packages/cli/src/project-anchor.test.ts
@@ -2,6 +2,7 @@ import { spawn } from 'node:child_process';
import { mkdirSync, mkdtempSync, realpathSync, rmSync, writeFileSync } from 'node:fs';
import { hostname, tmpdir } from 'node:os';
import { join } from 'node:path';
+import { getMachineId } from '@inkeep/open-knowledge-server';
import { afterAll, afterEach, describe, expect, test } from 'vitest';
import { getInvocationCwd, recordInvocationCwd, resolveProjectAnchor } from './project-anchor.ts';
@@ -197,7 +198,13 @@ describe('CLI preAction project anchoring (cold spawn)', () => {
try {
writeFileSync(
join(targetDir, '.ok', 'local', 'server.lock'),
- JSON.stringify({ pid: sleeper.pid, port: 4242, hostname: hostname() }),
+ JSON.stringify({
+ pid: sleeper.pid,
+ port: 4242,
+ hostname: hostname(),
+ machineId: getMachineId(),
+ startedAt: new Date().toISOString(),
+ }),
);
const { exitCode, stdout, stderr } = spawnCli(['--cwd', sub, 'stop', './target']);
diff --git a/packages/cli/tests/e2e/cli-linux-e2e.ts b/packages/cli/tests/e2e/cli-linux-e2e.ts
index 7d978a48e..033ad0602 100644
--- a/packages/cli/tests/e2e/cli-linux-e2e.ts
+++ b/packages/cli/tests/e2e/cli-linux-e2e.ts
@@ -1,12 +1,28 @@
import { type ChildProcess, execFileSync, spawn, spawnSync } from 'node:child_process';
-import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
+import {
+ existsSync,
+ mkdirSync,
+ mkdtempSync,
+ readFileSync,
+ realpathSync,
+ rmSync,
+ writeFileSync,
+} from 'node:fs';
import { connect } from 'node:net';
import { tmpdir } from 'node:os';
import { dirname, join, resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
+import { getMachineId } from '@inkeep/open-knowledge-server';
import { Client } from '@modelcontextprotocol/sdk/client/index.js';
import { StdioClientTransport } from '@modelcontextprotocol/sdk/client/stdio.js';
import { afterAll, beforeAll, describe, expect, test } from 'vitest';
+import {
+ legacyServerLock,
+ lockWrittenBeforeProcessStarted,
+ ownedProcessHasExited,
+ startOwnedProcess,
+ stopOwnedProcess,
+} from '../../../../test-support/owned-process.test-helper.ts';
import { installPackedCli } from './packed-install.test-helper';
const HERE = dirname(fileURLToPath(import.meta.url));
@@ -435,4 +451,88 @@ describe(`CLI Linux e2e (${SUT_MODE} SUT)`, () => {
const lockGone = await waitFor(() => !existsSync(H.lockPath), { timeoutMs: 5_000 });
expect(lockGone, 'server.lock was not released on shutdown').toBe(true);
}, 20_000);
+
+ test('8. a lock whose pid now belongs to another process reads stale, is pruned, and is never signalled', async () => {
+ const projectDir = realpathSync(mkdtempSync(join(tmpdir(), 'ok-e2e-reused-pid-')));
+ const lockPath = join(projectDir, '.ok', 'local', 'server.lock');
+ const standIn = await startOwnedProcess();
+ let server: ChildProcess | null = null;
+ let serverLog = '';
+ try {
+ execFileSync('git', ['init', '-q'], { cwd: projectDir });
+ expect(runOk(['init', '--no-mcp'], { cwd: projectDir, timeoutMs: 60_000 }).status).toBe(0);
+ mkdirSync(dirname(lockPath), { recursive: true });
+ writeFileSync(
+ lockPath,
+ JSON.stringify(
+ legacyServerLock({
+ pid: standIn.pid,
+ startedAt: lockWrittenBeforeProcessStarted(standIn),
+ machineId: getMachineId(),
+ worktreeRoot: projectDir,
+ }),
+ ),
+ { mode: 0o600 },
+ );
+
+ const status = runOk(['status', '--json'], { cwd: projectDir });
+ expect(status.status, status.stderr).toBe(0);
+ const report = JSON.parse(status.stdout) as { server: { state: string; alive: boolean } };
+ expect(report.server).toMatchObject({ state: 'reused-pid', alive: false });
+
+ const stop = runOk(['stop', projectDir], { cwd: projectDir });
+ expect(stop.status, stop.stderr).toBe(0);
+ expect(
+ await waitFor(() => ownedProcessHasExited(standIn), { timeoutMs: 2_000 }),
+ 'ok stop signalled the stand-in',
+ ).toBe(false);
+
+ const clean = runOk(['clean'], { cwd: projectDir });
+ expect(clean.status, clean.stderr).toBe(0);
+ expect(clean.stdout).toContain('Pruned 1 stale lock: server (reused-pid)');
+ expect(existsSync(lockPath)).toBe(false);
+
+ server = spawn(NODE, [H.cliPath, 'start'], {
+ cwd: projectDir,
+ env: { ...process.env, ...HERMETIC_ENV },
+ stdio: ['ignore', 'pipe', 'pipe'],
+ });
+ server.stdout?.on('data', (d) => {
+ serverLog += d;
+ });
+ server.stderr?.on('data', (d) => {
+ serverLog += d;
+ });
+ const readLock = (): Record<string, unknown> | null => {
+ try {
+ return JSON.parse(readFileSync(lockPath, 'utf8')) as Record<string, unknown>;
+ } catch {
+ return null;
+ }
+ };
+ const booted = await waitFor(
+ () => serverLog.includes('Press Ctrl+C to stop') && readLock()?.pid === server?.pid,
+ { timeoutMs: 45_000 },
+ );
+ expect(
+ booted,
+ `ok start never finished booting under its own lock\nServer log:\n${serverLog}`,
+ ).toBe(true);
+ expect(readLock()?.processStart).toEqual(expect.any(Object));
+
+ const exited = new Promise<void>((res) => server?.once('exit', () => res()));
+ const stopStarted = runOk(['stop', projectDir], { cwd: projectDir, timeoutMs: 30_000 });
+ expect(stopStarted.status, stopStarted.stderr).toBe(0);
+ await exited;
+ expect(existsSync(lockPath)).toBe(false);
+ expect(
+ await waitFor(() => ownedProcessHasExited(standIn), { timeoutMs: 2_000 }),
+ 'ok stop signalled the stand-in',
+ ).toBe(false);
+ } finally {
+ if (server && server.exitCode === null && server.signalCode === null) server.kill('SIGKILL');
+ await stopOwnedProcess(standIn);
+ rmSync(projectDir, { recursive: true, force: true });
+ }
+ }, 120_000);
});
diff --git a/packages/cli/tests/support/windows-process-start-reader.test-helper.ts b/packages/cli/tests/support/windows-process-start-reader.test-helper.ts
new file mode 100644
index 000000000..6f9b38dd5
--- /dev/null
+++ b/packages/cli/tests/support/windows-process-start-reader.test-helper.ts
@@ -0,0 +1,8 @@
+export {};
+
+if (process.platform === 'win32' && process.env.OK_TEST_WINDOWS_PROCESS_START_READER === '1') {
+ const { registerWindowsProcessStartReader } = await import(
+ '../../src/commands/holder-process-start.ts'
+ );
+ registerWindowsProcessStartReader();
+}
diff --git a/packages/cli/vitest.config.ts b/packages/cli/vitest.config.ts
index 836986ffa..4f9920914 100644
--- a/packages/cli/vitest.config.ts
+++ b/packages/cli/vitest.config.ts
@@ -19,7 +19,13 @@ export default defineConfig({
...okVitestBase,
test: {
...okVitestBase.test,
- setupFiles: [...okVitestBase.test.setupFiles, serverAuthorityRegistrySetup],
+ setupFiles: [
+ ...okVitestBase.test.setupFiles,
+ serverAuthorityRegistrySetup,
+ fileURLToPath(
+ new URL('./tests/support/windows-process-start-reader.test-helper.ts', import.meta.url),
+ ),
+ ],
exclude: [...okVitestBase.test.exclude, '.ok-skill-publish-*'],
},
});
diff --git a/packages/core/src/desktop-bridge.ts b/packages/core/src/desktop-bridge.ts
index c66aff99f..06eb267d8 100644
--- a/packages/core/src/desktop-bridge.ts
+++ b/packages/core/src/desktop-bridge.ts
@@ -981,7 +981,8 @@ export interface OkRecentRemovedMissingInfo {
export type OkServerRestartFailure =
| { readonly ok: false; readonly reason: 'eperm' | 'other' }
- | { readonly ok: false; readonly reason: 'other-channel'; readonly holderChannel: string };
+ | { readonly ok: false; readonly reason: 'other-channel'; readonly holderChannel: string }
+ | { readonly ok: false; readonly reason: 'unverified'; readonly sigtermSent: boolean };
export type OkServerRestartOutcome = { readonly ok: true } | OkServerRestartFailure;
diff --git a/packages/desktop/src/main/holder-stop-prompt.test.ts b/packages/desktop/src/main/holder-stop-prompt.test.ts
index 3ec389768..c78903fb5 100644
--- a/packages/desktop/src/main/holder-stop-prompt.test.ts
+++ b/packages/desktop/src/main/holder-stop-prompt.test.ts
@@ -1,5 +1,6 @@
-import { describe, expect, test } from 'vitest';
+import { afterEach, describe, expect, test, vi } from 'vitest';
import {
+ describeStopFailure,
failedOpenHolder,
type HolderStopPromptDeps,
type HolderStopPromptInput,
@@ -93,6 +94,18 @@ describe('promptHolderStop', () => {
});
});
+ test.each([true, false])(
+ 'an unverified holder reports whether SIGTERM was sent (%s) instead of retrying',
+ async (sigtermSent) => {
+ expect(
+ await run(baseInput(), 0, { ok: false, reason: 'unverified', sigtermSent }),
+ ).toMatchObject({
+ result: { kind: 'stop-failed', reason: 'unverified', sigtermSent },
+ calls: ['dialog', 'stop'],
+ });
+ },
+ );
+
test('a holder that changed while the dialog was open is re-prompted from a fresh open, not retried as if it stopped', async () => {
expect(await run(baseInput(), 0, { ok: false, reason: 'holder-changed' })).toMatchObject({
result: { kind: 'reopened', opened: true },
@@ -172,3 +185,50 @@ describe('promptHolderStop', () => {
expect(lookups).toEqual(['/tmp/dragon']);
});
});
+
+describe('describeStopFailure', () => {
+ test('tells the user a holder that was asked to stop may still exit', () => {
+ expect(
+ describeStopFailure(
+ { kind: 'stop-failed', reason: 'unverified', sigtermSent: true },
+ '/tmp/dragon',
+ 'darwin',
+ ),
+ ).toBe(
+ "OpenKnowledge asked the process holding this project's server lock to stop, but could not confirm that it is this project's server, so it did not force it to quit. Wait a moment and try again. If the project is still locked, stop the server where it was started, or restart your computer.",
+ );
+ });
+
+ test('says an unverified holder that was never signalled was not stopped', () => {
+ expect(
+ describeStopFailure(
+ { kind: 'stop-failed', reason: 'unverified', sigtermSent: false },
+ '/tmp/dragon',
+ 'darwin',
+ ),
+ ).toBe(
+ "Could not confirm that the process holding this project's server lock is still its server, so it was not stopped. Stop the server where it was started, or restart your computer, then try again.",
+ );
+ });
+
+ test('offers ok stop only for a failure the identity check did not cause', () => {
+ expect(
+ describeStopFailure({ kind: 'stop-failed', reason: 'other' }, '/tmp/dragon', 'darwin'),
+ ).toBe(
+ "Could not stop the conflicting server. Quit it manually (`ok stop --force '/tmp/dragon'`) and try again.",
+ );
+ });
+
+ describe('on Beta', () => {
+ afterEach(() => vi.unstubAllEnvs());
+
+ test('names the Beta CLI in the manual stop command', () => {
+ vi.stubEnv('OK_CHANNEL', 'beta');
+ expect(
+ describeStopFailure({ kind: 'stop-failed', reason: 'other' }, '/tmp/dragon', 'darwin'),
+ ).toBe(
+ "Could not stop the conflicting server. Quit it manually (`ok-beta stop --force '/tmp/dragon'`) and try again.",
+ );
+ });
+ });
+});
diff --git a/packages/desktop/src/main/holder-stop-prompt.ts b/packages/desktop/src/main/holder-stop-prompt.ts
index 60e55c75d..3ce3d9901 100644
--- a/packages/desktop/src/main/holder-stop-prompt.ts
+++ b/packages/desktop/src/main/holder-stop-prompt.ts
@@ -1,4 +1,5 @@
import { desktopChannelLabel } from '@inkeep/open-knowledge-core';
+import { stopCommandLine } from './stop-command.ts';
import {
type ForceStopOutcome,
isOtherChannelHolderError,
@@ -49,6 +50,7 @@ export type HolderStopPromptResult =
| { kind: 'declined' }
| { kind: 'reopened'; opened: boolean }
| { kind: 'stop-failed'; reason: 'eperm' | 'other' }
+ | { kind: 'stop-failed'; reason: 'unverified'; sigtermSent: boolean }
| { kind: 'retried'; opened: boolean };
export async function promptHolderStop(
@@ -94,5 +96,29 @@ export async function promptHolderStop(
);
if (stop.ok) return { kind: 'retried', opened: await deps.retryOpen() };
if (stop.reason === 'holder-changed') return { kind: 'reopened', opened: await deps.reopen() };
+ if (stop.reason === 'unverified') {
+ return { kind: 'stop-failed', reason: 'unverified', sigtermSent: stop.sigtermSent };
+ }
return { kind: 'stop-failed', reason: stop.reason };
}
+
+export function describeStopFailure(
+ failure: Extract<HolderStopPromptResult, { kind: 'stop-failed' }>,
+ projectPath: string,
+ platform: NodeJS.Platform,
+): string {
+ switch (failure.reason) {
+ case 'eperm':
+ return 'The conflicting server belongs to another user account and cannot be stopped from here. Quit it from that account and try again.';
+ case 'unverified':
+ return failure.sigtermSent
+ ? "OpenKnowledge asked the process holding this project's server lock to stop, but could not confirm that it is this project's server, so it did not force it to quit. Wait a moment and try again. If the project is still locked, stop the server where it was started, or restart your computer."
+ : "Could not confirm that the process holding this project's server lock is still its server, so it was not stopped. Stop the server where it was started, or restart your computer, then try again.";
+ case 'other':
+ return `Could not stop the conflicting server. Quit it manually (\`${stopCommandLine(projectPath, platform, { force: true })}\`) and try again.`;
+ default: {
+ const _exhaustive: never = failure;
+ return _exhaustive;
+ }
+ }
+}
diff --git a/packages/desktop/src/main/index.ts b/packages/desktop/src/main/index.ts
index cd1f53804..1d645309e 100644
--- a/packages/desktop/src/main/index.ts
+++ b/packages/desktop/src/main/index.ts
@@ -34,6 +34,7 @@ import {
type ProjectAiIntegrationsResult,
previewContent,
readExistingMcpEntry,
+ registerWindowsProcessStartReader,
removeOwnMcpEntry,
removeProjectSkill,
removeUserGlobalSkillBundle,
@@ -93,6 +94,7 @@ import {
type ResolvedSkillHost,
RUNTIME_VERSION,
readBundleDecision,
+ readProcessStart,
readServerLock,
readServerPackageVersion,
recordSkillInstallEvent,
@@ -301,7 +303,7 @@ import {
import { createBootBudgetDirSizeProbe } from './fs-walk-budget.ts';
import { ensureGitAvailable } from './git-preflight-handler.ts';
import { readCanonicalGitHubRemoteUrl } from './git-remote.ts';
-import { failedOpenHolder, promptHolderStop } from './holder-stop-prompt.ts';
+import { describeStopFailure, failedOpenHolder, promptHolderStop } from './holder-stop-prompt.ts';
import { agentConnectionsAvailability, isSupportedInstallShape } from './install-shape.ts';
import {
combineInstanceLabels,
@@ -438,7 +440,7 @@ import {
setPreferredWindowVibrancy,
type VibrancyMaterial,
} from './reduced-transparency-handler.ts';
-import { removeGitFolder } from './remove-git-folder.ts';
+import { stopServerThenRemoveGitFolder } from './remove-git-folder.ts';
import { attachRendererConsoleCapture } from './renderer-console-capture.ts';
import { installRendererFileProtocol } from './renderer-file-protocol.ts';
import { createRendererReadySink, type RendererReadySink } from './renderer-ready-sink.ts';
@@ -589,6 +591,7 @@ import {
type BrowserWindowLike,
bringWindowToFront,
collabUrlFromApiOrigin,
+ heldChildServer,
setWindowInstanceLabel,
type UtilityProcessLike,
WindowManager,
@@ -1505,7 +1508,13 @@ function ensureWindowManager() {
'spawnCliServer: child_process.spawn did not return a pid after spawn-event resolution.',
);
}
- return { pid, readExit: () => exitRecord };
+ childRef.on('error', (err) => {
+ getLogger('server-spawn').warn(
+ { event: 'desktop-spawned-server-signal-failed', err, pid },
+ 'signalling a spawned server failed',
+ );
+ });
+ return heldChildServer(childRef, pid, () => exitRecord);
};
wm = new WindowManager({
@@ -1600,6 +1609,7 @@ function ensureWindowManager() {
readServerLock: (lockDir) => readServerLock(lockDir),
removeServerLock: (lockDir, expected) => breakServerLockHeldBy(lockDir, expected),
isProcessAlive: (pid) => isProcessAlive(pid),
+ readProcessStart: (pid) => readProcessStart(pid),
hostname: () => osHostname(),
probeWsUpgrade: (url, timeoutMs) => probeWsUpgrade(url, timeoutMs),
realpathSync: (p) => realpathSync(p),
@@ -2313,10 +2323,7 @@ async function openProjectOrFallbackToNavigator(
flushDesktopLogger();
showErrorOverNavigator(
'Unable to open project',
- `${projectPath}\n\n` +
- (prompt.reason === 'eperm'
- ? 'The conflicting server belongs to another user account and cannot be stopped from here. Quit it from that account and try again.'
- : `Could not stop the conflicting server. Quit it manually (\`${stopCommandLine(projectPath, process.platform, { force: true })}\`) and try again.`),
+ `${projectPath}\n\n${describeStopFailure(prompt, projectPath, process.platform)}`,
);
} else {
getLogger('project').info(
@@ -5384,25 +5391,12 @@ function registerIpcHandlers() {
});
handle('ok:fs:remove-git-folder', async (_event, gitRoot) => {
- if (typeof gitRoot === 'string' && recentGitRoots.has(gitRoot)) {
- try {
- const outcome = await runStop({
- lockDir: resolveLockDir(gitRoot),
- force: true,
- log: (msg) => getLogger('project').info({ gitRoot }, `[remove-git-folder] ${msg}`),
- });
- getLogger('project').info(
- { gitRoot, stopped: outcome.stopped.length, hadTargets: outcome.hadTargets },
- 'remove-git-folder: stopped worktree server before .git removal',
- );
- } catch (err) {
- getLogger('project').warn(
- { gitRoot, err },
- 'remove-git-folder: worktree server stop failed',
- );
- }
- }
- await removeGitFolder(gitRoot, { allowedGitRoots: recentGitRoots });
+ await stopServerThenRemoveGitFolder(gitRoot, {
+ allowedGitRoots: recentGitRoots,
+ lockDirFor: resolveLockDir,
+ runStop,
+ log: getLogger('project'),
+ });
return undefined;
});
@@ -6065,6 +6059,7 @@ installStdioBrokenPipeGuard(process, {
app.commandLine.appendSwitch('use-system-ca');
trustSystemCertificates();
+registerWindowsProcessStartReader(getLogger('process-start'));
applyDevShmPosture({
platform: process.platform,
diff --git a/packages/desktop/src/main/project-session.test.ts b/packages/desktop/src/main/project-session.test.ts
index 6458e9ab5..2dc38c843 100644
--- a/packages/desktop/src/main/project-session.test.ts
+++ b/packages/desktop/src/main/project-session.test.ts
@@ -23,11 +23,19 @@ function createHarness() {
const contexts = {
firstFile: {
projectPath: '/notes',
- ephemeral: { projectDir: '/tmp/first-file', pid: 11, lockDir: '/tmp/first-file/.ok/local' },
+ ephemeral: {
+ projectDir: '/tmp/first-file',
+ server: { pid: 11, readExit: () => null, signal: () => {} },
+ lockDir: '/tmp/first-file/.ok/local',
+ },
},
secondFile: {
projectPath: '/notes',
- ephemeral: { projectDir: '/tmp/second-file', pid: 12, lockDir: '/tmp/second-file/.ok/local' },
+ ephemeral: {
+ projectDir: '/tmp/second-file',
+ server: { pid: 12, readExit: () => null, signal: () => {} },
+ lockDir: '/tmp/second-file/.ok/local',
+ },
},
project: { projectPath: '/notes' },
otherProject: { projectPath: '/other' },
diff --git a/packages/desktop/src/main/remove-git-folder.test.ts b/packages/desktop/src/main/remove-git-folder.test.ts
index 229139f27..786cd3e3b 100644
--- a/packages/desktop/src/main/remove-git-folder.test.ts
+++ b/packages/desktop/src/main/remove-git-folder.test.ts
@@ -9,8 +9,9 @@ import {
} from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
+import type { runStop } from '@inkeep/open-knowledge';
import { afterEach, beforeEach, describe, expect, test } from 'vitest';
-import { removeGitFolder } from './remove-git-folder.ts';
+import { removeGitFolder, stopServerThenRemoveGitFolder } from './remove-git-folder.ts';
let tmpRoot: string;
@@ -144,3 +145,133 @@ describe('removeGitFolder — pure helper backing ok:fs:remove-git-folder', () =
expect(existsSync(join(gitRoot, '.git'))).toBe(true);
});
});
+
+describe('stopServerThenRemoveGitFolder', () => {
+ const notStopped = 'remove-git-folder: the worktree server was not stopped; removing .git anyway';
+ let gitRoot: string;
+ let lockDir: string;
+
+ beforeEach(() => {
+ gitRoot = join(tmpRoot, 'proj');
+ lockDir = join(gitRoot, '.ok', 'local');
+ mkdirSync(join(gitRoot, '.git', 'objects'), { recursive: true });
+ });
+
+ function harness(stop: () => ReturnType<typeof runStop>) {
+ const info: Array<{ data: Record<string, unknown>; message: string }> = [];
+ const warn: Array<{ data: Record<string, unknown>; message: string }> = [];
+ const stopCalls: Array<Parameters<typeof runStop>[0]> = [];
+ const deps = {
+ allowedGitRoots: new Set([gitRoot]),
+ lockDirFor: (root: string) => join(root, '.ok', 'local'),
+ runStop: (stopDeps: Parameters<typeof runStop>[0]) => {
+ stopCalls.push(stopDeps);
+ return stop();
+ },
+ log: {
+ info: (data: Record<string, unknown>, message: string) => info.push({ data, message }),
+ warn: (data: Record<string, unknown>, message: string) => warn.push({ data, message }),
+ },
+ };
+ return { info, warn, stopCalls, deps };
+ }
+
+ test('removes .git after a refused stop, logging the holder, its lock, the cause and the native failures', async () => {
+ const lockPath = join(lockDir, 'server.lock');
+ const { info, warn, stopCalls, deps } = harness(async () => ({
+ stopped: [],
+ failed: [
+ {
+ target: { name: 'server', pid: 4242 },
+ error: 'Could not read when process 4242 started',
+ unverified: {
+ cause: 'process-start-unreadable',
+ pid: 4242,
+ lockPath,
+ platform: 'win32',
+ nativeFailures: [{ kind: 'query-failed', reason: 'Access is denied. (os error 5)' }],
+ },
+ },
+ ],
+ hadTargets: true,
+ }));
+
+ await stopServerThenRemoveGitFolder(gitRoot, deps);
+
+ expect(stopCalls).toEqual([expect.objectContaining({ lockDir, force: true })]);
+ expect(info).toEqual([]);
+ expect(warn).toEqual([
+ {
+ data: {
+ gitRoot,
+ lockPid: 4242,
+ lockPath,
+ cause: 'process-start-unreadable',
+ nativeFailures: [{ kind: 'query-failed', reason: 'Access is denied. (os error 5)' }],
+ error: 'Could not read when process 4242 started',
+ },
+ message: notStopped,
+ },
+ ]);
+ expect(existsSync(join(gitRoot, '.git'))).toBe(false);
+ });
+
+ test('reports a signalled server as stopped, then removes .git', async () => {
+ const { info, warn, deps } = harness(async () => ({
+ stopped: [{ name: 'server', pid: 4242, port: 3001 }],
+ failed: [],
+ hadTargets: true,
+ }));
+
+ await stopServerThenRemoveGitFolder(gitRoot, deps);
+
+ expect(info).toEqual([
+ {
+ data: { gitRoot, stopped: 1 },
+ message: 'remove-git-folder: stopped worktree server before .git removal',
+ },
+ ]);
+ expect(warn).toEqual([]);
+ expect(existsSync(join(gitRoot, '.git'))).toBe(false);
+ });
+
+ test('records a declined stop as not stopped and still removes .git', async () => {
+ const { warn, deps } = harness(async () => ({
+ stopped: [],
+ failed: [],
+ hadTargets: true,
+ declined: { clients: 2 },
+ }));
+
+ await stopServerThenRemoveGitFolder(gitRoot, deps);
+
+ expect(warn).toEqual([{ data: { gitRoot, declined: { clients: 2 } }, message: notStopped }]);
+ expect(existsSync(join(gitRoot, '.git'))).toBe(false);
+ });
+
+ test('records a stop that throws as not stopped and still removes .git', async () => {
+ const failure = new Error('lock unreadable');
+ const { warn, deps } = harness(async () => {
+ throw failure;
+ });
+
+ await stopServerThenRemoveGitFolder(gitRoot, deps);
+
+ expect(warn).toEqual([{ data: { gitRoot, err: failure }, message: notStopped }]);
+ expect(existsSync(join(gitRoot, '.git'))).toBe(false);
+ });
+
+ test('stops nothing for a root no probe surfaced and refuses its removal', async () => {
+ const { stopCalls, deps } = harness(async () => ({
+ stopped: [],
+ failed: [],
+ hadTargets: false,
+ }));
+
+ await expect(
+ stopServerThenRemoveGitFolder(gitRoot, { ...deps, allowedGitRoots: new Set<string>() }),
+ ).rejects.toThrow(/not surfaced by a recent probe/);
+ expect(stopCalls).toEqual([]);
+ expect(existsSync(join(gitRoot, '.git'))).toBe(true);
+ });
+});
diff --git a/packages/desktop/src/main/remove-git-folder.ts b/packages/desktop/src/main/remove-git-folder.ts
index c5c475edc..4051e57ac 100644
--- a/packages/desktop/src/main/remove-git-folder.ts
+++ b/packages/desktop/src/main/remove-git-folder.ts
@@ -1,5 +1,7 @@
import { existsSync, promises as fsPromises, realpathSync } from 'node:fs';
import { basename, isAbsolute, join, resolve } from 'node:path';
+import type { runStop } from '@inkeep/open-knowledge';
+import type { DesktopLogger } from './desktop-logger.ts';
const REJECT_PREFIX = 'ok:fs:remove-git-folder rejected:';
@@ -7,6 +9,68 @@ interface RemoveGitFolderDeps {
readonly allowedGitRoots: ReadonlySet<string>;
}
+interface StopThenRemoveGitFolderDeps extends RemoveGitFolderDeps {
+ readonly lockDirFor: (gitRoot: string) => string;
+ readonly runStop: (deps: Parameters<typeof runStop>[0]) => ReturnType<typeof runStop>;
+ readonly log: Pick<DesktopLogger, 'info' | 'warn'>;
+}
+
+const NOT_STOPPED = 'remove-git-folder: the worktree server was not stopped; removing .git anyway';
+
+export async function stopServerThenRemoveGitFolder(
+ gitRoot: unknown,
+ deps: StopThenRemoveGitFolderDeps,
+): Promise<void> {
+ if (typeof gitRoot === 'string' && deps.allowedGitRoots.has(gitRoot)) {
+ await stopServerBeforeGitRemoval(gitRoot, deps);
+ }
+ await removeGitFolder(gitRoot, deps);
+}
+
+async function stopServerBeforeGitRemoval(
+ gitRoot: string,
+ deps: StopThenRemoveGitFolderDeps,
+): Promise<void> {
+ const { log } = deps;
+ let outcome: Awaited<ReturnType<typeof runStop>>;
+ try {
+ outcome = await deps.runStop({
+ lockDir: deps.lockDirFor(gitRoot),
+ force: true,
+ log: (msg) => log.info({ gitRoot }, `[remove-git-folder] ${msg}`),
+ });
+ } catch (err) {
+ log.warn({ gitRoot, err }, NOT_STOPPED);
+ return;
+ }
+ if (outcome.stopped.length > 0) {
+ log.info(
+ { gitRoot, stopped: outcome.stopped.length },
+ 'remove-git-folder: stopped worktree server before .git removal',
+ );
+ }
+ if (outcome.declined !== undefined) {
+ log.warn({ gitRoot, declined: outcome.declined }, NOT_STOPPED);
+ }
+ for (const { target, error, unverified } of outcome.failed) {
+ log.warn(
+ {
+ gitRoot,
+ lockPid: target.pid,
+ lockPath: unverified?.lockPath,
+ cause: unverified?.cause,
+ ...(unverified !== undefined &&
+ 'nativeFailures' in unverified &&
+ unverified.nativeFailures.length > 0
+ ? { nativeFailures: unverified.nativeFailures }
+ : {}),
+ error,
+ },
+ NOT_STOPPED,
+ );
+ }
+}
+
export async function removeGitFolder(gitRoot: unknown, deps: RemoveGitFolderDeps): Promise<void> {
if (typeof gitRoot !== 'string' || gitRoot.length === 0) {
throw new Error(`${REJECT_PREFIX} gitRoot must be a non-empty string`);
diff --git a/packages/desktop/src/main/signal-detached-server-stop.test.ts b/packages/desktop/src/main/signal-detached-server-stop.test.ts
index 6a5af286b..6068914be 100644
--- a/packages/desktop/src/main/signal-detached-server-stop.test.ts
+++ b/packages/desktop/src/main/signal-detached-server-stop.test.ts
@@ -1,10 +1,19 @@
import { describe, expect, test, vi } from 'vitest';
import {
+ type SpawnedServer,
signalDetachedServerStop,
signalStopOwnedUtilityForks,
type UtilityProcessLike,
} from './window-manager.ts';
+function child(
+ pid: number,
+ signal: (signal: NodeJS.Signals) => void,
+ exit: ReturnType<SpawnedServer['readExit']> = null,
+): SpawnedServer {
+ return { pid, readExit: () => exit, signal };
+}
+
function ctx(
ownsServer: boolean,
kill: ((signal: NodeJS.Signals) => void) | null,
@@ -18,52 +27,50 @@ function ctx(
}
describe('signalDetachedServerStop (before-quit-for-update teardown)', () => {
- test('sends SIGTERM to every detached server pid', () => {
- const killProbe = vi.fn((_pid: number, _signal: number | NodeJS.Signals) => {});
- const signalled = signalDetachedServerStop(
- [
- ['/proj/a', 101],
- ['/proj/b', 202],
- ],
- killProbe,
- );
+ test('sends SIGTERM to every running detached server through its child', () => {
+ const signals: Array<[number, NodeJS.Signals]> = [];
+ const signalled = signalDetachedServerStop([
+ ['/proj/a', child(101, (signal) => signals.push([101, signal]))],
+ ['/proj/b', child(202, (signal) => signals.push([202, signal]))],
+ ]);
expect(signalled).toBe(2);
- expect(killProbe).toHaveBeenCalledTimes(2);
- expect(killProbe).toHaveBeenNthCalledWith(1, 101, 'SIGTERM');
- expect(killProbe).toHaveBeenNthCalledWith(2, 202, 'SIGTERM');
+ expect(signals).toEqual([
+ [101, 'SIGTERM'],
+ [202, 'SIGTERM'],
+ ]);
});
- test('treats an already-exited server (ESRCH) as done without logging a failure', () => {
- const killProbe = vi.fn((_pid: number, _signal: number | NodeJS.Signals) => {
- const err = new Error('no such process') as NodeJS.ErrnoException;
- err.code = 'ESRCH';
- throw err;
- });
+ test('skips a server whose child has exited, so its pid is never signalled', () => {
+ const signal = vi.fn();
const log = { warn: vi.fn((_o: object, _m: string) => {}) };
- const signalled = signalDetachedServerStop([['/proj/gone', 303]], killProbe, log);
+ const signalled = signalDetachedServerStop(
+ [['/proj/gone', child(303, signal, { code: 0, signal: null })]],
+ log,
+ );
expect(signalled).toBe(0);
+ expect(signal).not.toHaveBeenCalled();
expect(log.warn).not.toHaveBeenCalled();
});
- test('logs but never throws on a non-ESRCH signal failure, and continues to later pids', () => {
- const killProbe = vi.fn((pid: number, _signal: number | NodeJS.Signals) => {
- if (pid === 404) {
- const err = new Error('operation not permitted') as NodeJS.ErrnoException;
- err.code = 'EPERM';
- throw err;
- }
- });
+ test('logs but never throws on a signal failure, and continues to later servers', () => {
+ const later = vi.fn();
const log = { warn: vi.fn((_o: object, _m: string) => {}) };
const signalled = signalDetachedServerStop(
[
- ['/proj/locked', 404],
- ['/proj/ok', 505],
+ [
+ '/proj/locked',
+ child(404, () => {
+ const err = new Error('operation not permitted') as NodeJS.ErrnoException;
+ err.code = 'EPERM';
+ throw err;
+ }),
+ ],
+ ['/proj/ok', child(505, later)],
],
- killProbe,
log,
);
expect(signalled).toBe(1);
- expect(killProbe).toHaveBeenCalledTimes(2);
+ expect(later).toHaveBeenCalledWith('SIGTERM');
expect(log.warn).toHaveBeenCalledTimes(1);
expect(log.warn.mock.calls[0]?.[0]).toMatchObject({
event: 'update-install-server-stop-failed',
@@ -74,9 +81,7 @@ describe('signalDetachedServerStop (before-quit-for-update teardown)', () => {
});
test('no entries (already drained — e.g. the Relaunch-now path) is a clean no-op', () => {
- const killProbe = vi.fn((_pid: number, _signal: number | NodeJS.Signals) => {});
- expect(signalDetachedServerStop([], killProbe)).toBe(0);
- expect(killProbe).not.toHaveBeenCalled();
+ expect(signalDetachedServerStop([])).toBe(0);
});
});
diff --git a/packages/desktop/src/main/window-manager.ts b/packages/desktop/src/main/window-manager.ts
index 32c1eb70e..6eab7d7d1 100644
--- a/packages/desktop/src/main/window-manager.ts
+++ b/packages/desktop/src/main/window-manager.ts
@@ -1,3 +1,4 @@
+import type { EventEmitter } from 'node:events';
import { readFileSync, realpathSync } from 'node:fs';
import { basename, dirname, join, resolve } from 'node:path';
import {
@@ -10,7 +11,13 @@ import {
sliceLastSpawnAttempt,
} from '@inkeep/open-knowledge-core';
import type { KeepaliveHandle } from '@inkeep/open-knowledge-core/keepalive';
-import { getLocalDir } from '@inkeep/open-knowledge-server';
+import {
+ assessLockHolder,
+ getLocalDir,
+ type LockHolderIdentity,
+ type ProcessStart,
+ readProcessStart,
+} from '@inkeep/open-knowledge-server';
import {
BOOT_HEARTBEAT_EVENTS,
isUtilityInitPhase,
@@ -69,6 +76,31 @@ const RECOVERY_CALLERS: Record<
const EPHEMERAL_SERVER_WATCH_POLL_MS = 3_000;
+function terminationOutcome(term: TerminationResult): string {
+ if (term.ok) return term.outcome;
+ if (term.reason === 'unverified')
+ return term.sigtermSent ? 'sigterm-sent-unverified' : 'unverified';
+ return term.reason;
+}
+
+function restartFailure(term: Extract<TerminationResult, { ok: false }>): OkServerRestartOutcome {
+ switch (term.reason) {
+ case 'eperm':
+ case 'other':
+ return { ok: false, reason: term.reason };
+ case 'unverified':
+ return { ok: false, reason: 'unverified', sigtermSent: term.sigtermSent };
+ default: {
+ const exhaustive: never = term;
+ return exhaustive;
+ }
+ }
+}
+
+function terminationEscalated(term: TerminationResult): boolean {
+ return term.ok && term.outcome === 'stopped' && term.escalated;
+}
+
function isValidLockPidLocal(value: unknown): value is number {
if (typeof value !== 'number') return false;
if (!Number.isInteger(value)) return false;
@@ -163,6 +195,7 @@ export interface ServerLockMetadataLike {
machineId?: string;
draining?: boolean;
channel?: string;
+ processStart?: unknown;
}
export type OtherChannelHolderError = Error & {
@@ -236,7 +269,25 @@ type ChannelConflict = { holderChannel: string; selfChannel: DesktopProductName
export type ForceStopOutcome =
| { ok: true }
- | { ok: false; reason: 'eperm' | 'other' | 'holder-changed' };
+ | { ok: false; reason: 'eperm' | 'other' | 'holder-changed' }
+ | { ok: false; reason: 'unverified'; sigtermSent: boolean };
+
+type TerminationTarget =
+ | { kind: 'lock'; lock: LockHolderIdentity }
+ | { kind: 'spawned'; server: SpawnedServer };
+
+type TerminationResult =
+ | { ok: true; outcome: 'stopped'; escalated: boolean }
+ | { ok: true; outcome: 'already-gone' | 'holder-replaced' }
+ | { ok: false; reason: 'eperm' | 'other' }
+ | { ok: false; reason: 'unverified'; sigtermSent: boolean };
+
+type RawServerLockFields = Partial<
+ Record<
+ 'pid' | 'port' | 'url' | 'channel' | 'startedAt' | 'machineId' | 'hostname' | 'processStart',
+ unknown
+ >
+>;
function loopbackOriginFromUrl(url: unknown): string | null {
if (typeof url !== 'string' || url.length === 0) return null;
@@ -294,7 +345,7 @@ interface ProjectContext {
ownsServer: boolean;
ephemeral?: {
projectDir: string;
- pid: number;
+ server: SpawnedServer;
lockDir: string;
};
}
@@ -324,15 +375,51 @@ interface CreateProjectWindowOpts {
freshlyCreated?: boolean;
}
+export interface SpawnedServer {
+ readonly pid: number;
+ readExit(): { code: number | null; signal: string | null } | null;
+ signal(signal: NodeJS.Signals): void;
+}
+
+export function heldChildServer(
+ child: Pick<EventEmitter, 'on' | 'off'> & { kill(signal: NodeJS.Signals): boolean },
+ pid: number,
+ readExit: SpawnedServer['readExit'],
+): SpawnedServer {
+ return {
+ pid,
+ readExit,
+ signal: (signal) => {
+ let failure: Error | undefined;
+ const recordFailure = (err: Error): void => {
+ failure = err;
+ };
+ child.on('error', recordFailure);
+ try {
+ if (child.kill(signal)) return;
+ } finally {
+ child.off('error', recordFailure);
+ }
+ throw (
+ failure ??
+ Object.assign(
+ new Error(`kill ${signal} was not delivered: the server has already exited`),
+ {
+ code: 'ESRCH',
+ syscall: 'kill',
+ },
+ )
+ );
+ },
+ };
+}
+
type SpawnCliServer = (opts: {
contentDir: string;
reactShellDistDir: string;
singleFile?: string;
projectDir?: string;
-}) => Promise<{
- pid: number;
- readExit?: () => { code: number | null; signal: string | null } | null;
-}>;
+}) => Promise<SpawnedServer>;
export interface WindowManagerDeps {
createWindow(opts: {
@@ -377,6 +464,7 @@ export interface WindowManagerDeps {
readServerLock?(lockDir: string): ServerLockMetadataLike | null;
removeServerLock?(lockDir: string, expected: { pid: number }): boolean;
isProcessAlive?(pid: number): boolean;
+ readProcessStart?(pid: number): ProcessStart | null;
hostname?(): string;
probeWsUpgrade?(url: string, timeoutMs: number): Promise<boolean>;
utilityInitTimeoutMs?: number;
@@ -405,14 +493,15 @@ export interface WindowManagerDeps {
}
export function signalDetachedServerStop(
- entries: ReadonlyArray<readonly [string, number]>,
- killProbe: (pid: number, signal: number | NodeJS.Signals) => void,
+ entries: ReadonlyArray<readonly [string, SpawnedServer]>,
log?: { warn(obj: object, msg: string): void },
): number {
let signalled = 0;
- for (const [projectPath, pid] of entries) {
+ for (const [projectPath, server] of entries) {
+ if (server.readExit() !== null) continue;
+ const { pid } = server;
try {
- killProbe(pid, 'SIGTERM');
+ server.signal('SIGTERM');
signalled++;
} catch (err) {
const code = (err as NodeJS.ErrnoException).code;
@@ -475,7 +564,7 @@ export class WindowManager {
private readonly loadingContextByWindow = new Map<BrowserWindowLike, ProjectContext>();
- private readonly spawnedDetachedPids = new Map<string, number>();
+ private readonly spawnedDetachedServers = new Map<string, SpawnedServer>();
private readonly ephemeralPendingByPath = new Map<string, Promise<ProjectContext>>();
@@ -563,10 +652,12 @@ export class WindowManager {
async stopAllOwnedServers(): Promise<void> {
signalStopOwnedUtilityForks(this.windowsByPath.values(), this.deps.log);
- const stopOne = async (canonicalKey: string, pid: number): Promise<void> => {
+ const stopOne = async (canonicalKey: string, server: SpawnedServer): Promise<void> => {
const projectPath = canonicalKey;
+ const { pid } = server;
+ if (server.readExit() !== null) return;
try {
- this.deps.killProbe(pid, 'SIGTERM');
+ server.signal('SIGTERM');
} catch (err) {
if ((err as NodeJS.ErrnoException).code === 'ESRCH') {
return;
@@ -577,7 +668,7 @@ export class WindowManager {
const graceMs = this.deps.sigtermGraceMs ?? DEFAULT_SIGTERM_GRACE_MS;
const deadline = Date.now() + graceMs;
while (Date.now() < deadline) {
- if (!this.isPidAlive(pid)) return;
+ if (server.readExit() !== null) return;
await new Promise<void>((resolveSleep) => {
this.deps.setTimeout(() => {
resolveSleep();
@@ -585,8 +676,9 @@ export class WindowManager {
});
}
}
+ if (server.readExit() !== null) return;
try {
- this.deps.killProbe(pid, 'SIGKILL');
+ server.signal('SIGKILL');
this.deps.log?.warn(
{ event: 'auto-update-server-stop-escalated', pid, projectPath },
'[window-manager] SIGTERM grace expired — escalated to SIGKILL',
@@ -606,15 +698,15 @@ export class WindowManager {
);
}
};
- const entries = [...this.spawnedDetachedPids.entries()];
- this.spawnedDetachedPids.clear();
+ const entries = [...this.spawnedDetachedServers.entries()];
+ this.spawnedDetachedServers.clear();
const ephemeralSessions = [...this.windowsByPath.values()]
.map((ctx) => ctx.ephemeral)
.filter((e): e is NonNullable<ProjectContext['ephemeral']> => e !== undefined);
await Promise.all([
- ...entries.map(([key, pid]) => stopOne(key, pid)),
+ ...entries.map(([key, server]) => stopOne(key, server)),
...ephemeralSessions.map((session) => this.teardownEphemeralSession(session)),
]);
}
@@ -622,14 +714,14 @@ export class WindowManager {
signalStopAllOwnedServers(): void {
signalStopOwnedUtilityForks(this.windowsByPath.values(), this.deps.log);
- const detached = [...this.spawnedDetachedPids.entries()];
- this.spawnedDetachedPids.clear();
+ const detached = [...this.spawnedDetachedServers.entries()];
+ this.spawnedDetachedServers.clear();
const ephemeral = [...this.windowsByPath.values()]
.map((ctx) => ctx.ephemeral)
.filter((e): e is NonNullable<ProjectContext['ephemeral']> => e !== undefined)
- .map((e) => [e.projectDir, e.pid] as const);
+ .map((e) => [e.projectDir, e.server] as const);
const entries = [...detached, ...ephemeral];
- const signalled = signalDetachedServerStop(entries, this.deps.killProbe, this.deps.log);
+ const signalled = signalDetachedServerStop(entries, this.deps.log);
if (entries.length > 0) {
this.deps.log?.info(
{ event: 'update-install-server-stop', count: entries.length, signalled },
@@ -649,10 +741,70 @@ export class WindowManager {
}
}
+ private runningSpawnedServer(canonicalKey: string): SpawnedServer | undefined {
+ const server = this.spawnedDetachedServers.get(canonicalKey);
+ if (server === undefined || server.readExit() === null) return server;
+ this.spawnedDetachedServers.delete(canonicalKey);
+ return undefined;
+ }
+
+ private heldChildExit(handle: Pick<SpawnedServer, 'pid' | 'readExit'>): {
+ exit: ReturnType<SpawnedServer['readExit']>;
+ exited: boolean | undefined;
+ } {
+ const exit = handle.readExit();
+ if (exit !== null) return { exit, exited: true };
+ const isAlive = this.deps.isProcessAlive;
+ return { exit, exited: isAlive ? !isAlive(handle.pid) : undefined };
+ }
+
+ private readHolderStart(pid: number): ProcessStart | null {
+ const read = this.deps.readProcessStart;
+ return read ? read(pid) : readProcessStart(pid);
+ }
+
+ private settleUnlessLockWriter(
+ lock: LockHolderIdentity,
+ sigtermSent: boolean,
+ ): TerminationResult | null {
+ const assessment = assessLockHolder(lock, {
+ isAlive: (pid) => this.isPidAlive(pid),
+ readProcessStart: (pid) => this.readHolderStart(pid),
+ });
+ const withheld = sigtermSent ? 'SIGKILL' : 'SIGTERM';
+ switch (assessment.verdict) {
+ case 'writer':
+ return null;
+ case 'gone':
+ return sigtermSent
+ ? { ok: true, outcome: 'stopped', escalated: false }
+ : { ok: true, outcome: 'already-gone' };
+ case 'replaced':
+ this.deps.log?.warn(
+ { event: 'desktop-lock-holder-replaced', lockPid: lock.pid, sigtermSent },
+ `[window-manager] lock pid now belongs to a process that did not write the lock; not sending it ${withheld}`,
+ );
+ return sigtermSent
+ ? { ok: true, outcome: 'stopped', escalated: false }
+ : { ok: true, outcome: 'holder-replaced' };
+ case 'unverified':
+ this.deps.log?.warn(
+ {
+ event: 'desktop-lock-holder-unverified',
+ lockPid: lock.pid,
+ sigtermSent,
+ cause: assessment.cause,
+ },
+ `[window-manager] could not verify that the lock pid is the server that wrote the lock; not sending it ${withheld}`,
+ );
+ return { ok: false, reason: 'unverified', sigtermSent };
+ }
+ }
+
private isEphemeralServerAlive(ctx: ProjectContext): boolean {
const eph = ctx.ephemeral;
if (eph === undefined) return true;
- if (!this.isPidAlive(eph.pid)) return false;
+ if (eph.server.readExit() !== null) return false;
const reader = this.deps.readServerLock;
if (!reader) return true;
const lock = reader(eph.lockDir);
@@ -685,9 +837,10 @@ export class WindowManager {
);
}
- private sigtermSpawnOrphan(pid: number, projectPath: string, message: string): void {
+ private sigtermSpawnOrphan(server: SpawnedServer, projectPath: string, message: string): void {
+ const { pid } = server;
try {
- this.deps.killProbe(pid, 'SIGTERM');
+ server.signal('SIGTERM');
} catch (signalErr) {
const code = (signalErr as NodeJS.ErrnoException).code;
if (code !== 'ESRCH') {
@@ -699,36 +852,69 @@ export class WindowManager {
}
}
- private async terminateServerByPid(
- pid: number,
- ): Promise<{ ok: true; escalated: boolean } | { ok: false; reason: 'eperm' | 'other' }> {
+ private async terminateServerByPid(target: TerminationTarget): Promise<TerminationResult> {
+ if (target.kind === 'spawned') return this.terminateSpawnedServer(target.server);
+ const { lock } = target;
+ const settled = this.settleUnlessLockWriter(lock, false);
+ if (settled !== null) return settled;
try {
- this.deps.killProbe(pid, 'SIGTERM');
+ this.deps.killProbe(lock.pid, 'SIGTERM');
} catch (err) {
const code = (err as NodeJS.ErrnoException).code;
- if (code === 'ESRCH') return { ok: true, escalated: false };
+ if (code === 'ESRCH') return { ok: true, outcome: 'already-gone' };
return { ok: false, reason: code === 'EPERM' ? 'eperm' : 'other' };
}
- {
- const graceMs = this.deps.sigtermGraceMs ?? RESTART_SIGTERM_GRACE_MS;
- const deadline = Date.now() + graceMs;
- while (Date.now() < deadline) {
- if (!this.isPidAlive(pid)) return { ok: true, escalated: false };
- await new Promise<void>((resolveSleep) => {
- this.deps.setTimeout(() => resolveSleep(), DEFAULT_SIGTERM_POLL_MS);
- });
- }
+ if (await this.awaitExitWithinGrace(() => !this.isPidAlive(lock.pid))) {
+ return { ok: true, outcome: 'stopped', escalated: false };
+ }
+ const settledAfterGrace = this.settleUnlessLockWriter(lock, true);
+ if (settledAfterGrace !== null) return settledAfterGrace;
+ try {
+ this.deps.killProbe(lock.pid, 'SIGKILL');
+ return { ok: true, outcome: 'stopped', escalated: true };
+ } catch (err) {
+ const code = (err as NodeJS.ErrnoException).code;
+ if (code === 'ESRCH') return { ok: true, outcome: 'stopped', escalated: true };
+ return { ok: false, reason: code === 'EPERM' ? 'eperm' : 'other' };
+ }
+ }
+
+ private async terminateSpawnedServer(server: SpawnedServer): Promise<TerminationResult> {
+ const exited = () => server.readExit() !== null;
+ if (exited()) return { ok: true, outcome: 'already-gone' };
+ try {
+ server.signal('SIGTERM');
+ } catch (err) {
+ const code = (err as NodeJS.ErrnoException).code;
+ if (code === 'ESRCH') return { ok: true, outcome: 'already-gone' };
+ return { ok: false, reason: code === 'EPERM' ? 'eperm' : 'other' };
}
+ if (await this.awaitExitWithinGrace(exited)) {
+ return { ok: true, outcome: 'stopped', escalated: false };
+ }
+ if (exited()) return { ok: true, outcome: 'stopped', escalated: false };
try {
- this.deps.killProbe(pid, 'SIGKILL');
- return { ok: true, escalated: true };
+ server.signal('SIGKILL');
+ return { ok: true, outcome: 'stopped', escalated: true };
} catch (err) {
const code = (err as NodeJS.ErrnoException).code;
- if (code === 'ESRCH') return { ok: true, escalated: true };
+ if (code === 'ESRCH') return { ok: true, outcome: 'stopped', escalated: true };
return { ok: false, reason: code === 'EPERM' ? 'eperm' : 'other' };
}
}
+ private async awaitExitWithinGrace(hasExited: () => boolean): Promise<boolean> {
+ const graceMs = this.deps.sigtermGraceMs ?? RESTART_SIGTERM_GRACE_MS;
+ const deadline = Date.now() + graceMs;
+ while (Date.now() < deadline) {
+ if (hasExited()) return true;
+ await new Promise<void>((resolveSleep) => {
+ this.deps.setTimeout(() => resolveSleep(), DEFAULT_SIGTERM_POLL_MS);
+ });
+ }
+ return false;
+ }
+
private async breakUnservingHolderLock(args: {
lockDir: string;
lock: Pick<ServerLockMetadataLike, 'pid' | 'url'> & { port: unknown };
@@ -745,7 +931,7 @@ export class WindowManager {
event,
outcome: 'eperm-stale-lock-break-skipped',
reason: 'booting',
- pid: lock.pid,
+ lockPid: lock.pid,
projectPath,
},
'[window-manager] holder has not bound a port yet — leaving its lock alone',
@@ -757,7 +943,7 @@ export class WindowManager {
event,
outcome: 'eperm-stale-lock-break-decided',
reason: 'unservable-origin',
- pid: lock.pid,
+ lockPid: lock.pid,
port,
projectPath,
},
@@ -782,14 +968,14 @@ export class WindowManager {
broke = this.deps.removeServerLock?.(lockDir, { pid: lock.pid }) ?? false;
} catch (err) {
this.deps.log?.warn(
- { event, outcome: 'eperm-stale-lock-break-failed', err, pid: lock.pid, projectPath },
+ { event, outcome: 'eperm-stale-lock-break-failed', err, lockPid: lock.pid, projectPath },
'[window-manager] could not break the stale lock',
);
return false;
}
if (!broke) {
this.deps.log?.warn(
- { event, outcome: 'eperm-stale-lock-break-declined', pid: lock.pid, projectPath },
+ { event, outcome: 'eperm-stale-lock-break-declined', lockPid: lock.pid, projectPath },
'[window-manager] stale lock was not broken',
);
return false;
@@ -798,7 +984,7 @@ export class WindowManager {
{
event,
outcome: 'eperm-stale-lock-broken',
- pid: lock.pid,
+ lockPid: lock.pid,
port: lock.port,
projectPath,
},
@@ -825,24 +1011,18 @@ export class WindowManager {
expectedHolder?: { pid: number; channel: string },
): Promise<ForceStopOutcome> {
const lockDir = getLocalDir(resolve(projectPath));
- let pid: unknown;
- let rawPort: unknown;
- let rawUrl: unknown;
- let rawChannel: unknown;
+ let raw: RawServerLockFields | null;
try {
- const raw = JSON.parse(readFileSync(join(lockDir, 'server.lock'), 'utf-8')) as {
- pid?: unknown;
- port?: unknown;
- url?: unknown;
- channel?: unknown;
- };
- pid = raw?.pid;
- rawPort = raw?.port;
- rawUrl = raw?.url;
- rawChannel = raw?.channel;
+ raw = JSON.parse(
+ readFileSync(join(lockDir, 'server.lock'), 'utf-8'),
+ ) as RawServerLockFields | null;
} catch {
return { ok: true };
}
+ const pid = raw?.pid;
+ const rawPort = raw?.port;
+ const rawUrl = raw?.url;
+ const rawChannel = raw?.channel;
if (!isValidLockPidLocal(pid) || pid === process.pid) {
return { ok: true };
}
@@ -868,7 +1048,7 @@ export class WindowManager {
{
event: 'desktop-force-stop-conflicting-server',
outcome: 'holder-changed',
- pid,
+ lockPid: pid,
holderChannel,
expectedPid: expectedHolder.pid,
expectedHolderChannel: expectedHolder.channel,
@@ -878,7 +1058,16 @@ export class WindowManager {
);
return { ok: false, reason: 'holder-changed' };
}
- const term = await this.terminateServerByPid(pid);
+ const term = await this.terminateServerByPid({
+ kind: 'lock',
+ lock: {
+ pid,
+ startedAt: typeof raw?.startedAt === 'string' ? raw.startedAt : undefined,
+ machineId: typeof raw?.machineId === 'string' ? raw.machineId : undefined,
+ hostname: typeof raw?.hostname === 'string' ? raw.hostname : undefined,
+ processStart: raw?.processStart,
+ },
+ });
if (!term.ok && term.reason === 'eperm') {
const broke = await this.breakUnservingHolderLock({
lockDir,
@@ -895,14 +1084,15 @@ export class WindowManager {
this.deps.log?.info(
{
event: 'desktop-force-stop-conflicting-server',
- pid,
+ lockPid: pid,
holderChannel,
projectPath,
- outcome: term.ok ? 'stopped' : term.reason,
+ outcome: terminationOutcome(term),
},
'[window-manager] force-stopped conflicting server holder on user request',
);
- return term.ok ? { ok: true } : term;
+ if (term.ok) return { ok: true };
+ return term;
}
async restartAttachedServer(
@@ -924,7 +1114,12 @@ export class WindowManager {
return { ok: false, reason: 'other-channel', holderChannel: conflict.holderChannel };
}
if (lock && isValidLockPidLocal(lock.pid)) {
- const term = await this.terminateServerByPid(lock.pid);
+ const held = this.runningSpawnedServer(canonicalKey);
+ const target: TerminationTarget =
+ held !== undefined && held.pid === lock.pid
+ ? { kind: 'spawned', server: held }
+ : { kind: 'lock', lock };
+ const term = await this.terminateServerByPid(target);
if (!term.ok) {
const broke =
term.reason === 'eperm' &&
@@ -938,21 +1133,21 @@ export class WindowManager {
this.deps.log?.warn(
{
event: 'desktop-server-restart',
- outcome: term.reason,
- pid: lock.pid,
+ outcome: terminationOutcome(term),
+ lockPid: lock.pid,
projectPath: resolved,
},
'[window-manager] server restart could not terminate the attached server',
);
- return term;
+ return restartFailure(term);
}
} else {
this.deps.log?.info(
{
event: 'desktop-server-restart',
- outcome: 'terminated',
- escalated: term.escalated,
- pid: lock.pid,
+ outcome: terminationOutcome(term),
+ escalated: terminationEscalated(term),
+ lockPid: lock.pid,
appRuntime: this.deps.selfRuntimeVersion ?? null,
projectPath: resolved,
},
@@ -1202,17 +1397,17 @@ export class WindowManager {
throw otherChannelHolderError(attached, attachedConflict);
}
if (attached) {
- const isForeign = this.spawnedDetachedPids.get(canonicalKey) !== attached.pid;
+ const isForeign = this.runningSpawnedServer(canonicalKey)?.pid !== attached.pid;
let reclaimed = false;
if (this.deps.reclaimForeignServerInDev === true && isForeign) {
- const term = await this.terminateServerByPid(attached.pid);
+ const term = await this.terminateServerByPid({ kind: 'lock', lock: attached });
if (term.ok) {
this.deps.log?.info(
{
event: 'desktop-dev-reclaim',
- outcome: 'terminated',
- escalated: term.escalated,
- pid: attached.pid,
+ outcome: terminationOutcome(term),
+ escalated: terminationEscalated(term),
+ lockPid: attached.pid,
projectPath,
},
'[window-manager] dev-mode reclaimed foreign server; spawning fresh own-build server',
@@ -1222,8 +1417,8 @@ export class WindowManager {
this.deps.log?.warn(
{
event: 'desktop-dev-reclaim',
- outcome: term.reason,
- pid: attached.pid,
+ outcome: terminationOutcome(term),
+ lockPid: attached.pid,
projectPath,
},
'[window-manager] dev-mode reclaim could not terminate the foreign server; attaching to it instead',
@@ -1239,15 +1434,15 @@ export class WindowManager {
{ protocolVersion: selfProtocol, runtimeVersion: selfRuntime },
);
if (drift.relation === 'older' || drift.relation === 'newer') {
- const term = await this.terminateServerByPid(attached.pid);
+ const term = await this.terminateServerByPid({ kind: 'lock', lock: attached });
if (term.ok) {
this.deps.log?.info(
{
event: 'desktop-upgrade-reconcile',
- outcome: 'terminated',
- escalated: term.escalated,
+ outcome: terminationOutcome(term),
+ escalated: terminationEscalated(term),
relation: drift.relation,
- pid: attached.pid,
+ lockPid: attached.pid,
projectPath,
},
'[window-manager] first launch after upgrade — auto-terminated pre-upgrade server; spawning fresh own-version server',
@@ -1257,9 +1452,9 @@ export class WindowManager {
this.deps.log?.warn(
{
event: 'desktop-upgrade-reconcile',
- outcome: term.reason,
+ outcome: terminationOutcome(term),
relation: drift.relation,
- pid: attached.pid,
+ lockPid: attached.pid,
projectPath,
},
'[window-manager] upgrade auto-terminate failed; attaching to the pre-upgrade server (the manual version-drift prompt still offers a restart)',
@@ -1299,7 +1494,7 @@ export class WindowManager {
contentDir: projectPath,
reactShellDistDir,
});
- this.spawnedDetachedPids.set(canonicalKey, handle.pid);
+ this.spawnedDetachedServers.set(canonicalKey, handle);
const POLL_DEADLINE_MS = this.deps.spawnLockPollDeadlineMs ?? SPAWN_STARTUP_DEADLINE_MS;
const { lock, waitedDeadlineMs } = await this.pollServerLock(
lockDir,
@@ -1308,16 +1503,13 @@ export class WindowManager {
this.deps.spawnLockProgressDeadlineMs,
);
if (lock === null) {
- const childExited = this.deps.isProcessAlive
- ? !this.deps.isProcessAlive(handle.pid)
- : undefined;
- const childExit = handle.readExit?.() ?? null;
+ const { exit: childExit, exited: childExited } = this.heldChildExit(handle);
this.sigtermSpawnOrphan(
- handle.pid,
+ handle,
projectPath,
'[window-manager] SIGTERM on orphan after spawn-lock-timeout failed',
);
- this.spawnedDetachedPids.delete(canonicalKey);
+ this.spawnedDetachedServers.delete(canonicalKey);
throw this.buildSpawnFailureError({
pid: handle.pid,
exit: childExit,
@@ -1330,16 +1522,13 @@ export class WindowManager {
const adoptedForeignLock = lock.pid !== handle.pid;
const unusableLock = !lockIsAttachable(lock);
if (unusableLock || (adoptedForeignLock && !(await this.probeForeignLockWithGrace(lock)))) {
- const childExited = this.deps.isProcessAlive
- ? !this.deps.isProcessAlive(handle.pid)
- : undefined;
- const childExit = handle.readExit?.() ?? null;
+ const { exit: childExit, exited: childExited } = this.heldChildExit(handle);
this.sigtermSpawnOrphan(
- handle.pid,
+ handle,
projectPath,
'[window-manager] SIGTERM on orphan after refusing a stale lock failed',
);
- this.spawnedDetachedPids.delete(canonicalKey);
+ this.spawnedDetachedServers.delete(canonicalKey);
const staleConflict = adoptedForeignLock ? this.otherChannelOf(lock) : null;
this.deps.log?.warn(
{
@@ -1385,11 +1574,11 @@ export class WindowManager {
const adoptedConflict = adoptedForeignLock ? this.otherChannelOf(lock) : null;
if (adoptedConflict !== null) {
this.sigtermSpawnOrphan(
- handle.pid,
+ handle,
projectPath,
"[window-manager] SIGTERM on orphan after refusing another channel's server failed",
);
- this.spawnedDetachedPids.delete(canonicalKey);
+ this.spawnedDetachedServers.delete(canonicalKey);
this.logOtherChannelRefusal({
lock,
holderChannel: adoptedConflict.holderChannel,
@@ -1484,19 +1673,6 @@ export class WindowManager {
this.deps.recordServerExit?.({ lockDir, pid: utility.pid ?? null, code });
this.windowsByPath.delete(canonicalKey);
this.deps.onUtilityExit?.(utility);
- const pid = utility.pid;
- if (typeof pid === 'number') {
- this.deps.setTimeout(() => {
- try {
- this.deps.killProbe(pid, 0);
- this.deps.log?.warn(
- { pid },
- 'utility pid still alive 1s after exit event — sending SIGTERM',
- );
- this.deps.killProbe(pid, 'SIGTERM');
- } catch {}
- }, 1000);
- }
});
const additionalArguments = [
@@ -1747,7 +1923,7 @@ export class WindowManager {
event: 'desktop-ephemeral-stale-entry',
canonicalKey,
windowAlive,
- pid: existing.ephemeral?.pid,
+ pid: existing.ephemeral?.server.pid,
apiOrigin: existing.apiOrigin,
lockDir: existing.ephemeral?.lockDir,
},
@@ -1821,12 +1997,9 @@ export class WindowManager {
this.deps.spawnLockProgressDeadlineMs,
);
if (lock === null) {
- const childExited = this.deps.isProcessAlive
- ? !this.deps.isProcessAlive(handle.pid)
- : undefined;
- const childExit = handle.readExit?.() ?? null;
+ const { exit: childExit, exited: childExited } = this.heldChildExit(handle);
try {
- this.deps.killProbe(handle.pid, 'SIGTERM');
+ handle.signal('SIGTERM');
} catch (signalErr) {
const code = (signalErr as NodeJS.ErrnoException).code;
if (code !== 'ESRCH') {
@@ -1894,7 +2067,7 @@ export class WindowManager {
window,
utility: null,
ownsServer: false,
- ephemeral: { projectDir: tempProjectDir, pid: handle.pid, lockDir },
+ ephemeral: { projectDir: tempProjectDir, server: handle, lockDir },
};
let watchHandle: unknown;
let watchCleared = false;
@@ -1919,7 +2092,7 @@ export class WindowManager {
window.destroy?.();
await this.teardownEphemeralSession({
projectDir: tempProjectDir,
- pid: handle.pid,
+ server: handle,
lockDir,
});
throw err;
@@ -1967,7 +2140,7 @@ export class WindowManager {
this.deps.log?.warn(
{
event: 'desktop-ephemeral-server-exited',
- pid: eph.pid,
+ pid: eph.server.pid,
lockDir: eph.lockDir,
file: opts.canonicalFilePath,
},
@@ -1981,7 +2154,7 @@ export class WindowManager {
event: 'desktop-ephemeral-watch-probe-failed',
err,
file: opts.canonicalFilePath,
- pid: context.ephemeral?.pid,
+ pid: context.ephemeral?.server.pid,
lockDir: context.ephemeral?.lockDir,
},
'[window-manager] ephemeral server-liveness probe threw — skipping this poll cycle',
@@ -1995,17 +2168,17 @@ export class WindowManager {
private async teardownEphemeralSession(session: {
projectDir: string;
- pid: number;
+ server: SpawnedServer;
lockDir: string;
}): Promise<void> {
try {
- const term = await this.terminateServerByPid(session.pid);
+ const term = await this.terminateServerByPid({ kind: 'spawned', server: session.server });
if (!term.ok) {
this.deps.log?.warn(
{
event: 'desktop-ephemeral-teardown',
- outcome: term.reason,
- pid: session.pid,
+ outcome: terminationOutcome(term),
+ pid: session.server.pid,
projectDir: session.projectDir,
},
'[window-manager] ephemeral server termination did not confirm; removing temp dir anyway',
@@ -2026,7 +2199,7 @@ export class WindowManager {
{
event: 'desktop-ephemeral-teardown-unexpected',
err,
- pid: session.pid,
+ pid: session.server.pid,
projectDir: session.projectDir,
},
'[window-manager] unexpected error tearing down ephemeral session',
@@ -2114,7 +2287,7 @@ export class WindowManager {
private async pollServerLock(
lockDir: string,
deadlineMs: number,
- child?: { pid: number },
+ child?: Pick<SpawnedServer, 'pid' | 'readExit'>,
progressDeadlineMs?: number,
): Promise<{ lock: ServerLockMetadataLike | null; waitedDeadlineMs: number }> {
const POLL_INTERVAL_MS = 50;
@@ -2132,13 +2305,16 @@ export class WindowManager {
if (lock !== null && lock.draining !== true && lock.port > 0 && lock.kind !== undefined) {
return { lock, waitedDeadlineMs: effectiveDeadlineMs };
}
- if (child !== undefined && isAlive !== undefined && !isAlive(child.pid)) {
+ if (child !== undefined && child.readExit() !== null) {
const winnerStillStarting =
- lock !== null && lock.draining !== true && lock.pid !== child.pid && isAlive(lock.pid);
+ lock !== null &&
+ lock.draining !== true &&
+ lock.pid !== child.pid &&
+ isAlive?.(lock.pid) === true;
if (!winnerStillStarting) return { lock: null, waitedDeadlineMs: effectiveDeadlineMs };
}
if (Date.now() >= deadline) {
- const childStarting = child !== undefined && isAlive?.(child.pid) === true;
+ const childStarting = child !== undefined && child.readExit() === null;
const holderStarting =
lock !== null && lock.draining !== true && isAlive?.(lock.pid) === true;
const stillStarting =
@@ -2169,7 +2345,7 @@ export class WindowManager {
pid: child?.pid,
lockDir,
elapsedMs: nowMs - started,
- childAlive: child !== undefined ? isAlive?.(child.pid) === true : undefined,
+ childAlive: child !== undefined ? child.readExit() === null : undefined,
lockPublished: lock !== null,
extended,
},
diff --git a/packages/desktop/src/utility/server-entry.ts b/packages/desktop/src/utility/server-entry.ts
index 3abc4d962..abb023e14 100644
--- a/packages/desktop/src/utility/server-entry.ts
+++ b/packages/desktop/src/utility/server-entry.ts
@@ -223,6 +223,7 @@ export function setupUtility(deps: SetupUtilityDeps): UtilityHandle {
import('@inkeep/open-knowledge-server'),
]);
markInitPhase('imports-resolved');
+ cli.registerWindowsProcessStartReader(serverNamespace.getLogger('process-start'));
const {
detectGh,
detectGhAccounts,
diff --git a/packages/desktop/tests/main/held-child-server.test.ts b/packages/desktop/tests/main/held-child-server.test.ts
new file mode 100644
index 000000000..6cb91ee16
--- /dev/null
+++ b/packages/desktop/tests/main/held-child-server.test.ts
@@ -0,0 +1,65 @@
+import { ChildProcess } from 'node:child_process';
+import { constants } from 'node:os';
+import { getSystemErrorMap } from 'node:util';
+import { describe, expect, test, vi } from 'vitest';
+import { heldChildServer } from '../../src/main/window-manager.ts';
+
+function libuvErrno(name: string): number {
+ for (const [errno, [errName]] of getSystemErrorMap()) if (errName === name) return errno;
+ throw new Error(`libuv has no error named ${name}`);
+}
+
+function childWithHandle(handle: { kill(signal: number): number } | null): ChildProcess {
+ const child = Object.create(ChildProcess.prototype) as ChildProcess & { _handle: unknown };
+ child._handle = handle;
+ child.on('error', () => {});
+ return child;
+}
+
+function thrownBy(signal: () => void): unknown {
+ try {
+ signal();
+ } catch (err) {
+ return err;
+ }
+ return undefined;
+}
+
+describe('heldChildServer', () => {
+ test('delivers the signal through the held child', () => {
+ const kill = vi.fn((_signal: number) => 0);
+ const server = heldChildServer(childWithHandle({ kill }), 4242, () => null);
+
+ expect(thrownBy(() => server.signal('SIGKILL'))).toBeUndefined();
+ expect(kill).toHaveBeenCalledWith(constants.signals.SIGKILL);
+ });
+
+ test('throws the errno the operating system refused the signal with', () => {
+ const server = heldChildServer(
+ childWithHandle({ kill: () => libuvErrno('EPERM') }),
+ 4242,
+ () => null,
+ );
+
+ expect(thrownBy(() => server.signal('SIGKILL'))).toMatchObject({
+ code: 'EPERM',
+ syscall: 'kill',
+ });
+ });
+
+ test('throws ESRCH when the operating system reports no such process', () => {
+ const server = heldChildServer(
+ childWithHandle({ kill: () => libuvErrno('ESRCH') }),
+ 4242,
+ () => null,
+ );
+
+ expect(thrownBy(() => server.signal('SIGTERM'))).toMatchObject({ code: 'ESRCH' });
+ });
+
+ test('throws ESRCH once the held child has exited and released its handle', () => {
+ const server = heldChildServer(childWithHandle(null), 4242, () => ({ code: 0, signal: null }));
+
+ expect(thrownBy(() => server.signal('SIGTERM'))).toMatchObject({ code: 'ESRCH' });
+ });
+});
diff --git a/packages/desktop/tests/main/held-child.test-helper.ts b/packages/desktop/tests/main/held-child.test-helper.ts
new file mode 100644
index 000000000..244161a11
--- /dev/null
+++ b/packages/desktop/tests/main/held-child.test-helper.ts
@@ -0,0 +1,32 @@
+import { EventEmitter } from 'node:events';
+import {
+ heldChildServer,
+ type SpawnedServer,
+ type WindowManagerDeps,
+} from '../../src/main/window-manager.ts';
+
+type ExitRecord = ReturnType<SpawnedServer['readExit']>;
+
+export function heldChild(
+ deps: Pick<WindowManagerDeps, 'killProbe' | 'isProcessAlive'>,
+ pid: number,
+ readExit?: () => ExitRecord,
+): SpawnedServer {
+ const exitRecord =
+ readExit ??
+ ((): ExitRecord => (deps.isProcessAlive?.(pid) === false ? { code: 0, signal: null } : null));
+ const child = Object.assign(new EventEmitter(), {
+ kill: (signal: NodeJS.Signals): boolean => {
+ if (exitRecord() !== null) return false;
+ try {
+ deps.killProbe(pid, signal);
+ return true;
+ } catch (err) {
+ if ((err as NodeJS.ErrnoException).code !== 'ESRCH') child.emit('error', err);
+ return false;
+ }
+ },
+ });
+ child.on('error', () => {});
+ return heldChildServer(child, pid, exitRecord);
+}
diff --git a/packages/desktop/tests/main/window-manager-ephemeral.test.ts b/packages/desktop/tests/main/window-manager-ephemeral.test.ts
index 0887e5aea..0a6995f8a 100644
--- a/packages/desktop/tests/main/window-manager-ephemeral.test.ts
+++ b/packages/desktop/tests/main/window-manager-ephemeral.test.ts
@@ -8,9 +8,11 @@ import {
type EphemeralOpenIdentity,
type ServerLockMetadataLike,
SPAWN_WAIT_HEARTBEAT_MS,
+ type SpawnedServer,
WindowManager,
type WindowManagerDeps,
} from '../../src/main/window-manager.ts';
+import { heldChild } from './held-child.test-helper.ts';
interface FakeWindow extends BrowserWindowLike {
fireClose: () => void;
@@ -171,7 +173,7 @@ function buildEphemeralEnv(): EphemeralEnv {
capabilities: ['ws'],
});
}
- return { pid };
+ return heldChild(env.deps, pid);
},
},
};
@@ -240,7 +242,7 @@ describe('createEphemeralWindow', () => {
expect(ctx.ephemeral).toEqual({
projectDir: '/tmp/ok-ephemeral-1',
- pid: 42001,
+ server: expect.objectContaining({ pid: 42001 }),
lockDir: getLocalDir('/tmp/ok-ephemeral-1'),
});
@@ -364,7 +366,7 @@ describe('createEphemeralWindow', () => {
recordingKill(pid, signal);
if (signal === 'SIGTERM') exitRecord = { code: null, signal: 'SIGTERM' };
};
- env.deps.spawnSingleFileServer = async () => ({ pid: 42001, readExit: () => exitRecord });
+ env.deps.spawnSingleFileServer = async () => heldChild(env.deps, 42001, () => exitRecord);
const wm = new WindowManager(env.deps);
const err = await wm
@@ -466,6 +468,30 @@ describe('createEphemeralWindow', () => {
expect(env.removedDirs).toEqual(['/tmp/ok-ephemeral-1']);
});
+ test('closing the window after its ephemeral server exited signals nothing to the reused pid', async () => {
+ let exit: { code: number | null; signal: string | null } | null = null;
+ const spawn = env.deps.spawnSingleFileServer;
+ if (!spawn) throw new Error('spawnSingleFileServer is not wired');
+ env.deps.spawnSingleFileServer = async (opts) => ({
+ ...(await spawn(opts)),
+ readExit: () => exit,
+ signal: () => {},
+ });
+ const wm = new WindowManager(env.deps);
+ await wm.createEphemeralWindow({
+ canonicalFilePath: FILE,
+ contentDir: PARENT,
+ docName: 'todo',
+ });
+ exit = { code: 1, signal: null };
+
+ env.windows[0]?.fireClose();
+ await wait(20);
+
+ expect(env.killCalls.filter((k) => k.signal !== 0)).toEqual([]);
+ expect(env.removedDirs).toEqual(['/tmp/ok-ephemeral-1']);
+ });
+
test('stopAllOwnedServers reaps an open ephemeral session (server + temp dir)', async () => {
const wm = new WindowManager(env.deps);
await wm.createEphemeralWindow({
@@ -482,10 +508,9 @@ describe('createEphemeralWindow', () => {
test('signalStopAllOwnedServers (before-quit-for-update) SIGTERMs detached + ephemeral pids and drains the detached map', async () => {
const wm = new WindowManager(env.deps);
- (wm as unknown as { spawnedDetachedPids: Map<string, number> }).spawnedDetachedPids.set(
- '/proj/detached',
- 77001,
- );
+ (
+ wm as unknown as { spawnedDetachedServers: Map<string, SpawnedServer> }
+ ).spawnedDetachedServers.set('/proj/detached', heldChild(env.deps, 77001));
await wm.createEphemeralWindow({
canonicalFilePath: FILE,
contentDir: PARENT,
@@ -523,7 +548,7 @@ describe('createEphemeralWindow', () => {
docName: 'todo',
});
- expect(ctx.ephemeral?.pid).toBe(42001);
+ expect(ctx.ephemeral?.server.pid).toBe(42001);
expect(env.spawnCalls).toHaveLength(1);
});
@@ -536,7 +561,7 @@ describe('createEphemeralWindow', () => {
});
expect(env.spawnCalls).toHaveLength(1);
const deadApiOrigin = first.apiOrigin;
- const deadPid = first.ephemeral?.pid;
+ const deadPid = first.ephemeral?.server.pid;
expect(deadPid).toBeDefined();
env.killServer(deadPid as number);
@@ -550,7 +575,7 @@ describe('createEphemeralWindow', () => {
expect(env.spawnCalls).toHaveLength(2);
expect(second.apiOrigin).not.toBe(deadApiOrigin);
- expect(second.ephemeral?.pid).not.toBe(deadPid);
+ expect(second.ephemeral?.server.pid).not.toBe(deadPid);
expect(wm.getWindowFor(FILE)?.apiOrigin).toBe(second.apiOrigin);
expect(env.removedDirs).toContain(first.ephemeral?.projectDir);
});
@@ -581,7 +606,7 @@ describe('createEphemeralWindow', () => {
contentDir: PARENT,
docName: 'todo',
});
- const pid = ctx.ephemeral?.pid as number;
+ const pid = ctx.ephemeral?.server.pid as number;
expect(wm.getWindowFor(FILE)).toBe(ctx);
tick(intervals, 1);
@@ -638,7 +663,7 @@ describe('createEphemeralWindow', () => {
const realReader = env.deps.readServerLock as NonNullable<WindowManagerDeps['readServerLock']>;
const stale = realReader(lockDir) as ServerLockMetadataLike;
- env.killServer(first.ephemeral?.pid as number);
+ env.killServer(first.ephemeral?.server.pid as number);
env.deps.readServerLock = (dir) => (dir === lockDir ? stale : realReader(dir));
const second = await wm.createEphemeralWindow({
@@ -659,7 +684,7 @@ describe('createEphemeralWindow', () => {
contentDir: PARENT,
docName: 'todo',
});
- env.killServer(first.ephemeral?.pid as number);
+ env.killServer(first.ephemeral?.server.pid as number);
const second = await wm.createEphemeralWindow({
canonicalFilePath: FILE,
contentDir: PARENT,
@@ -705,7 +730,7 @@ describe('createEphemeralWindow', () => {
contentDir: PARENT,
docName: 'todo',
});
- env.killServer(first.ephemeral?.pid as number);
+ env.killServer(first.ephemeral?.server.pid as number);
const second = await wm.createEphemeralWindow({
canonicalFilePath: FILE,
contentDir: PARENT,
@@ -720,7 +745,7 @@ describe('createEphemeralWindow', () => {
expect(wm.getWindowFor(FILE)).toBe(second);
expect(env.removedDirs).not.toContain(second.ephemeral?.projectDir);
- expect(env.killCalls.filter((k) => k.pid === second.ephemeral?.pid)).toEqual([]);
+ expect(env.killCalls.filter((k) => k.pid === second.ephemeral?.server.pid)).toEqual([]);
});
});
@@ -788,7 +813,7 @@ describe('ephemeral keepalive lifecycle', () => {
contentDir: PARENT,
docName: 'todo',
});
- env.killServer(first.ephemeral?.pid as number);
+ env.killServer(first.ephemeral?.server.pid as number);
await wm.createEphemeralWindow({
canonicalFilePath: FILE,
contentDir: PARENT,
@@ -824,7 +849,7 @@ describe('ephemeral keepalive lifecycle', () => {
});
expect(ka.handles[0]?.closed).toBe(false);
- env.killServer(ctx.ephemeral?.pid as number);
+ env.killServer(ctx.ephemeral?.server.pid as number);
tick(intervals, 1);
await wait(20);
@@ -843,7 +868,7 @@ describe('ephemeral keepalive lifecycle', () => {
});
expect(ka.handles[0]?.closed).toBe(false);
- env.killServer(first.ephemeral?.pid as number);
+ env.killServer(first.ephemeral?.server.pid as number);
env.deps.spawnSingleFileServer = async () => {
throw Object.assign(new Error('spawn boom'), { kind: 'spawn-error' });
};
@@ -875,7 +900,7 @@ describe('restartEphemeralServer', () => {
contentDir: PARENT,
docName: 'todo',
});
- const deadPid = first.ephemeral?.pid as number;
+ const deadPid = first.ephemeral?.server.pid as number;
const deadOrigin = first.apiOrigin;
const deadTempDir = first.ephemeral?.projectDir as string;
@@ -902,7 +927,7 @@ describe('restartEphemeralServer', () => {
contentDir: PARENT,
docName: 'todo',
});
- const livePid = first.ephemeral?.pid as number;
+ const livePid = first.ephemeral?.server.pid as number;
const liveTempDir = first.ephemeral?.projectDir as string;
const liveOrigin = first.apiOrigin;
@@ -930,14 +955,14 @@ describe('restartEphemeralServer', () => {
const orphanClose = vi.fn();
(env.windows[0] as FakeWindow).close = orphanClose as unknown as FakeWindow['close'];
- env.killServer(orphan.ephemeral?.pid as number);
+ env.killServer(orphan.ephemeral?.server.pid as number);
const live = await wm.createEphemeralWindow({
canonicalFilePath: FILE,
contentDir: PARENT,
docName: 'todo',
});
await wait(20);
- const livePid = live.ephemeral?.pid as number;
+ const livePid = live.ephemeral?.server.pid as number;
expect(wm.getWindowFor(FILE)).toBe(live);
expect(orphanClose).toHaveBeenCalled();
expect(wm.getEphemeralIdentityForWindow(orphan.window)).toBeDefined();
@@ -963,7 +988,7 @@ describe('restartEphemeralServer', () => {
});
const firstClose = vi.fn();
(env.windows[0] as FakeWindow).close = firstClose as unknown as FakeWindow['close'];
- env.killServer(first.ephemeral?.pid as number);
+ env.killServer(first.ephemeral?.server.pid as number);
const second = await wm.createEphemeralWindow({
canonicalFilePath: FILE,
@@ -973,7 +998,7 @@ describe('restartEphemeralServer', () => {
await wait(20);
const secondClose = vi.fn();
(env.windows[1] as FakeWindow).close = secondClose as unknown as FakeWindow['close'];
- env.killServer(second.ephemeral?.pid as number);
+ env.killServer(second.ephemeral?.server.pid as number);
firstClose.mockClear();
secondClose.mockClear();
@@ -997,7 +1022,7 @@ describe('restartEphemeralServer', () => {
contentDir: PARENT,
docName: 'todo',
});
- env.killServer(first.ephemeral?.pid as number);
+ env.killServer(first.ephemeral?.server.pid as number);
const reopened = await wm.createEphemeralWindow({
canonicalFilePath: FILE,
@@ -1022,7 +1047,7 @@ describe('restartEphemeralServer', () => {
contentDir: PARENT,
docName: 'todo',
});
- env.killServer(first.ephemeral?.pid as number);
+ env.killServer(first.ephemeral?.server.pid as number);
const reopened = await wm.createEphemeralWindow({
canonicalFilePath: FILE,
contentDir: PARENT,
@@ -1031,7 +1056,7 @@ describe('restartEphemeralServer', () => {
await wait(20);
expect(env.windows[0]?.isDestroyed()).toBe(true);
- env.killServer(reopened.ephemeral?.pid as number);
+ env.killServer(reopened.ephemeral?.server.pid as number);
const outcome = await wm.restartEphemeralServer(
identityOf(wm, reopened.window),
@@ -1056,7 +1081,7 @@ describe('restartEphemeralServer', () => {
docName: 'todo',
});
const identity = identityOf(wm, first.window);
- env.killServer(first.ephemeral?.pid as number);
+ env.killServer(first.ephemeral?.server.pid as number);
env.deps.spawnSingleFileServer = async () => {
throw Object.assign(new Error('spawn boom'), { kind: 'spawn-error' });
};
@@ -1102,7 +1127,7 @@ describe('restartServerForWindow (IPC routing seam)', () => {
contentDir: PARENT,
docName: 'todo',
});
- env.killServer(first.ephemeral?.pid as number);
+ env.killServer(first.ephemeral?.server.pid as number);
const outcome = await wm.restartServerForWindow(first.window, '/some/unrelated/project', {});
await wait(20);
diff --git a/packages/desktop/tests/main/window-manager-lock-pid-reuse.test.ts b/packages/desktop/tests/main/window-manager-lock-pid-reuse.test.ts
new file mode 100644
index 000000000..de15a698d
--- /dev/null
+++ b/packages/desktop/tests/main/window-manager-lock-pid-reuse.test.ts
@@ -0,0 +1,124 @@
+import { mkdirSync, mkdtempSync, realpathSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import {
+ getLocalDir,
+ getMachineId,
+ isProcessAlive,
+ readServerLock,
+} from '@inkeep/open-knowledge-server';
+import { afterEach, describe, expect, test } from 'vitest';
+import {
+ legacyServerLock,
+ lockWrittenBeforeProcessStarted,
+ lockWrittenByRunningProcess,
+ type OwnedProcess,
+ startOwnedProcess,
+ stopOwnedProcess,
+} from '../../../../test-support/owned-process.test-helper.ts';
+import { WindowManager, type WindowManagerDeps } from '../../src/main/window-manager.ts';
+
+interface SignalRecord {
+ pid: number;
+ signal: number | NodeJS.Signals;
+}
+
+const owned: OwnedProcess[] = [];
+const projects: string[] = [];
+
+afterEach(async () => {
+ for (const holder of owned.splice(0)) await stopOwnedProcess(holder);
+ for (const project of projects.splice(0)) rmSync(project, { recursive: true, force: true });
+});
+
+async function seedProject(
+ startedAt: (holder: OwnedProcess) => string,
+): Promise<{ projectPath: string; holder: OwnedProcess }> {
+ const holder = await startOwnedProcess();
+ owned.push(holder);
+ const projectPath = realpathSync(mkdtempSync(join(tmpdir(), 'ok-desktop-lock-pid-reuse-')));
+ projects.push(projectPath);
+ const lockDir = getLocalDir(projectPath);
+ mkdirSync(lockDir, { recursive: true });
+ writeFileSync(
+ join(lockDir, 'server.lock'),
+ JSON.stringify(
+ legacyServerLock({
+ pid: holder.pid,
+ startedAt: startedAt(holder),
+ machineId: getMachineId(),
+ worktreeRoot: projectPath,
+ }),
+ ),
+ { mode: 0o600 },
+ );
+ return { projectPath, holder };
+}
+
+function windowManagerWithLockReaders(signals: SignalRecord[]): WindowManager {
+ const deps: WindowManagerDeps = {
+ createWindow: () => {
+ throw new Error('no project window is opened in this test');
+ },
+ forkUtility: () => {
+ throw new Error('no utility process is started in this test');
+ },
+ utilityEntryPath: '/fake/utility-entry.js',
+ rendererEntryPath: '/fake/renderer/index.html',
+ terminalAuthAvailable: true,
+ appVersion: '9.9.9-test',
+ setTimeout: (cb, ms) => setTimeout(cb, ms),
+ killProbe: (pid, signal) => {
+ signals.push({ pid, signal });
+ },
+ showGate: { register: () => () => {}, fireThemeApplied: () => {} },
+ readServerLock: (lockDir) => readServerLock(lockDir),
+ isProcessAlive: (pid) => isProcessAlive(pid),
+ sigtermGraceMs: 0,
+ };
+ return new WindowManager(deps);
+}
+
+function signalsTo(signals: SignalRecord[], pid: number): SignalRecord[] {
+ return signals.filter((record) => record.pid === pid && record.signal !== 0);
+}
+
+describe('desktop server termination when the lock pid now belongs to a process started after the lock was written', () => {
+ test('Stop Server & Retry signals nothing to that process', async () => {
+ const { projectPath, holder } = await seedProject(lockWrittenBeforeProcessStarted);
+ const signals: SignalRecord[] = [];
+
+ await windowManagerWithLockReaders(signals).forceStopConflictingServer(projectPath);
+
+ expect(signalsTo(signals, holder.pid)).toEqual([]);
+ });
+
+ test('restarting the attached server signals nothing to that process', async () => {
+ const { projectPath, holder } = await seedProject(lockWrittenBeforeProcessStarted);
+ const signals: SignalRecord[] = [];
+
+ await windowManagerWithLockReaders(signals).restartAttachedServer(projectPath);
+
+ expect(signalsTo(signals, holder.pid)).toEqual([]);
+ });
+});
+
+describe('desktop server termination when the lock is held by the live process that wrote it', () => {
+ test('Stop Server & Retry sends SIGTERM to the writer', async () => {
+ const { projectPath, holder } = await seedProject(lockWrittenByRunningProcess);
+ const signals: SignalRecord[] = [];
+
+ await windowManagerWithLockReaders(signals).forceStopConflictingServer(projectPath);
+
+ expect(signalsTo(signals, holder.pid)[0]).toEqual({ pid: holder.pid, signal: 'SIGTERM' });
+ });
+
+ test('restarting the attached server sends SIGTERM to the writer', async () => {
+ const { projectPath, holder } = await seedProject(lockWrittenByRunningProcess);
+ const signals: SignalRecord[] = [];
+
+ await windowManagerWithLockReaders(signals).restartAttachedServer(projectPath);
+
+ expect(signalsTo(signals, holder.pid)[0]).toEqual({ pid: holder.pid, signal: 'SIGTERM' });
+ });
+});
diff --git a/packages/desktop/tests/main/window-manager-safety-net.test.ts b/packages/desktop/tests/main/window-manager-safety-net.test.ts
index 7ce39d92f..313775f81 100644
--- a/packages/desktop/tests/main/window-manager-safety-net.test.ts
+++ b/packages/desktop/tests/main/window-manager-safety-net.test.ts
@@ -1,4 +1,4 @@
-import { getLocalDir } from '@inkeep/open-knowledge-server';
+import { getLocalDir, getMachineId } from '@inkeep/open-knowledge-server';
import { describe, expect, test, vi } from 'vitest';
import type { AssetOpenResult } from '../../src/main/asset-allowlist.ts';
import type { ShowGateRegistry } from '../../src/main/show-gate.ts';
@@ -9,6 +9,7 @@ import {
WindowManager,
type WindowManagerDeps,
} from '../../src/main/window-manager.ts';
+import { heldChild } from './held-child.test-helper.ts';
interface MockUtility extends UtilityProcessLike {
fire: (msg: unknown) => void;
@@ -127,6 +128,10 @@ function buildEnv(): TestEnv {
appVersion: '9.9.9-test',
setTimeout: () => null,
killProbe: vi.fn(() => {}),
+ readProcessStart: () => ({
+ startedAtMs: Date.parse('2026-01-01T00:00:00Z'),
+ clock: 'creation',
+ }),
showGate,
safetyNet: { openExternal, openAsset },
};
@@ -154,7 +159,7 @@ function withEphemeralDeps(env: TestEnv): void {
capabilities: ['ws'],
});
}
- return { pid: spawnedPid };
+ return heldChild(env.deps, spawnedPid);
};
}
@@ -214,6 +219,7 @@ describe('WindowManager factory attaches the external-link safety net', () => {
const liveLock: ServerLockMetadataLike = {
pid: 65792,
hostname: 'my-host',
+ machineId: getMachineId(),
port: 59534,
startedAt: '2026-04-17T20:23:20.713Z',
worktreeRoot: '/tmp/dragon',
@@ -239,7 +245,7 @@ describe('WindowManager factory attaches the external-link safety net', () => {
});
env.deps.spawnDetachedServer = async () => {
spawned = true;
- return { pid: 6666 };
+ return heldChild(env.deps, 6666);
};
const wm = new WindowManager(env.deps);
diff --git a/packages/desktop/tests/main/window-manager.test.ts b/packages/desktop/tests/main/window-manager.test.ts
index 6f315c540..db5e8b642 100644
--- a/packages/desktop/tests/main/window-manager.test.ts
+++ b/packages/desktop/tests/main/window-manager.test.ts
@@ -4,6 +4,7 @@ import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { setTimeout as wait } from 'node:timers/promises';
import { DEFAULT_SERVER_HOST, formatSpawnAttemptHeader } from '@inkeep/open-knowledge-core';
+import { getMachineId } from '@inkeep/open-knowledge-server';
import { afterEach, beforeEach, describe, expect, test, vi } from 'vitest';
import { failedOpenHolder, promptHolderStop } from '../../src/main/holder-stop-prompt.ts';
import { resolveLocalOpCliInvocation } from '../../src/main/local-op-cli-invocation.ts';
@@ -15,6 +16,7 @@ import {
isOtherChannelHolderError,
otherChannelStopOffer,
type ServerLockMetadataLike,
+ type SpawnedServer,
type UtilityProcessLike,
WindowManager,
type WindowManagerDeps,
@@ -32,6 +34,7 @@ import {
utilityInitPhaseEvent,
} from '../../src/shared/boot-narration.ts';
import { parseDeclaredPhaseBudget } from '../smoke/_helpers/launch-readiness.ts';
+import { heldChild } from './held-child.test-helper.ts';
interface MockUtility extends UtilityProcessLike {
fire: (msg: unknown) => void;
@@ -207,6 +210,10 @@ function buildEnv(): TestEnv {
return null;
},
killProbe,
+ readProcessStart: () => ({
+ startedAtMs: Date.parse('2026-01-01T00:00:00Z'),
+ clock: 'creation',
+ }),
activateApp,
showGate,
log: {
@@ -218,6 +225,15 @@ function buildEnv(): TestEnv {
};
}
+const LOCK_HOLDER_EVENTS = new Set([
+ 'desktop-server-restart',
+ 'desktop-force-stop-conflicting-server',
+ 'desktop-dev-reclaim',
+ 'desktop-upgrade-reconcile',
+ 'desktop-lock-holder-replaced',
+ 'desktop-lock-holder-unverified',
+]);
+
describe('WindowManager', () => {
let env: TestEnv;
@@ -225,6 +241,18 @@ describe('WindowManager', () => {
env = buildEnv();
});
+ afterEach(() => {
+ const holderRecords = env.logEntries
+ .map((entry) => entry.payload as Record<string, unknown>)
+ .filter((payload) => LOCK_HOLDER_EVENTS.has(String(payload.event)));
+ for (const payload of holderRecords) {
+ expect(
+ payload,
+ 'a lock-holder record names its holder under lockPid, not pid',
+ ).not.toHaveProperty('pid');
+ }
+ });
+
test('createProjectWindow sets BrowserWindow title to "<projectName> — OpenKnowledge" (spawn path)', async () => {
const wm = new WindowManager(env.deps);
const promise = wm.createProjectWindow({ projectPath: '/tmp/dragon-wiki' });
@@ -328,7 +356,7 @@ describe('WindowManager', () => {
});
test('single-file spawning does not switch project opens away from the utility', async () => {
- const singleFileSpawn = vi.fn(async () => ({ pid: 88001 }));
+ const singleFileSpawn = vi.fn(async () => heldChild(env.deps, 88001));
env.deps.spawnSingleFileServer = singleFileSpawn;
const wm = new WindowManager(env.deps);
@@ -516,7 +544,7 @@ describe('WindowManager', () => {
expect(env.utilities[0]?.postMessage).toHaveBeenCalledWith({ type: 'shutdown' });
});
- test('utility exit removes project from map AND schedules liveness probe (D39)', async () => {
+ test('utility exit removes project from map and schedules no follow-up signal', async () => {
const wm = new WindowManager(env.deps);
const p = wm.createProjectWindow({ projectPath: '/tmp/exit-test' });
env.utilities[0]?.fire({ type: 'ready', port: 51003, apiOrigin: 'http://localhost:51003' });
@@ -526,8 +554,7 @@ describe('WindowManager', () => {
env.utilities[0]?.fireExit(0);
expect(wm.windowCount()).toBe(0);
- const livenessProbe = env.timers.find((t) => t.ms === 1000);
- expect(livenessProbe).toBeDefined();
+ expect(env.timers.find((t) => t.ms === 1000)).toBeUndefined();
});
test('getOpenProjectPaths is empty with no project windows open', () => {
@@ -560,37 +587,18 @@ describe('WindowManager', () => {
expect(wm.getOpenProjectPaths()).toEqual(['/tmp/delta']);
});
- test('liveness probe sends SIGTERM if pid still alive 1s after exit', async () => {
+ test('the utility pid is never signalled after its exit event, even while that pid is alive', async () => {
const wm = new WindowManager(env.deps);
const p = wm.createProjectWindow({ projectPath: '/tmp/zombie-test' });
env.utilities[0]?.fire({ type: 'ready', port: 51004, apiOrigin: 'http://localhost:51004' });
await p;
- const utilityPid = env.utilities[0]?.pid;
+ const scheduledBeforeExit = env.timers.length;
env.utilities[0]?.fireExit(0);
- const livenessProbe = env.timers.find((t) => t.ms === 1000);
- expect(livenessProbe).toBeDefined();
-
- livenessProbe?.cb();
- expect(env.killProbe).toHaveBeenCalledWith(utilityPid, 0);
- expect(env.killProbe).toHaveBeenCalledWith(utilityPid, 'SIGTERM');
- });
-
- test('liveness probe is silent if pid is truly gone (probe throws)', async () => {
- env.killProbe = vi.fn(() => {
- throw new Error('No such process');
- });
- env.deps.killProbe = env.killProbe;
- const wm = new WindowManager(env.deps);
- const p = wm.createProjectWindow({ projectPath: '/tmp/clean-exit' });
- env.utilities[0]?.fire({ type: 'ready', port: 51005, apiOrigin: 'http://localhost:51005' });
- await p;
+ expect(wm.windowCount()).toBe(0);
+ for (let i = scheduledBeforeExit; i < env.timers.length; i++) env.timers[i]?.cb();
- env.utilities[0]?.fireExit(0);
- const livenessProbe = env.timers.find((t) => t.ms === 1000);
- expect(livenessProbe).toBeDefined();
- expect(() => livenessProbe?.cb()).not.toThrow();
- expect(env.killProbe).toHaveBeenCalledTimes(1);
+ expect(env.killProbe).not.toHaveBeenCalled();
});
test('runClean (when provided) is called before forking utility', async () => {
@@ -780,6 +788,7 @@ describe('WindowManager', () => {
const liveLock: ServerLockMetadataLike = {
pid: 65792,
hostname: 'my-host',
+ machineId: getMachineId(),
port: 59534,
startedAt: '2026-04-17T20:23:20.713Z',
worktreeRoot: '/tmp/dragon',
@@ -866,7 +875,7 @@ describe('WindowManager', () => {
readServerLock: () => stableLock,
probeWsUpgrade: () => Promise.resolve(false),
});
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: 22020 });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, 22020));
env.deps.spawnLockPollDeadlineMs = 50;
env.deps.spawnLockProgressDeadlineMs = 50;
@@ -920,7 +929,7 @@ describe('WindowManager', () => {
stopLog: [
{
event: 'desktop-force-stop-conflicting-server',
- pid: 65792,
+ lockPid: 65792,
holderChannel: 'stable',
outcome: 'stopped',
},
@@ -938,7 +947,7 @@ describe('WindowManager', () => {
{
event: 'desktop-force-stop-conflicting-server',
outcome: 'holder-changed',
- pid: 70001,
+ lockPid: 70001,
holderChannel: 'stable',
expectedPid: 65792,
expectedHolderChannel: 'stable',
@@ -1016,7 +1025,7 @@ describe('WindowManager', () => {
isProcessAlive: (pid) => !(pid === STALE_PID && stopped()),
probeWsUpgrade: (url) => Promise.resolve(!url.includes(String(stableLock.port))),
});
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: 22020 });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, 22020));
env.deps.spawnLockPollDeadlineMs = 50;
env.deps.spawnLockProgressDeadlineMs = 50;
const projectPath = mkdtempSync(join(tmpdir(), 'ok-other-channel-stale-'));
@@ -1492,7 +1501,7 @@ describe('WindowManager', () => {
env.deps.killProbe = killProbe;
env.deps.spawnDetachedServer = async () => {
spawned = true;
- return { pid: 6666 };
+ return heldChild(env.deps, 6666);
};
const restarted: Array<{ projectPath: string; apiOrigin: string }> = [];
@@ -1507,6 +1516,13 @@ describe('WindowManager', () => {
const outcome = await wm.restartAttachedServer('/tmp/dragon');
expect(outcome).toEqual({ ok: true });
expect(killProbe).toHaveBeenCalledWith(5555, 'SIGTERM');
+ expect(env.logEntries.map((entry) => entry.payload)).toContainEqual(
+ expect.objectContaining({
+ event: 'desktop-server-restart',
+ outcome: 'stopped',
+ lockPid: 5555,
+ }),
+ );
expect(env.windows.length).toBe(2);
const ctx = wm.getContextForBrowserWindow(env.windows[1] as BrowserWindowLike);
expect(ctx?.port).toBe(60000);
@@ -1549,7 +1565,7 @@ describe('WindowManager', () => {
});
env.deps.spawnDetachedServer = async () => {
spawned = true;
- return { pid: 6666 };
+ return heldChild(env.deps, 6666);
};
env.deps.onProjectServerRestarted = () => {
throw new Error('recreate boom');
@@ -1602,7 +1618,7 @@ describe('WindowManager', () => {
env.deps.removeServerLock = removeServerLock;
env.deps.spawnDetachedServer = () => {
spawned = true;
- return Promise.resolve({ pid: FRESH_PID });
+ return Promise.resolve(heldChild(env.deps, FRESH_PID));
};
const wm = new WindowManager(env.deps);
@@ -1617,6 +1633,13 @@ describe('WindowManager', () => {
expect(removeServerLock.mock.calls[0]?.[0]).toMatch(/[/\\]\.ok[/\\]local$/);
expect(removeServerLock.mock.calls[0]?.[0]).toContain('/tmp/dragon');
expect(removeServerLock.mock.calls[0]?.[1]).toEqual({ pid: STALE_PID });
+ expect(env.logEntries.map((entry) => entry.payload)).toContainEqual(
+ expect.objectContaining({
+ event: 'desktop-server-restart',
+ outcome: 'eperm-stale-lock-broken',
+ lockPid: STALE_PID,
+ }),
+ );
});
test('restartAttachedServer survives a removeServerLock that throws', async () => {
@@ -1631,7 +1654,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(false);
- const spawn = vi.fn(() => Promise.resolve({ pid: 22018 }));
+ const spawn = vi.fn(() => Promise.resolve(heldChild(env.deps, 22018)));
env.deps.spawnDetachedServer = spawn;
env.deps.killProbe = vi.fn(() => {
throw Object.assign(new Error('eperm'), { code: 'EPERM' });
@@ -1656,7 +1679,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(false);
- const spawn = vi.fn(() => Promise.resolve({ pid: 22018 }));
+ const spawn = vi.fn(() => Promise.resolve(heldChild(env.deps, 22018)));
env.deps.spawnDetachedServer = spawn;
env.deps.killProbe = vi.fn(() => {
throw Object.assign(new Error('eperm'), { code: 'EPERM' });
@@ -1744,7 +1767,7 @@ describe('WindowManager', () => {
env.deps.probeWsUpgrade = () => Promise.resolve(true);
env.deps.spawnDetachedServer = () => {
spawned = true;
- return Promise.resolve({ pid: 22018 });
+ return Promise.resolve(heldChild(env.deps, 22018));
};
env.deps.killProbe = vi.fn(() => {
throw Object.assign(new Error('eperm'), { code: 'EPERM' });
@@ -1820,7 +1843,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(false);
- const spawn = vi.fn(() => Promise.resolve({ pid: 22018 }));
+ const spawn = vi.fn(() => Promise.resolve(heldChild(env.deps, 22018)));
env.deps.spawnDetachedServer = spawn;
env.deps.killProbe = vi.fn(() => {
throw Object.assign(new Error('kaboom'), { code: 'EIO' });
@@ -1835,6 +1858,119 @@ describe('WindowManager', () => {
expect(env.windows.length).toBe(0);
});
+ test('restartAttachedServer reports an unverifiable holder as a failure without signalling or spawning', async () => {
+ const removeServerLock = vi.fn(() => true);
+ env.deps.removeServerLock = removeServerLock;
+ env.deps.readServerLock = () => ({ ...liveLock, pid: 5151, port: 42118 });
+ env.deps.isProcessAlive = () => true;
+ env.deps.readProcessStart = () => null;
+ const spawn = vi.fn(() => Promise.resolve(heldChild(env.deps, 22019)));
+ env.deps.spawnDetachedServer = spawn;
+ const killProbe = vi.fn();
+ env.deps.killProbe = killProbe;
+
+ const outcome = await new WindowManager(env.deps).restartAttachedServer('/tmp/dragon');
+
+ expect(outcome).toEqual({ ok: false, reason: 'unverified', sigtermSent: false });
+ expect(killProbe.mock.calls.filter(([, signal]) => signal !== 0)).toEqual([]);
+ expect(removeServerLock).not.toHaveBeenCalled();
+ expect(spawn).not.toHaveBeenCalled();
+ expect(env.logEntries.map((entry) => entry.payload)).toContainEqual(
+ expect.objectContaining({
+ event: 'desktop-server-restart',
+ outcome: 'unverified',
+ lockPid: 5151,
+ }),
+ );
+ });
+
+ test('restartAttachedServer stops a server the desktop spawned through its held child even when the holder start is unreadable', async () => {
+ const OWN_CHILD_PID = 22030;
+ const FRESH_PID = 22031;
+ const ownLock = { ...liveLock, pid: OWN_CHILD_PID };
+ const freshLock = { ...liveLock, pid: FRESH_PID, port: 60000 };
+ let spawns = 0;
+ let ownChildStopped = false;
+ enableAttachProbe({
+ readServerLock: () => {
+ if (spawns === 0) return null;
+ if (spawns === 1) return ownChildStopped ? null : ownLock;
+ return freshLock;
+ },
+ isProcessAlive: (pid) => (pid === OWN_CHILD_PID ? !ownChildStopped : true),
+ });
+ env.deps.readProcessStart = () => null;
+ const killProbe = vi.fn((pid: number, signal: NodeJS.Signals | number) => {
+ if (pid === OWN_CHILD_PID && signal === 'SIGTERM') ownChildStopped = true;
+ });
+ env.deps.killProbe = killProbe;
+ const heldSignals: NodeJS.Signals[] = [];
+ env.deps.spawnDetachedServer = async () => {
+ spawns++;
+ if (spawns > 1) return heldChild(env.deps, FRESH_PID);
+ const own = heldChild(env.deps, OWN_CHILD_PID);
+ return {
+ ...own,
+ signal: (signal: NodeJS.Signals) => {
+ heldSignals.push(signal);
+ own.signal(signal);
+ },
+ };
+ };
+
+ const wm = new WindowManager(env.deps);
+ await wm.createProjectWindow({ projectPath: '/tmp/dragon' });
+ expect(spawns).toBe(1);
+
+ const outcome = await wm.restartAttachedServer('/tmp/dragon');
+
+ expect(outcome).toEqual({ ok: true });
+ expect(heldSignals).toEqual(['SIGTERM']);
+ expect(killProbe).toHaveBeenCalledWith(OWN_CHILD_PID, 'SIGTERM');
+ expect(spawns).toBe(2);
+ const recreated = env.windows[1];
+ if (!recreated) throw new Error('no recreated window');
+ expect(wm.getContextForBrowserWindow(recreated as BrowserWindowLike)?.port).toBe(60000);
+ });
+
+ test('restartAttachedServer takes the lock path when the lock names a pid other than the held child', async () => {
+ const OWN_CHILD_PID = 22030;
+ const OTHER_PID = 22040;
+ let lockPid = OWN_CHILD_PID;
+ let spawns = 0;
+ enableAttachProbe({
+ readServerLock: () => (spawns === 0 ? null : { ...liveLock, pid: lockPid }),
+ isProcessAlive: () => true,
+ });
+ env.deps.readProcessStart = () => null;
+ const killProbe = vi.fn();
+ env.deps.killProbe = killProbe;
+ const heldSignals: NodeJS.Signals[] = [];
+ env.deps.spawnDetachedServer = async () => {
+ spawns++;
+ const own = heldChild(env.deps, OWN_CHILD_PID);
+ return {
+ ...own,
+ signal: (signal: NodeJS.Signals) => {
+ heldSignals.push(signal);
+ own.signal(signal);
+ },
+ };
+ };
+
+ const wm = new WindowManager(env.deps);
+ await wm.createProjectWindow({ projectPath: '/tmp/dragon' });
+ expect(spawns).toBe(1);
+ lockPid = OTHER_PID;
+
+ const outcome = await wm.restartAttachedServer('/tmp/dragon');
+
+ expect(outcome).toEqual({ ok: false, reason: 'unverified', sigtermSent: false });
+ expect(heldSignals).toEqual([]);
+ expect(killProbe.mock.calls.filter(([, signal]) => signal !== 0)).toEqual([]);
+ expect(spawns).toBe(1);
+ });
+
test('restartAttachedServer still refuses eperm when the unkillable holder IS serving', async () => {
const liveHolderLock = {
...liveLock,
@@ -1842,7 +1978,7 @@ describe('WindowManager', () => {
protocolVersion: 1,
runtimeVersion: '0.8.0',
};
- const spawn = vi.fn(() => Promise.resolve({ pid: 22018 }));
+ const spawn = vi.fn(() => Promise.resolve(heldChild(env.deps, 22018)));
env.deps.readServerLock = () => liveHolderLock;
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
@@ -1925,7 +2061,7 @@ describe('WindowManager', () => {
rejectSpawn = reject;
});
spawned = true;
- return { pid: 6666 };
+ return heldChild(env.deps, 6666);
};
return {
release: () => resolveSpawn?.(),
@@ -2048,6 +2184,93 @@ describe('WindowManager', () => {
expect(lifecycleSends.length).toBe(0);
});
+ function holderReplacedAfterSigterm(lock: ServerLockMetadataLike) {
+ const signals: Array<{ pid: number; signal: NodeJS.Signals | number }> = [];
+ let sigtermSent = false;
+ env.deps.killProbe = (pid, signal) => {
+ if (signal === 'SIGTERM') sigtermSent = true;
+ signals.push({ pid, signal });
+ };
+ env.deps.sigtermGraceMs = 0;
+ env.deps.readProcessStart = () => ({
+ startedAtMs: Date.parse(sigtermSent ? '2026-07-08T00:00:00Z' : '2026-01-01T00:00:00Z'),
+ clock: 'creation',
+ });
+ enableAttachProbe({
+ readServerLock: () => (sigtermSent ? null : lock),
+ isProcessAlive: () => true,
+ });
+ return { signals };
+ }
+
+ test('dev reclaim checks the holder again before SIGKILL and spares a process that replaced it', async () => {
+ env.deps.reclaimForeignServerInDev = true;
+ const lock = { ...liveLock, pid: 64351, machineId: getMachineId() };
+ const { signals } = holderReplacedAfterSigterm(lock);
+
+ const wm = new WindowManager(env.deps);
+ const promise = wm.createProjectWindow({ projectPath: '/tmp/dragon' });
+ for (let i = 0; i < 50 && env.utilities.length === 0; i++) await wait(0);
+ env.utilities[0]?.fire({ type: 'ready', port: 52777, apiOrigin: 'http://localhost:52777' });
+ await promise;
+
+ expect(signals.filter((record) => record.signal !== 0)).toEqual([
+ { pid: 64351, signal: 'SIGTERM' },
+ ]);
+ expect(env.logEntries.map((entry) => entry.payload)).toContainEqual(
+ expect.objectContaining({ event: 'desktop-lock-holder-replaced', lockPid: 64351 }),
+ );
+ expect(env.logEntries.map((entry) => entry.payload)).toContainEqual(
+ expect.objectContaining({
+ event: 'desktop-dev-reclaim',
+ outcome: 'stopped',
+ lockPid: 64351,
+ }),
+ );
+ });
+
+ test('upgrade reconcile checks the holder again before SIGKILL and spares a process that replaced it', async () => {
+ env.deps.selfProtocolVersion = 1;
+ env.deps.selfRuntimeVersion = '0.8.2';
+ env.deps.isFirstLaunchAfterUpgrade = () => true;
+ const lock = {
+ ...liveLock,
+ pid: 64352,
+ machineId: getMachineId(),
+ protocolVersion: 1,
+ runtimeVersion: '0.8.0',
+ };
+ const { signals } = holderReplacedAfterSigterm(lock);
+ const freshLock = { ...lock, pid: 6666, port: 60000, runtimeVersion: '0.8.2' };
+ let spawned = false;
+ const readLock = env.deps.readServerLock;
+ env.deps.readServerLock = (dir) => (spawned ? freshLock : (readLock?.(dir) ?? null));
+ env.deps.spawnDetachedServer = async () => {
+ spawned = true;
+ return heldChild(env.deps, 6666);
+ };
+
+ const ctx = await new WindowManager(env.deps).createProjectWindow({
+ projectPath: '/tmp/dragon',
+ });
+
+ expect(ctx.port).toBe(60000);
+
+ expect(signals.filter((record) => record.signal !== 0)).toEqual([
+ { pid: 64352, signal: 'SIGTERM' },
+ ]);
+ expect(env.logEntries.map((entry) => entry.payload)).toContainEqual(
+ expect.objectContaining({ event: 'desktop-lock-holder-replaced', lockPid: 64352 }),
+ );
+ expect(env.logEntries.map((entry) => entry.payload)).toContainEqual(
+ expect.objectContaining({
+ event: 'desktop-upgrade-reconcile',
+ outcome: 'stopped',
+ lockPid: 64352,
+ }),
+ );
+ });
+
test('without reclaimForeignServerInDev (production default), a foreign server is attached — no termination, no notice', async () => {
const killProbe = vi.fn(() => {});
env.deps.killProbe = killProbe;
@@ -2091,7 +2314,7 @@ describe('WindowManager', () => {
env.deps.killProbe = killProbe;
env.deps.spawnDetachedServer = async () => {
spawned = true;
- return { pid: 6666 };
+ return heldChild(env.deps, 6666);
};
const wm = new WindowManager(env.deps);
@@ -2136,7 +2359,7 @@ describe('WindowManager', () => {
env.deps.killProbe = killProbe;
env.deps.spawnDetachedServer = async () => {
spawned = true;
- return { pid: 6666 };
+ return heldChild(env.deps, 6666);
};
const wm = new WindowManager(env.deps);
const ctx = await wm.createProjectWindow({ projectPath: '/tmp/dragon' });
@@ -2203,6 +2426,13 @@ describe('WindowManager', () => {
});
const wm = new WindowManager(env.deps);
const ctx = await wm.createProjectWindow({ projectPath: '/tmp/dragon' });
+ expect(env.logEntries.map((entry) => entry.payload)).toContainEqual(
+ expect.objectContaining({
+ event: 'desktop-upgrade-reconcile',
+ outcome: 'eperm',
+ lockPid: 65792,
+ }),
+ );
expect(ctx.ownsServer).toBe(false);
expect(ctx.port).toBe(59534);
expect(env.utilities.length).toBe(0);
@@ -2225,7 +2455,7 @@ describe('WindowManager', () => {
let spawned = false;
env.deps.spawnDetachedServer = async () => {
spawned = true;
- return { pid: 6666 };
+ return heldChild(env.deps, 6666);
};
env.deps.spawnLockPollDeadlineMs = 1000;
enableAttachProbe({ readServerLock: () => (spawned ? ownLock : null) });
@@ -2254,6 +2484,9 @@ describe('WindowManager', () => {
enableAttachProbe();
const wm = new WindowManager(env.deps);
const ctx = await wm.createProjectWindow({ projectPath: '/tmp/dragon' });
+ expect(env.logEntries.map((entry) => entry.payload)).toContainEqual(
+ expect.objectContaining({ event: 'desktop-dev-reclaim', outcome: 'eperm', lockPid: 65792 }),
+ );
expect(ctx.ownsServer).toBe(false);
expect(ctx.port).toBe(59534);
expect(env.utilities.length).toBe(0);
@@ -2322,7 +2555,10 @@ describe('WindowManager', () => {
});
test('foreign-host lock falls through (D44 case c)', async () => {
- enableAttachProbe({ hostname: () => 'different-host' });
+ enableAttachProbe({
+ hostname: () => 'different-host',
+ readServerLock: () => ({ ...liveLock, machineId: undefined }),
+ });
const wm = new WindowManager(env.deps);
const p = wm.createProjectWindow({ projectPath: '/tmp/dragon' });
@@ -2421,7 +2657,7 @@ describe('WindowManager', () => {
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- const spawn = vi.fn(() => Promise.resolve({ pid: 88001 }));
+ const spawn = vi.fn(() => Promise.resolve(heldChild(env.deps, 88001)));
env.deps.spawnDetachedServer = spawn;
const wm = new WindowManager(env.deps);
@@ -2454,15 +2690,15 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: 88001 });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, 88001));
const wm = new WindowManager(env.deps);
await wm.createProjectWindow({ projectPath: '/tmp/spawned-project' });
- const pids = (wm as unknown as { spawnedDetachedPids: Map<string, number> })
- .spawnedDetachedPids;
+ const pids = (wm as unknown as { spawnedDetachedServers: Map<string, SpawnedServer> })
+ .spawnedDetachedServers;
expect(pids.size).toBe(1);
- expect([...pids.values()]).toEqual([88001]);
+ expect([...pids.values()].map((server) => server.pid)).toEqual([88001]);
});
test('lock-poll timeout surfaces spawn-lock-timeout error', async () => {
@@ -2471,7 +2707,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: 88001 });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, 88001));
env.deps.spawnLockPollDeadlineMs = 1;
const wm = new WindowManager(env.deps);
@@ -2483,8 +2719,8 @@ describe('WindowManager', () => {
});
expect(env.windows.length).toBe(0);
- const pids = (wm as unknown as { spawnedDetachedPids: Map<string, number> })
- .spawnedDetachedPids;
+ const pids = (wm as unknown as { spawnedDetachedServers: Map<string, SpawnedServer> })
+ .spawnedDetachedServers;
expect(pids.size).toBe(0);
});
@@ -2497,7 +2733,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: 88001 });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, 88001));
const killed: number[] = [];
env.deps.killProbe = (pid: number) => {
killed.push(pid);
@@ -2522,7 +2758,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: 88001 });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, 88001));
const killed: number[] = [];
env.deps.killProbe = (pid: number) => {
killed.push(pid);
@@ -2543,7 +2779,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: 88001 });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, 88001));
env.deps.spawnLockPollDeadlineMs = 1;
env.deps.spawnLockProgressDeadlineMs = 25;
@@ -2569,7 +2805,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = (pid: number) => pid === winnerPid;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: 88001 });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, 88001));
env.deps.spawnLockPollDeadlineMs = 5;
env.deps.spawnLockProgressDeadlineMs = 30_000;
@@ -2586,7 +2822,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => false;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: 88001 });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, 88001));
env.deps.spawnLockPollDeadlineMs = 1;
env.deps.spawnLockProgressDeadlineMs = 60_000;
@@ -2600,6 +2836,33 @@ describe('WindowManager', () => {
expect(err?.message).toMatch(/exited before binding/);
});
+ test('a child whose exit was recorded is not granted the extension while its pid reads alive', async () => {
+ enableSyncTimers();
+ env.deps.readServerLock = () => null;
+ env.deps.isProcessAlive = () => true;
+ env.deps.hostname = () => 'my-host';
+ env.deps.probeWsUpgrade = () => Promise.resolve(true);
+ env.deps.spawnDetachedServer = () =>
+ Promise.resolve(heldChild(env.deps, 88001, () => ({ code: 1, signal: null })));
+ env.deps.spawnLockPollDeadlineMs = 1;
+ env.deps.spawnLockProgressDeadlineMs = 300;
+
+ const wm = new WindowManager(env.deps);
+ const err = await wm.createProjectWindow({ projectPath: '/tmp/recycled-child-pid' }).then(
+ () => null,
+ (e: unknown) => e as Error & { kind?: string },
+ );
+
+ expect(err?.kind).toBe('spawn-lock-timeout');
+ expect(err?.message).toMatch(/exited before binding/);
+ expect(
+ env.logEntries.filter(
+ (entry) =>
+ (entry.payload as { event?: string }).event === 'desktop-spawn-wait-extended',
+ ),
+ ).toEqual([]);
+ });
+
const spawnStderrRoots: string[] = [];
afterEach(() => {
while (spawnStderrRoots.length > 0) {
@@ -2624,7 +2887,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: 88001 });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, 88001));
env.deps.spawnLockPollDeadlineMs = 1;
env.deps.spawnLockProgressDeadlineMs = 5;
@@ -2647,7 +2910,7 @@ describe('WindowManager', () => {
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
env.deps.spawnDetachedServer = () =>
- Promise.resolve({ pid: 88001, readExit: () => ({ code: 1, signal: null }) });
+ Promise.resolve(heldChild(env.deps, 88001, () => ({ code: 1, signal: null })));
env.deps.spawnLockPollDeadlineMs = 1;
const wm = new WindowManager(env.deps);
@@ -2673,7 +2936,7 @@ describe('WindowManager', () => {
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
env.deps.spawnDetachedServer = () =>
- Promise.resolve({ pid: 88001, readExit: () => ({ code: 1, signal: null }) });
+ Promise.resolve(heldChild(env.deps, 88001, () => ({ code: 1, signal: null })));
env.deps.spawnLockPollDeadlineMs = 1;
const wm = new WindowManager(env.deps);
@@ -2698,7 +2961,7 @@ describe('WindowManager', () => {
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
env.deps.spawnDetachedServer = () =>
- Promise.resolve({ pid: 88001, readExit: () => ({ code: 1, signal: null }) });
+ Promise.resolve(heldChild(env.deps, 88001, () => ({ code: 1, signal: null })));
env.deps.spawnLockPollDeadlineMs = 1;
const wm = new WindowManager(env.deps);
@@ -2723,7 +2986,7 @@ describe('WindowManager', () => {
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
env.deps.spawnDetachedServer = () =>
- Promise.resolve({ pid: 88001, readExit: () => ({ code: 1, signal: null }) });
+ Promise.resolve(heldChild(env.deps, 88001, () => ({ code: 1, signal: null })));
env.deps.spawnLockPollDeadlineMs = 1;
const wm = new WindowManager(env.deps);
@@ -2743,7 +3006,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: 88001 });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, 88001));
env.deps.spawnLockPollDeadlineMs = 1;
const wm = new WindowManager(env.deps);
@@ -2762,7 +3025,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = undefined;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: 88001 });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, 88001));
env.deps.spawnLockPollDeadlineMs = 1;
const wm = new WindowManager(env.deps);
@@ -2786,7 +3049,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => false;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: 88001 });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, 88001));
env.deps.spawnLockPollDeadlineMs = 500;
const wm = new WindowManager(env.deps);
@@ -2825,7 +3088,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = (pid) => pid === WINNER_PID;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: 88001 });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, 88001));
env.deps.spawnLockPollDeadlineMs = 500;
const wm = new WindowManager(env.deps);
@@ -2845,7 +3108,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: OUR_CHILD_PID });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, OUR_CHILD_PID));
const killCalls: Array<{ pid: number; signal: NodeJS.Signals | number }> = [];
env.deps.killProbe = (pid, signal) => {
killCalls.push({ pid, signal });
@@ -2916,7 +3179,7 @@ describe('WindowManager', () => {
probed.push(url);
return Promise.resolve(false);
};
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: OUR_CHILD_PID });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, OUR_CHILD_PID));
const killCalls: Array<{ pid: number; signal: NodeJS.Signals | number }> = [];
env.deps.killProbe = (pid, signal) => {
killCalls.push({ pid, signal });
@@ -2960,7 +3223,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: OUR_CHILD_PID });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, OUR_CHILD_PID));
const killCalls: Array<{ pid: number; signal: NodeJS.Signals | number }> = [];
env.deps.killProbe = (pid, signal) => {
killCalls.push({ pid, signal });
@@ -3009,7 +3272,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: 22018 });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, 22018));
env.deps.spawnLockPollDeadlineMs = 50;
env.deps.spawnLockProgressDeadlineMs = 50;
@@ -3038,7 +3301,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: OUR_CHILD_PID });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, OUR_CHILD_PID));
const killCalls: Array<{ pid: number; signal: NodeJS.Signals | number }> = [];
env.deps.killProbe = (pid, signal) => {
killCalls.push({ pid, signal });
@@ -3063,6 +3326,36 @@ describe('WindowManager', () => {
expect(killCalls).toContainEqual({ pid: OUR_CHILD_PID, signal: 'SIGTERM' });
});
+ test('a refused spawn reports its held child exited while that pid reads alive', async () => {
+ enableSyncTimers();
+ const OUR_CHILD_PID = 22019;
+ const ownLock = {
+ ...spawnedLock,
+ pid: OUR_CHILD_PID,
+ port: '42117',
+ } as unknown as ServerLockMetadataLike;
+ env.deps.readServerLock = () => ownLock;
+ env.deps.isProcessAlive = () => true;
+ env.deps.hostname = () => 'my-host';
+ env.deps.probeWsUpgrade = () => Promise.resolve(true);
+ env.deps.spawnDetachedServer = () =>
+ Promise.resolve(heldChild(env.deps, OUR_CHILD_PID, () => ({ code: 1, signal: null })));
+ env.deps.spawnLockPollDeadlineMs = 50;
+ env.deps.spawnLockProgressDeadlineMs = 50;
+
+ const wm = new WindowManager(env.deps);
+ const failure = await wm.createProjectWindow({ projectPath: '/tmp/spawned-project' }).then(
+ () => null,
+ (err: unknown) => err,
+ );
+
+ expect(failure).toMatchObject({
+ kind: 'stale-lock-holder',
+ childExited: true,
+ exitCode: 1,
+ });
+ });
+
test('a healthy cross-pid lock-race winner is still attached (probe passes)', async () => {
enableSyncTimers();
const WINNER_PID = 870;
@@ -3080,7 +3373,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = (pid) => pid === WINNER_PID;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- const spawn = vi.fn(() => Promise.resolve({ pid: OUR_CHILD_PID }));
+ const spawn = vi.fn(() => Promise.resolve(heldChild(env.deps, OUR_CHILD_PID)));
env.deps.spawnDetachedServer = spawn;
env.deps.spawnLockPollDeadlineMs = 500;
@@ -3106,7 +3399,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: OUR_CHILD_PID });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, OUR_CHILD_PID));
env.deps.spawnLockPollDeadlineMs = 50;
env.deps.spawnLockProgressDeadlineMs = 50;
@@ -3134,7 +3427,7 @@ describe('WindowManager', () => {
probes++;
return Promise.resolve(probes >= 3);
};
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: OUR_CHILD_PID });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, OUR_CHILD_PID));
env.deps.spawnLockPollDeadlineMs = 50;
env.deps.spawnLockProgressDeadlineMs = 50;
@@ -3161,7 +3454,7 @@ describe('WindowManager', () => {
probed.push(url);
return Promise.resolve(true);
};
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: OUR_CHILD_PID });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, OUR_CHILD_PID));
env.deps.spawnLockPollDeadlineMs = 500;
const wm = new WindowManager(env.deps);
@@ -3178,10 +3471,7 @@ describe('WindowManager', () => {
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
env.deps.spawnDetachedServer = () =>
- Promise.resolve({
- pid: 88001,
- readExit: () => ({ code: 1, signal: null }),
- });
+ Promise.resolve(heldChild(env.deps, 88001, () => ({ code: 1, signal: null })));
env.deps.spawnLockPollDeadlineMs = 500;
const wm = new WindowManager(env.deps);
@@ -3197,10 +3487,7 @@ describe('WindowManager', () => {
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
env.deps.spawnDetachedServer = () =>
- Promise.resolve({
- pid: 88001,
- readExit: () => ({ code: null, signal: 'SIGKILL' }),
- });
+ Promise.resolve(heldChild(env.deps, 88001, () => ({ code: null, signal: 'SIGKILL' })));
env.deps.spawnLockPollDeadlineMs = 500;
const wm = new WindowManager(env.deps);
@@ -3218,7 +3505,8 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: 88001, readExit: () => null });
+ env.deps.spawnDetachedServer = () =>
+ Promise.resolve(heldChild(env.deps, 88001, () => null));
env.deps.spawnLockPollDeadlineMs = 1;
const wm = new WindowManager(env.deps);
@@ -3237,7 +3525,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: 88001 });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, 88001));
const wm = new WindowManager(env.deps);
const ctx = await wm.createProjectWindow({ projectPath: '/tmp/spawned-project' });
@@ -3246,8 +3534,8 @@ describe('WindowManager', () => {
expect(ctx.ownsServer).toBe(false);
env.windows[0]?.fireClose();
- const pids = (wm as unknown as { spawnedDetachedPids: Map<string, number> })
- .spawnedDetachedPids;
+ const pids = (wm as unknown as { spawnedDetachedServers: Map<string, SpawnedServer> })
+ .spawnedDetachedServers;
expect(pids.size).toBe(1);
expect(env.utilities.length).toBe(0);
});
@@ -3268,7 +3556,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- const spawn = vi.fn(() => Promise.resolve({ pid: 91001 }));
+ const spawn = vi.fn(() => Promise.resolve(heldChild(env.deps, 91001)));
env.deps.spawnDetachedServer = spawn;
const wm = new WindowManager(env.deps);
@@ -3285,7 +3573,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- const spawn = vi.fn(() => Promise.resolve({ pid: 99 }));
+ const spawn = vi.fn(() => Promise.resolve(heldChild(env.deps, 99)));
env.deps.spawnDetachedServer = spawn;
const wm = new WindowManager(env.deps);
@@ -3308,7 +3596,17 @@ describe('WindowManager', () => {
for (const dir of owned) rmSync(dir, { recursive: true, force: true });
for (const dir of owned) expect(existsSync(dir)).toBe(false);
});
- function seedRawLock(pid: number, overrides?: { port?: number }): string {
+ const recordedStartMatchingTheStubbedReading = {
+ startedAtMs: Date.parse('2026-01-01T00:00:00Z'),
+ };
+ function seedRawLock(
+ pid: number,
+ overrides?: { port?: number; machineId?: string; processStart?: object | null },
+ ): string {
+ const processStart =
+ overrides?.processStart === undefined
+ ? recordedStartMatchingTheStubbedReading
+ : overrides.processStart;
const projectPath = temporaryProject();
const lockDir = join(projectPath, '.ok', 'local');
mkdirSync(lockDir, { recursive: true });
@@ -3317,11 +3615,12 @@ describe('WindowManager', () => {
JSON.stringify({
pid,
hostname: 'some-old-hostname',
- machineId: 'not-this-machine',
+ machineId: overrides?.machineId ?? 'not-this-machine',
port: overrides?.port ?? 61000,
startedAt: '2026-07-07T00:00:00.000Z',
worktreeRoot: projectPath,
kind: 'interactive',
+ ...(processStart !== null && { processStart }),
}),
'utf-8',
);
@@ -3345,6 +3644,161 @@ describe('WindowManager', () => {
expect(killCalls).toContainEqual({ pid: 64321, signal: 'SIGTERM' });
});
+ test('does not signal an earlier-started holder of a lock from another machine id that records no process start', async () => {
+ const signals: Array<{ pid: number; signal: NodeJS.Signals | number }> = [];
+ env.deps.killProbe = (pid, signal) => {
+ signals.push({ pid, signal });
+ };
+ const projectPath = seedRawLock(64341, { processStart: null });
+
+ const outcome = await new WindowManager(env.deps).forceStopConflictingServer(projectPath);
+
+ expect(outcome).toEqual({ ok: false, reason: 'unverified', sigtermSent: false });
+ expect(signals.filter((record) => record.signal !== 0)).toEqual([]);
+ expect(env.logEntries.map((entry) => entry.payload)).toContainEqual(
+ expect.objectContaining({
+ event: 'desktop-lock-holder-unverified',
+ lockPid: 64341,
+ cause: 'foreign-host',
+ }),
+ );
+ });
+
+ test('does not signal the holder of a lock that records no owner', async () => {
+ const signals: Array<{ pid: number; signal: NodeJS.Signals | number }> = [];
+ env.deps.killProbe = (pid, signal) => {
+ signals.push({ pid, signal });
+ };
+ const projectPath = temporaryProject();
+ const lockDir = join(projectPath, '.ok', 'local');
+ mkdirSync(lockDir, { recursive: true });
+ writeFileSync(
+ join(lockDir, 'server.lock'),
+ JSON.stringify({
+ pid: 64344,
+ port: 61000,
+ startedAt: '2026-07-07T00:00:00.000Z',
+ worktreeRoot: projectPath,
+ }),
+ 'utf-8',
+ );
+
+ const outcome = await new WindowManager(env.deps).forceStopConflictingServer(projectPath);
+
+ expect(outcome).toEqual({ ok: false, reason: 'unverified', sigtermSent: false });
+ expect(signals.filter((record) => record.signal !== 0)).toEqual([]);
+ expect(env.logEntries.map((entry) => entry.payload)).toContainEqual(
+ expect.objectContaining({
+ event: 'desktop-lock-holder-unverified',
+ lockPid: 64344,
+ cause: 'unverified-owner',
+ }),
+ );
+ });
+
+ test('reports a holder that exited before any signal as already gone, not replaced', async () => {
+ const signals: Array<{ pid: number; signal: NodeJS.Signals | number }> = [];
+ env.deps.killProbe = (pid, signal) => {
+ signals.push({ pid, signal });
+ };
+ env.deps.isProcessAlive = () => false;
+ const projectPath = seedRawLock(64342, { machineId: getMachineId() });
+
+ const outcome = await new WindowManager(env.deps).forceStopConflictingServer(projectPath);
+
+ expect(outcome).toEqual({ ok: true });
+ expect(signals.filter((record) => record.signal !== 0)).toEqual([]);
+ expect(env.logEntries.map((entry) => entry.payload)).toContainEqual(
+ expect.objectContaining({
+ event: 'desktop-force-stop-conflicting-server',
+ lockPid: 64342,
+ outcome: 'already-gone',
+ }),
+ );
+ });
+
+ test('reports SIGTERM sent and SIGKILL withheld when the holder cannot be verified after the grace period', async () => {
+ const signals: Array<{ pid: number; signal: NodeJS.Signals | number }> = [];
+ env.deps.killProbe = (pid, signal) => {
+ signals.push({ pid, signal });
+ };
+ env.deps.sigtermGraceMs = 0;
+ const readings = [
+ { startedAtMs: Date.parse('2026-01-01T00:00:00Z'), clock: 'creation' as const },
+ null,
+ ];
+ env.deps.readProcessStart = () => (readings.length > 0 ? (readings.shift() ?? null) : null);
+ const projectPath = seedRawLock(64343, { machineId: getMachineId() });
+
+ const outcome = await new WindowManager(env.deps).forceStopConflictingServer(projectPath);
+
+ expect(outcome).toEqual({ ok: false, reason: 'unverified', sigtermSent: true });
+ expect(signals.filter((record) => record.signal !== 0)).toEqual([
+ { pid: 64343, signal: 'SIGTERM' },
+ ]);
+ expect(env.logEntries.map((entry) => entry.payload)).toContainEqual(
+ expect.objectContaining({
+ event: 'desktop-lock-holder-unverified',
+ lockPid: 64343,
+ sigtermSent: true,
+ cause: 'process-start-unreadable',
+ }),
+ );
+ expect(env.logEntries.map((entry) => entry.payload)).toContainEqual(
+ expect.objectContaining({
+ event: 'desktop-force-stop-conflicting-server',
+ lockPid: 64343,
+ outcome: 'sigterm-sent-unverified',
+ }),
+ );
+ });
+
+ test('checks the holder again before SIGKILL and spares a process that replaced it during the grace period', async () => {
+ const signals: Array<{ pid: number; signal: NodeJS.Signals | number }> = [];
+ env.deps.killProbe = (pid, signal) => {
+ signals.push({ pid, signal });
+ };
+ env.deps.sigtermGraceMs = 0;
+ const readings = [
+ { startedAtMs: Date.parse('2026-01-01T00:00:00Z'), clock: 'creation' as const },
+ { startedAtMs: Date.parse('2026-07-08T00:00:00Z'), clock: 'creation' as const },
+ ];
+ env.deps.readProcessStart = () => readings.shift() ?? null;
+ const projectPath = seedRawLock(64331, { machineId: getMachineId() });
+
+ const outcome = await new WindowManager(env.deps).forceStopConflictingServer(projectPath);
+
+ expect(outcome).toEqual({ ok: true });
+ expect(signals.filter((record) => record.signal !== 0)).toEqual([
+ { pid: 64331, signal: 'SIGTERM' },
+ ]);
+ expect(env.logEntries.map((entry) => entry.payload)).toContainEqual(
+ expect.objectContaining({ event: 'desktop-lock-holder-replaced', lockPid: 64331 }),
+ );
+ });
+
+ test('signals nothing and reports a failure when the holder cannot be identified', async () => {
+ const signals: Array<{ pid: number; signal: NodeJS.Signals | number }> = [];
+ env.deps.killProbe = (pid, signal) => {
+ signals.push({ pid, signal });
+ };
+ env.deps.sigtermGraceMs = 0;
+ env.deps.readProcessStart = () => null;
+ const projectPath = seedRawLock(64332, { machineId: getMachineId() });
+
+ const outcome = await new WindowManager(env.deps).forceStopConflictingServer(projectPath);
+
+ expect(outcome).toEqual({ ok: false, reason: 'unverified', sigtermSent: false });
+ expect(signals.filter((record) => record.signal !== 0)).toEqual([]);
+ expect(env.logEntries.map((entry) => entry.payload)).toContainEqual(
+ expect.objectContaining({
+ event: 'desktop-lock-holder-unverified',
+ lockPid: 64332,
+ cause: 'process-start-unreadable',
+ }),
+ );
+ });
+
test('no lock file → ok without signalling anything', async () => {
const killCalls: number[] = [];
env.deps.killProbe = (pid) => {
@@ -3646,7 +4100,7 @@ describe('WindowManager', () => {
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
let nextSpawnPid = 91001;
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: nextSpawnPid++ });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, nextSpawnPid++));
const killCalls: Array<{ pid: number; signal: NodeJS.Signals | 0 }> = [];
env.deps.killProbe = (pid: number, signal: NodeJS.Signals | 0) => {
@@ -3666,8 +4120,8 @@ describe('WindowManager', () => {
readCounts = new Map<string, number>();
await wm.createProjectWindow({ projectPath: '/tmp/proj-b' });
- const pidsBefore = (wm as unknown as { spawnedDetachedPids: Map<string, number> })
- .spawnedDetachedPids;
+ const pidsBefore = (wm as unknown as { spawnedDetachedServers: Map<string, SpawnedServer> })
+ .spawnedDetachedServers;
expect(pidsBefore.size).toBe(2);
await wm.stopAllOwnedServers();
@@ -3693,7 +4147,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: 91001 });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, 91001));
env.deps.sigtermGraceMs = 5;
const killCalls: Array<{ pid: number; signal: NodeJS.Signals | 0 }> = [];
@@ -3734,7 +4188,7 @@ describe('WindowManager', () => {
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: 91001 });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, 91001));
env.deps.killProbe = () => {
const err = new Error('No such process') as NodeJS.ErrnoException;
@@ -3747,6 +4201,75 @@ describe('WindowManager', () => {
await expect(wm.stopAllOwnedServers()).resolves.toBeUndefined();
});
+ function spawnServerThatLaterExits() {
+ enableSyncTimers();
+ let readCount = 0;
+ env.deps.readServerLock = () => {
+ readCount++;
+ return readCount === 1 ? null : { ...spawnedLock, pid: 91001 };
+ };
+ env.deps.isProcessAlive = () => true;
+ env.deps.hostname = () => 'my-host';
+ env.deps.probeWsUpgrade = () => Promise.resolve(true);
+ let exit: { code: number | null; signal: string | null } | null = null;
+ const handleSignals: NodeJS.Signals[] = [];
+ env.deps.spawnDetachedServer = () =>
+ Promise.resolve({
+ pid: 91001,
+ readExit: () => exit,
+ signal: (signal: NodeJS.Signals) => {
+ if (exit === null) handleSignals.push(signal);
+ },
+ });
+ const killCalls: Array<{ pid: number; signal: NodeJS.Signals | 0 }> = [];
+ env.deps.killProbe = (pid: number, signal: NodeJS.Signals | 0) => {
+ killCalls.push({ pid, signal });
+ };
+ return {
+ exitServer: () => {
+ exit = { code: 1, signal: null };
+ },
+ handleSignals,
+ killCalls,
+ };
+ }
+
+ test('a spawned server that exited is not signalled, though its pid now belongs to another process', async () => {
+ const { exitServer, handleSignals, killCalls } = spawnServerThatLaterExits();
+ const wm = new WindowManager(env.deps);
+ await wm.createProjectWindow({ projectPath: '/tmp/crashed-project' });
+ exitServer();
+
+ await wm.stopAllOwnedServers();
+
+ expect(killCalls.filter((call) => call.signal !== 0)).toEqual([]);
+ expect(handleSignals).toEqual([]);
+ });
+
+ test('signalStopAllOwnedServers leaves a spawned server that exited, and its reused pid, alone', async () => {
+ const { exitServer, handleSignals, killCalls } = spawnServerThatLaterExits();
+ const wm = new WindowManager(env.deps);
+ await wm.createProjectWindow({ projectPath: '/tmp/crashed-project' });
+ exitServer();
+
+ wm.signalStopAllOwnedServers();
+
+ expect(killCalls.filter((call) => call.signal !== 0)).toEqual([]);
+ expect(handleSignals).toEqual([]);
+ });
+
+ test('a spawned server still running is stopped through the child it was spawned as', async () => {
+ const { handleSignals, killCalls } = spawnServerThatLaterExits();
+ env.deps.sigtermGraceMs = 5;
+ const wm = new WindowManager(env.deps);
+ await wm.createProjectWindow({ projectPath: '/tmp/wedged-project' });
+
+ await wm.stopAllOwnedServers();
+
+ expect(handleSignals).toEqual(['SIGTERM', 'SIGKILL']);
+ expect(killCalls.filter((call) => call.signal !== 0)).toEqual([]);
+ });
+
test('utility-fork (dev path, ownsServer=true) is SIGKILLed by stopAllOwnedServers', async () => {
delete env.deps.spawnDetachedServer;
const wm = new WindowManager(env.deps);
@@ -5425,7 +5948,7 @@ describe('WindowManager — local-op CLI invocation threading', () => {
env.deps.isProcessAlive = () => true;
env.deps.hostname = () => 'my-host';
env.deps.probeWsUpgrade = () => Promise.resolve(true);
- env.deps.spawnDetachedServer = () => Promise.resolve({ pid: 88001 });
+ env.deps.spawnDetachedServer = () => Promise.resolve(heldChild(env.deps, 88001));
}
test('a packaged win32 host opens a project window on the detached-spawn path it actually takes', async () => {
diff --git a/packages/desktop/tests/support/windows-process-start-reader.test-helper.ts b/packages/desktop/tests/support/windows-process-start-reader.test-helper.ts
new file mode 100644
index 000000000..78212c17a
--- /dev/null
+++ b/packages/desktop/tests/support/windows-process-start-reader.test-helper.ts
@@ -0,0 +1,8 @@
+export {};
+
+if (process.platform === 'win32' && process.env.OK_TEST_WINDOWS_PROCESS_START_READER === '1') {
+ const { registerWindowsProcessStartReader } = await import(
+ '../../../cli/src/commands/holder-process-start.ts'
+ );
+ registerWindowsProcessStartReader();
+}
diff --git a/packages/desktop/tests/utility/server-entry.test.ts b/packages/desktop/tests/utility/server-entry.test.ts
index 6c5122a51..c101d1709 100644
--- a/packages/desktop/tests/utility/server-entry.test.ts
+++ b/packages/desktop/tests/utility/server-entry.test.ts
@@ -1,10 +1,16 @@
import { execFileSync } from 'node:child_process';
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
+import { createRequire } from 'node:module';
import { tmpdir } from 'node:os';
import { resolve } from 'node:path';
import { setTimeout as wait } from 'node:timers/promises';
import { resolveServerRuntimeConfig } from '@inkeep/open-knowledge-core';
-import { ConfigSchema } from '@inkeep/open-knowledge-server';
+import {
+ ConfigSchema,
+ type ProcessStart,
+ readProcessStart,
+ registerNativeProcessStartReader,
+} from '@inkeep/open-knowledge-server';
import { beforeEach, describe, expect, type Mock, test, vi } from 'vitest';
import { configureTestGitRepository } from '../../../../test-support/configure-git-fixture.test-helper.ts';
import { UTILITY_INIT_PHASES } from '../../src/shared/boot-narration.ts';
@@ -668,6 +674,68 @@ describe('handleInit boot prelude (FR-16/17/18/19/22/24)', () => {
expect(ipcArgs?.projectDir).toBe('/projects/myrepo');
});
+ test('lets the server read Windows process starts before it prepares the boot environment', async () => {
+ const builtCliEntry = createRequire(import.meta.url).resolve('@inkeep/open-knowledge');
+ const nativeConfig: { readProcessStart: (pid: number) => number | null } =
+ createRequire(builtCliEntry)('./native/index.js');
+ const nativeRead = vi
+ .spyOn(nativeConfig, 'readProcessStart')
+ .mockReturnValue(1_789_000_000_456);
+ registerNativeProcessStartReader(() => null)();
+ await import('@inkeep/open-knowledge');
+ const readingBeforeInit = readProcessStart(4242, { platform: 'win32' });
+ let readingWhilePreparing: ProcessStart | null = null;
+ const prepare = vi.fn(() => {
+ readingWhilePreparing = readProcessStart(4242, { platform: 'win32' });
+ return Promise.resolve(makeFakePrepared());
+ });
+ const bootServer = vi.fn(() =>
+ Promise.resolve({
+ port: 4242,
+ destroy: vi.fn(() => Promise.resolve()),
+ degraded: [] as readonly string[],
+ }),
+ );
+
+ try {
+ const handle = setupUtility({
+ parentPort: env.parentPort,
+ importServer: () =>
+ Promise.resolve({
+ bootServer,
+ } as unknown as typeof import('@inkeep/open-knowledge-server')),
+ exit: env.exit,
+ parentPid: 99999,
+ killProbe: env.killProbe,
+ onSignal: (sig, h) => env.signalHandlers.set(sig, h),
+ setInterval: (cb, ms) => {
+ env.intervals.push({ cb, ms });
+ return { unref: vi.fn(() => {}), clear: env.intervalCancel };
+ },
+ prepareBootEnvironment: prepare,
+ });
+
+ env.parentPort.fire({
+ type: 'init',
+ opts: {
+ contentDir: '/projects/myrepo',
+ projectDir: '/projects/myrepo',
+ port: 0,
+ host: 'localhost',
+ },
+ });
+ await handle.readyPromise;
+
+ expect(prepare).toHaveBeenCalledTimes(1);
+ expect(readingBeforeInit).toBeNull();
+ expect(readingWhilePreparing).toEqual({ startedAtMs: 1_789_000_000_456, clock: 'creation' });
+ expect(nativeRead).toHaveBeenCalledWith(4242);
+ } finally {
+ nativeRead.mockRestore();
+ registerNativeProcessStartReader(() => null)();
+ }
+ });
+
test('OK_DEBUG_DESKTOP_BOOT_TRACE=1 logs the resolved config trace', async () => {
const fakeBooted = {
port: 4242,
diff --git a/packages/desktop/vitest.config.ts b/packages/desktop/vitest.config.ts
index 5c4ca9176..538db703e 100644
--- a/packages/desktop/vitest.config.ts
+++ b/packages/desktop/vitest.config.ts
@@ -26,6 +26,12 @@ export default defineConfig({
...okVitestBase,
test: {
...okVitestBase.test,
+ setupFiles: [
+ ...okVitestBase.test.setupFiles,
+ fileURLToPath(
+ new URL('./tests/support/windows-process-start-reader.test-helper.ts', import.meta.url),
+ ),
+ ],
include: ['**/*.test.ts?(x)', '**/*.test.mjs'],
exclude: [
...okVitestBase.test.exclude,
diff --git a/packages/server/src/embeddings/secrets-store.ts b/packages/server/src/embeddings/secrets-store.ts
index d6bd335b8..e1f725c99 100644
--- a/packages/server/src/embeddings/secrets-store.ts
+++ b/packages/server/src/embeddings/secrets-store.ts
@@ -23,6 +23,7 @@ import {
tracedUnlinkSync,
tracedWriteFileSync,
} from '../fs-traced.ts';
+import { sleepSyncMs } from '../sleep-sync.ts';
import { normalizeProviderId } from './embedder.ts';
const DEFAULT_ENDPOINT_ID = normalizeProviderId(DEFAULT_EMBEDDINGS_BASE_URL);
@@ -85,10 +86,6 @@ function keyHint(key: string | null): string | null {
const LOCK_TIMEOUT_MS = 2000;
const LOCK_STALE_MS = 10_000;
-function sleepSyncMs(ms: number): void {
- Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, ms);
-}
-
export class FileEmbeddingsBackend implements EmbeddingsSecretStore {
readonly backend = 'file' as const;
private readonly secretsFile: string;
diff --git a/packages/server/src/index.ts b/packages/server/src/index.ts
index 095b2597b..1a5baea4d 100644
--- a/packages/server/src/index.ts
+++ b/packages/server/src/index.ts
@@ -381,7 +381,7 @@ export {
type PinoLoggerConfig,
} from './logger.ts';
export { isAllowedWorkspaceHostHeader, isLoopbackAddress } from './loopback.ts';
-export { getMachineId } from './machine-id.ts';
+export { getMachineId, machineIdFilePath } from './machine-id.ts';
export {
type RenameRewriteResult,
rewriteMarkdownLinksForDocumentRename,
@@ -437,18 +437,42 @@ export {
} from './persistence.ts';
export { loadPrincipal } from './principal.ts';
export { isProcessAlive, isValidLockPid } from './process-alive.ts';
+export {
+ assessLockHolder,
+ classifyLockHolder,
+ isLockFromThisMachine,
+ type LockHolderAssessment,
+ type LockHolderIdentity,
+ type LockHolderVerdict,
+ type LockOwnerFields,
+ type ProcessStart,
+ type ProcessStartCause,
+ type ProcessStartReader,
+ type ProcessStartToken,
+ parseProcessStartToken,
+ processStartToken,
+ readProcessStart,
+ recordsLockOwner,
+ registerNativeProcessStartReader,
+ type UnverifiedLockHolderCause,
+} from './process-identity.ts';
export {
acquireProcessLock,
+ LOCK_FILE_ACCESS_REMEDY,
type LockName,
+ type LockRemoval,
lockBaseUrl,
lockFilePath,
ProcessLockCollisionError,
+ ProcessLockFileError,
type ProcessLockHandle,
type ProcessLockMetadata,
type ReadProcessLockResult,
readProcessLock,
readProcessLockDetailed,
releaseProcessLock,
+ removeLockIfUnchanged,
+ toProcessLockMetadata,
updateProcessLockPort,
} from './process-lock.ts';
export {
diff --git a/packages/server/src/lock-process-start-unrecorded.test.ts b/packages/server/src/lock-process-start-unrecorded.test.ts
new file mode 100644
index 000000000..03d59eaab
--- /dev/null
+++ b/packages/server/src/lock-process-start-unrecorded.test.ts
@@ -0,0 +1,114 @@
+import { mkdirSync, readFileSync } from 'node:fs';
+import { resolve } from 'node:path';
+import { LOCAL_DIR } from '@inkeep/open-knowledge-core';
+import { afterEach, describe, expect, type MockInstance, test, vi } from 'vitest';
+import { createTempDirFactory } from '../../../test-support/temp-dir.test-helper.ts';
+import { getLogger } from './logger.ts';
+import type { ProcessStart } from './process-identity.ts';
+import { acquireProcessLock, type ProcessLockMetadata } from './process-lock.ts';
+import { acquireLock, type LockMetadata, releaseLock } from './shadow-lock.ts';
+
+const ownStart = vi.hoisted(() => ({ value: null as ProcessStart | null }));
+
+vi.mock('./process-identity.ts', async (importOriginal) => {
+ const actual = await importOriginal<typeof import('./process-identity.ts')>();
+ return {
+ ...actual,
+ readProcessStart: (pid: number, options?: Parameters<typeof actual.readProcessStart>[1]) =>
+ pid === process.pid ? ownStart.value : actual.readProcessStart(pid, options),
+ };
+});
+
+const UNRECORDED_START_EVENT = 'process-start-unrecorded';
+const tempDir = createTempDirFactory(afterEach);
+
+const unrecordedStarts: ReadonlyArray<{ reason: string; start: ProcessStart | null }> = [
+ { reason: 'start-unreadable', start: null },
+ {
+ reason: 'boot-id-unreadable',
+ start: { startedAtMs: 1_700_000_000_000, clock: 'boot-derived', startTicks: 4_200 },
+ },
+];
+
+const READABLE_START: ProcessStart = { startedAtMs: 1_700_000_000_000, clock: 'creation' };
+
+afterEach(() => {
+ ownStart.value = null;
+ vi.restoreAllMocks();
+});
+
+function unrecordedStartWarnings(warn: MockInstance<(data: unknown, message: string) => void>) {
+ return warn.mock.calls.filter(
+ ([fields]) => (fields as { event?: unknown } | undefined)?.event === UNRECORDED_START_EVENT,
+ );
+}
+
+function acquireServerLock(): ProcessLockMetadata {
+ const lockDir = resolve(tempDir('ok-unrecorded-start-'), '.ok', LOCAL_DIR);
+ const handle = acquireProcessLock({
+ lockName: 'server',
+ lockDir,
+ metadata: { port: 0, worktreeRoot: lockDir },
+ });
+ try {
+ return JSON.parse(readFileSync(handle.lockPath, 'utf-8')) as ProcessLockMetadata;
+ } finally {
+ handle.release();
+ }
+}
+
+function acquireShadowLock(): LockMetadata {
+ const shadowDir = resolve(tempDir('ok-unrecorded-start-shadow-'), 'shadow');
+ mkdirSync(shadowDir, { recursive: true });
+ const lockPath = acquireLock(shadowDir, shadowDir);
+ try {
+ return JSON.parse(readFileSync(lockPath, 'utf-8')) as LockMetadata;
+ } finally {
+ releaseLock(shadowDir);
+ }
+}
+
+const lockWriters: ReadonlyArray<{
+ lock: string;
+ logger: string;
+ acquire: () => { pid: number; processStart?: unknown };
+}> = [
+ { lock: 'server', logger: 'process-lock', acquire: acquireServerLock },
+ { lock: 'shadow', logger: 'shadow-lock', acquire: acquireShadowLock },
+];
+
+describe.each(lockWriters)(
+ 'a $lock lock written by a process that cannot record its own start',
+ ({ logger, acquire }) => {
+ test.each(unrecordedStarts)(
+ 'logs one warning naming the cause ($reason)',
+ ({ reason, start }) => {
+ ownStart.value = start;
+ const warn = vi.spyOn(getLogger(logger), 'warn');
+
+ const lock = acquire();
+
+ expect(lock.pid).toBe(process.pid);
+ expect(lock).not.toHaveProperty('processStart');
+ const warnings = unrecordedStartWarnings(warn);
+ expect(warnings).toHaveLength(1);
+ expect(warnings[0]?.[0]).toEqual({
+ event: UNRECORDED_START_EVENT,
+ platform: process.platform,
+ reason,
+ });
+ expect(warnings[0]?.[1]).toContain('startedAt');
+ },
+ );
+
+ test('records the start and logs no such warning when the start is readable', () => {
+ ownStart.value = READABLE_START;
+ const warn = vi.spyOn(getLogger(logger), 'warn');
+
+ const lock = acquire();
+
+ expect(lock.processStart).toEqual({ startedAtMs: READABLE_START.startedAtMs });
+ expect(unrecordedStartWarnings(warn)).toEqual([]);
+ });
+ },
+);
diff --git a/packages/server/src/process-identity.test.ts b/packages/server/src/process-identity.test.ts
new file mode 100644
index 000000000..0aa992856
--- /dev/null
+++ b/packages/server/src/process-identity.test.ts
@@ -0,0 +1,1033 @@
+import { type ExecFileSyncOptionsWithStringEncoding, execFileSync } from 'node:child_process';
+import { hostname } from 'node:os';
+import { afterEach, describe, expect, test, vi } from 'vitest';
+import {
+ processStartTokenFromOs,
+ readProcessStartFromOs,
+} from '../../../test-support/os-process-start.test-helper.ts';
+import {
+ lockWrittenBeforeProcessStarted,
+ lockWrittenByRunningProcess,
+ type OwnedProcess,
+ startOwnedProcess,
+ stopOwnedProcess,
+} from '../../../test-support/owned-process.test-helper.ts';
+import { getLogger } from './logger.ts';
+import { getMachineId } from './machine-id.ts';
+import {
+ assessLockHolder,
+ classifyLockHolder,
+ createRecentProcessStartReader,
+ isLockFromThisMachine,
+ type LockHolderIdentity,
+ type LockHolderVerdict,
+ type ProcessStart,
+ parseProcessStartToken,
+ processStartToken,
+ readProcessStart,
+ readPsProcessStart,
+ registerNativeProcessStartReader,
+ type UnverifiedLockHolderCause,
+} from './process-identity.ts';
+
+const WRITTEN_AT = '2026-10-01T12:00:00.000Z';
+const WRITTEN_AT_MS = Date.parse(WRITTEN_AT);
+const THIRTY_SIX_HOURS_MS = 36 * 60 * 60 * 1000;
+const HOLDER_PID = 4242;
+const THIRTY_SIX_HOURS_BEFORE_WRITTEN_AT = new Date(
+ WRITTEN_AT_MS - THIRTY_SIX_HOURS_MS,
+).toISOString();
+
+function lockWrittenAt(extra: Partial<LockHolderIdentity> = {}): LockHolderIdentity {
+ return { pid: HOLDER_PID, startedAt: WRITTEN_AT, ...extra };
+}
+
+function creationStart(offsetFromWrittenMs: number): ProcessStart {
+ return { startedAtMs: WRITTEN_AT_MS + offsetFromWrittenMs, clock: 'creation' };
+}
+
+function bootDerivedStart(
+ offsetFromWrittenMs: number,
+ pair?: { bootId: string; startTicks: number },
+): ProcessStart {
+ return { startedAtMs: WRITTEN_AT_MS + offsetFromWrittenMs, clock: 'boot-derived', ...pair };
+}
+
+interface ComparatorRow {
+ name: string;
+ lock: LockHolderIdentity;
+ live: ProcessStart | null;
+ expected: Exclude<LockHolderVerdict, 'gone'>;
+ cause?: UnverifiedLockHolderCause;
+ refuseHolderStartedAfterLock?: boolean;
+}
+
+const BOOT_PAIR_TOKEN = { bootId: 'boot-a', startTicks: 5_000 };
+
+const comparatorRows: ComparatorRow[] = [
+ {
+ name: 'no reading of the holder is unverified',
+ lock: lockWrittenAt(),
+ live: null,
+ expected: 'unverified',
+ cause: 'process-start-unreadable',
+ },
+ {
+ name: 'a boot pair from another boot is replaced',
+ lock: lockWrittenAt({ processStart: BOOT_PAIR_TOKEN }),
+ live: bootDerivedStart(0, { bootId: 'boot-b', startTicks: 5_000 }),
+ expected: 'replaced',
+ },
+ {
+ name: 'equal ticks on the same boot are the writer even when startedAt is 36 h old',
+ lock: lockWrittenAt({ processStart: BOOT_PAIR_TOKEN }),
+ live: bootDerivedStart(THIRTY_SIX_HOURS_MS, { bootId: 'boot-a', startTicks: 5_000 }),
+ expected: 'writer',
+ },
+ {
+ name: 'later ticks on the same boot are replaced even when startedAt looks current',
+ lock: lockWrittenAt({ processStart: BOOT_PAIR_TOKEN }),
+ live: bootDerivedStart(0, { bootId: 'boot-a', startTicks: 5_001 }),
+ expected: 'replaced',
+ },
+ {
+ name: 'earlier ticks on the same boot are unverified',
+ lock: lockWrittenAt({ processStart: BOOT_PAIR_TOKEN }),
+ live: bootDerivedStart(0, { bootId: 'boot-a', startTicks: 4_999 }),
+ expected: 'unverified',
+ cause: 'process-start-mismatch',
+ },
+ {
+ name: 'a creation start 1000 ms after the token is the writer even when startedAt is 36 h old',
+ lock: lockWrittenAt({
+ startedAt: THIRTY_SIX_HOURS_BEFORE_WRITTEN_AT,
+ processStart: { startedAtMs: WRITTEN_AT_MS },
+ }),
+ live: creationStart(1_000),
+ expected: 'writer',
+ },
+ {
+ name: 'a creation start 1000 ms before the token is the writer even when startedAt is 36 h old',
+ lock: lockWrittenAt({
+ startedAt: THIRTY_SIX_HOURS_BEFORE_WRITTEN_AT,
+ processStart: { startedAtMs: WRITTEN_AT_MS },
+ }),
+ live: creationStart(-1_000),
+ expected: 'writer',
+ },
+ {
+ name: 'a creation start 1001 ms after the token is replaced even when startedAt looks current',
+ lock: lockWrittenAt({ processStart: { startedAtMs: WRITTEN_AT_MS - 2_000 } }),
+ live: creationStart(-2_000 + 1_001),
+ expected: 'replaced',
+ },
+ {
+ name: 'a creation start 1001 ms before the token is unverified',
+ lock: lockWrittenAt({ processStart: { startedAtMs: WRITTEN_AT_MS } }),
+ live: creationStart(-1_001),
+ expected: 'unverified',
+ cause: 'process-start-mismatch',
+ },
+ {
+ name: 'a legacy lock whose creation-clock holder started 9999 ms later is the writer',
+ lock: lockWrittenAt(),
+ live: creationStart(9_999),
+ expected: 'writer',
+ },
+ {
+ name: 'a legacy lock whose creation-clock holder started 10000 ms later is the writer',
+ lock: lockWrittenAt(),
+ live: creationStart(10_000),
+ expected: 'writer',
+ },
+ {
+ name: 'a legacy lock whose creation-clock holder started 10001 ms later is replaced',
+ lock: lockWrittenAt(),
+ live: creationStart(10_001),
+ expected: 'replaced',
+ },
+ {
+ name: 'a legacy lock whose boot-derived holder started 299999 ms later is the writer',
+ lock: lockWrittenAt(),
+ live: bootDerivedStart(299_999),
+ expected: 'writer',
+ },
+ {
+ name: 'a legacy lock whose boot-derived holder started 300001 ms later is replaced',
+ lock: lockWrittenAt(),
+ live: bootDerivedStart(300_001),
+ expected: 'replaced',
+ },
+ {
+ name: 'a legacy lock whose holder started before it was written is the writer',
+ lock: lockWrittenAt(),
+ live: creationStart(-60_000),
+ expected: 'writer',
+ },
+ {
+ name: 'refusing later holders, a legacy lock whose creation-clock holder started 1 ms later is unverified',
+ lock: lockWrittenAt(),
+ live: creationStart(1),
+ expected: 'unverified',
+ cause: 'process-started-after-lock',
+ refuseHolderStartedAfterLock: true,
+ },
+ {
+ name: 'refusing later holders, a legacy lock whose creation-clock holder started 9999 ms later is unverified',
+ lock: lockWrittenAt(),
+ live: creationStart(9_999),
+ expected: 'unverified',
+ cause: 'process-started-after-lock',
+ refuseHolderStartedAfterLock: true,
+ },
+ {
+ name: 'refusing later holders, a legacy lock whose creation-clock holder started 10001 ms later is replaced',
+ lock: lockWrittenAt(),
+ live: creationStart(10_001),
+ expected: 'replaced',
+ refuseHolderStartedAfterLock: true,
+ },
+ {
+ name: 'refusing later holders, a legacy lock whose creation-clock holder started as it was written is the writer',
+ lock: lockWrittenAt(),
+ live: creationStart(0),
+ expected: 'writer',
+ refuseHolderStartedAfterLock: true,
+ },
+ {
+ name: 'refusing later holders, a legacy lock whose boot-derived holder started 1 ms later is unverified',
+ lock: lockWrittenAt(),
+ live: bootDerivedStart(1),
+ expected: 'unverified',
+ cause: 'process-started-after-lock',
+ refuseHolderStartedAfterLock: true,
+ },
+ {
+ name: 'refusing later holders, a legacy lock whose boot-derived holder started 299999 ms later is unverified',
+ lock: lockWrittenAt(),
+ live: bootDerivedStart(299_999),
+ expected: 'unverified',
+ cause: 'process-started-after-lock',
+ refuseHolderStartedAfterLock: true,
+ },
+ {
+ name: 'refusing later holders, a legacy lock whose boot-derived holder started 300001 ms later is replaced',
+ lock: lockWrittenAt(),
+ live: bootDerivedStart(300_001),
+ expected: 'replaced',
+ refuseHolderStartedAfterLock: true,
+ },
+ {
+ name: 'refusing later holders, a legacy lock whose boot-derived holder started before it was written is the writer',
+ lock: lockWrittenAt(),
+ live: bootDerivedStart(-60_000),
+ expected: 'writer',
+ refuseHolderStartedAfterLock: true,
+ },
+ {
+ name: 'refusing later holders, a lock with a creation token is judged by the token alone even when its holder started after startedAt',
+ lock: lockWrittenAt({ processStart: { startedAtMs: WRITTEN_AT_MS } }),
+ live: creationStart(1_000),
+ expected: 'writer',
+ refuseHolderStartedAfterLock: true,
+ },
+ {
+ name: 'a legacy lock with an unparseable startedAt is unverified',
+ lock: lockWrittenAt({ startedAt: 'not a date' }),
+ live: creationStart(THIRTY_SIX_HOURS_MS),
+ expected: 'unverified',
+ cause: 'process-start-incomparable',
+ },
+ {
+ name: 'a legacy lock without startedAt is unverified',
+ lock: { pid: HOLDER_PID },
+ live: creationStart(THIRTY_SIX_HOURS_MS),
+ expected: 'unverified',
+ cause: 'process-start-incomparable',
+ },
+ {
+ name: 'a millisecond token read against a boot-derived clock is unverified',
+ lock: lockWrittenAt({ processStart: { startedAtMs: WRITTEN_AT_MS - THIRTY_SIX_HOURS_MS } }),
+ live: bootDerivedStart(1_000),
+ expected: 'unverified',
+ cause: 'process-start-incomparable',
+ },
+ {
+ name: 'a boot pair token read against a creation reading is unverified',
+ lock: lockWrittenAt({ processStart: BOOT_PAIR_TOKEN }),
+ live: creationStart(1_000),
+ expected: 'unverified',
+ cause: 'process-start-incomparable',
+ },
+ {
+ name: 'a boot pair token read against a boot-derived reading without a boot id is unverified',
+ lock: lockWrittenAt({ processStart: BOOT_PAIR_TOKEN }),
+ live: bootDerivedStart(1_000),
+ expected: 'unverified',
+ cause: 'process-start-incomparable',
+ },
+ {
+ name: 'a malformed token falls back to startedAt',
+ lock: lockWrittenAt({ processStart: { startedAtMs: 'soon' } }),
+ live: creationStart(10_001),
+ expected: 'replaced',
+ },
+ {
+ name: 'a token carrying both shapes is read as its boot pair',
+ lock: lockWrittenAt({
+ processStart: { startedAtMs: WRITTEN_AT_MS + 10_001, ...BOOT_PAIR_TOKEN },
+ }),
+ live: creationStart(10_001),
+ expected: 'unverified',
+ cause: 'process-start-incomparable',
+ },
+];
+
+describe('assessLockHolder', () => {
+ test.each(comparatorRows)(
+ '$name, naming the comparison that decided it',
+ ({ lock, live, expected, cause, refuseHolderStartedAfterLock }) => {
+ expect(
+ assessLockHolder(
+ { machineId: getMachineId(), ...lock },
+ {
+ isAlive: () => true,
+ readProcessStart: () => live,
+ sameMachine: true,
+ refuseHolderStartedAfterLock,
+ },
+ ),
+ ).toEqual(expected === 'unverified' ? { verdict: expected, cause } : { verdict: expected });
+ },
+ );
+
+ test('names another machine for its lock when the local start cannot be read either', () => {
+ expect(
+ assessLockHolder(lockWrittenAt({ machineId: 'another-machine-0000' }), {
+ isAlive: () => true,
+ readProcessStart: () => null,
+ }),
+ ).toEqual({ verdict: 'unverified', cause: 'foreign-host' });
+ });
+
+ test('names another machine for its lock when the recorded start contradicts the holder', () => {
+ expect(
+ assessLockHolder(
+ lockWrittenAt({
+ machineId: 'another-machine-0000',
+ processStart: { startedAtMs: WRITTEN_AT_MS },
+ }),
+ { isAlive: () => true, readProcessStart: () => creationStart(-60_000) },
+ ),
+ ).toEqual({ verdict: 'unverified', cause: 'foreign-host' });
+ });
+
+ test('does not read the start of a holder named by a lock that records no owner', () => {
+ const readPids: number[] = [];
+ expect(
+ assessLockHolder(lockWrittenAt(), {
+ isAlive: () => true,
+ readProcessStart: (pid) => {
+ readPids.push(pid);
+ return creationStart(-60_000);
+ },
+ }),
+ ).toEqual({ verdict: 'unverified', cause: 'unverified-owner' });
+ expect(readPids).toEqual([]);
+ });
+
+ test('a lock naming only another hostname with a startedAt in the future vouches for no holder', () => {
+ expect(
+ assessLockHolder(
+ { pid: HOLDER_PID, hostname: 'elsewhere.invalid', startedAt: '2099-01-01T00:00:00.000Z' },
+ {
+ isAlive: () => true,
+ readProcessStart: () => ({ startedAtMs: Date.now() - 60_000, clock: 'creation' }),
+ },
+ ),
+ ).toEqual({ verdict: 'unverified', cause: 'foreign-host' });
+ });
+
+ test('a lock from this machine with a startedAt ahead of the clock is refused for that reason', () => {
+ expect(
+ assessLockHolder(
+ { pid: HOLDER_PID, machineId: getMachineId(), startedAt: '2099-01-01T00:00:00.000Z' },
+ {
+ isAlive: () => true,
+ readProcessStart: () => ({ startedAtMs: Date.now() - 60_000, clock: 'creation' }),
+ },
+ ),
+ ).toEqual({ verdict: 'unverified', cause: 'recorded-start-in-future' });
+ });
+
+ test('a lock naming only this hostname cannot confirm an earlier-started holder', () => {
+ expect(
+ assessLockHolder(lockWrittenAt({ hostname: hostname() }), {
+ isAlive: () => true,
+ readProcessStart: () => creationStart(-60_000),
+ }),
+ ).toEqual({ verdict: 'unverified', cause: 'writer-identity-unrecorded' });
+ });
+
+ test('a lock whose format has no machine ID confirms an earlier-started holder on this hostname', () => {
+ expect(
+ assessLockHolder(lockWrittenAt({ hostname: hostname() }), {
+ isAlive: () => true,
+ readProcessStart: () => creationStart(-60_000),
+ sameMachine: true,
+ formatRecordsMachineId: false,
+ }),
+ ).toEqual({ verdict: 'writer' });
+ });
+
+ test('a lock naming only this hostname whose pid now belongs to a later process is replaced', () => {
+ expect(
+ assessLockHolder(lockWrittenAt({ hostname: hostname() }), {
+ isAlive: () => true,
+ readProcessStart: () => creationStart(THIRTY_SIX_HOURS_MS),
+ }),
+ ).toEqual({ verdict: 'replaced' });
+ });
+
+ test('a holder that exits while its start is read is gone', () => {
+ const liveness = [true, false];
+ expect(
+ assessLockHolder(lockWrittenAt({ machineId: getMachineId() }), {
+ isAlive: () => liveness.shift() ?? false,
+ readProcessStart: () => null,
+ }),
+ ).toEqual({ verdict: 'gone' });
+ });
+});
+
+describe('classifyLockHolder', () => {
+ test('a holder that is no longer running is gone without reading its start', () => {
+ const readPids: number[] = [];
+ const verdict = classifyLockHolder(lockWrittenAt({ machineId: getMachineId() }), {
+ isAlive: () => false,
+ readProcessStart: (pid) => {
+ readPids.push(pid);
+ return creationStart(0);
+ },
+ });
+ expect({ verdict, readPids }).toEqual({ verdict: 'gone', readPids: [] });
+ });
+
+ test('a running holder whose start cannot be read is unverified', () => {
+ expect(
+ classifyLockHolder(lockWrittenAt({ machineId: getMachineId() }), {
+ isAlive: () => true,
+ readProcessStart: () => null,
+ }),
+ ).toBe('unverified');
+ });
+
+ test('a later-started holder of a lock from this machine is replaced', () => {
+ expect(
+ classifyLockHolder(lockWrittenAt({ machineId: getMachineId() }), {
+ isAlive: () => true,
+ readProcessStart: () => creationStart(THIRTY_SIX_HOURS_MS),
+ }),
+ ).toBe('replaced');
+ });
+
+ test('a later-started holder of a lock from another machine is unverified', () => {
+ expect(
+ classifyLockHolder(lockWrittenAt({ machineId: 'another-machine-0000' }), {
+ isAlive: () => true,
+ readProcessStart: () => creationStart(THIRTY_SIX_HOURS_MS),
+ }),
+ ).toBe('unverified');
+ });
+
+ test('a caller that has established the lock is local keeps the replaced verdict', () => {
+ expect(
+ classifyLockHolder(lockWrittenAt({ machineId: 'another-machine-0000' }), {
+ isAlive: () => true,
+ readProcessStart: () => creationStart(THIRTY_SIX_HOURS_MS),
+ sameMachine: true,
+ }),
+ ).toBe('replaced');
+ });
+
+ test('a caller that has established the lock is foreign downgrades replaced to unverified', () => {
+ expect(
+ classifyLockHolder(lockWrittenAt({ machineId: getMachineId() }), {
+ isAlive: () => true,
+ readProcessStart: () => creationStart(THIRTY_SIX_HOURS_MS),
+ sameMachine: false,
+ }),
+ ).toBe('unverified');
+ });
+
+ test('an earlier-started holder of a lock from another machine id is unverified without a recorded start', () => {
+ expect(
+ classifyLockHolder(lockWrittenAt({ machineId: 'another-machine-0000' }), {
+ isAlive: () => true,
+ readProcessStart: () => creationStart(-60_000),
+ }),
+ ).toBe('unverified');
+ });
+
+ test('a holder matching the recorded start is the writer of a lock from another machine id', () => {
+ expect(
+ classifyLockHolder(
+ lockWrittenAt({
+ machineId: 'another-machine-0000',
+ processStart: { startedAtMs: WRITTEN_AT_MS - 60_000 },
+ }),
+ { isAlive: () => true, readProcessStart: () => creationStart(-60_000) },
+ ),
+ ).toBe('writer');
+ });
+
+ test('an earlier-started holder of a lock naming only another hostname is unverified', () => {
+ expect(
+ classifyLockHolder(lockWrittenAt({ hostname: 'elsewhere.invalid' }), {
+ isAlive: () => true,
+ readProcessStart: () => creationStart(-60_000),
+ }),
+ ).toBe('unverified');
+ });
+
+ test('an earlier-started holder of a lock that records no owner is unverified', () => {
+ expect(
+ classifyLockHolder(lockWrittenAt(), {
+ isAlive: () => true,
+ readProcessStart: () => creationStart(-60_000),
+ }),
+ ).toBe('unverified');
+ });
+
+ test('a reader that has established the lock is local judges an ownerless lock by its start', () => {
+ expect(
+ classifyLockHolder(lockWrittenAt(), {
+ isAlive: () => true,
+ readProcessStart: () => creationStart(THIRTY_SIX_HOURS_MS),
+ sameMachine: true,
+ }),
+ ).toBe('replaced');
+ });
+});
+
+describe('createRecentProcessStartReader', () => {
+ function recentReader(readings: Array<ProcessStart | null>, readTakesMs = 0) {
+ let nowMs = 0;
+ const readPids: number[] = [];
+ const read = createRecentProcessStartReader({
+ maxAgeMs: 1_000,
+ read: (pid) => {
+ readPids.push(pid);
+ nowMs += readTakesMs;
+ return readings.shift() ?? null;
+ },
+ now: () => nowMs,
+ });
+ return {
+ read,
+ readPids,
+ advance: (ms: number) => {
+ nowMs += ms;
+ },
+ };
+ }
+
+ test('answers a repeated read of a pid inside the maximum age without reading it again', () => {
+ const recent = recentReader([creationStart(0), creationStart(THIRTY_SIX_HOURS_MS)]);
+ expect(recent.read(HOLDER_PID)).toEqual(creationStart(0));
+ recent.advance(999);
+ expect(recent.read(HOLDER_PID)).toEqual(creationStart(0));
+ expect(recent.readPids).toEqual([HOLDER_PID]);
+ });
+
+ test('reads the pid again once its reading reaches the maximum age', () => {
+ const recent = recentReader([creationStart(0), creationStart(THIRTY_SIX_HOURS_MS)]);
+ recent.read(HOLDER_PID);
+ recent.advance(1_000);
+ expect(recent.read(HOLDER_PID)).toEqual(creationStart(THIRTY_SIX_HOURS_MS));
+ expect(recent.readPids).toEqual([HOLDER_PID, HOLDER_PID]);
+ });
+
+ test('does not retry a failed read inside the maximum age', () => {
+ const recent = recentReader([null, creationStart(0)]);
+ expect(recent.read(HOLDER_PID)).toBeNull();
+ recent.advance(500);
+ expect(recent.read(HOLDER_PID)).toBeNull();
+ expect(recent.readPids).toEqual([HOLDER_PID]);
+ });
+
+ test('counts the age of a reading from when its read returned, however long the read took', () => {
+ const recent = recentReader([creationStart(0), creationStart(THIRTY_SIX_HOURS_MS)], 1_500);
+ expect(recent.read(HOLDER_PID)).toEqual(creationStart(0));
+ recent.advance(999);
+ expect(recent.read(HOLDER_PID)).toEqual(creationStart(0));
+ expect(recent.readPids).toEqual([HOLDER_PID]);
+ });
+
+ test('holds a failed reading for as long as a timed-out ps read can block', () => {
+ const recent = recentReader([null, creationStart(0)], 5_000);
+ expect(recent.read(HOLDER_PID)).toBeNull();
+ recent.advance(4_999);
+ expect(recent.read(HOLDER_PID)).toBeNull();
+ expect(recent.readPids).toEqual([HOLDER_PID]);
+ recent.advance(1);
+ expect(recent.read(HOLDER_PID)).toEqual(creationStart(0));
+ expect(recent.readPids).toEqual([HOLDER_PID, HOLDER_PID]);
+ });
+
+ test('reads each pid on its own', () => {
+ const recent = recentReader([creationStart(0), creationStart(1)]);
+ recent.read(HOLDER_PID);
+ expect(recent.read(HOLDER_PID + 1)).toEqual(creationStart(1));
+ expect(recent.readPids).toEqual([HOLDER_PID, HOLDER_PID + 1]);
+ });
+});
+
+describe('isLockFromThisMachine', () => {
+ test('a matching machine id is this machine even when the hostname differs', () => {
+ expect(
+ isLockFromThisMachine({ machineId: getMachineId(), hostname: 'elsewhere.invalid' }),
+ ).toBe(true);
+ });
+
+ test('a different machine id is another machine even when the hostname matches', () => {
+ expect(isLockFromThisMachine({ machineId: 'another-machine-0000', hostname: hostname() })).toBe(
+ false,
+ );
+ });
+
+ test('without a machine id a matching hostname is this machine', () => {
+ expect(isLockFromThisMachine({ hostname: hostname() })).toBe(true);
+ });
+
+ test('without a machine id a different hostname is another machine', () => {
+ expect(isLockFromThisMachine({ hostname: 'elsewhere.invalid' })).toBe(false);
+ });
+});
+
+describe('processStartToken', () => {
+ test('a reading with a boot pair records the pair', () => {
+ expect(
+ processStartToken({
+ startedAtMs: WRITTEN_AT_MS,
+ clock: 'boot-derived',
+ startTicks: 7_311,
+ bootId: 'boot-a',
+ }),
+ ).toEqual({ bootId: 'boot-a', startTicks: 7_311 });
+ });
+
+ test('a creation reading records its milliseconds', () => {
+ expect(processStartToken(creationStart(0))).toEqual({ startedAtMs: WRITTEN_AT_MS });
+ });
+
+ test('a boot-derived reading without a boot id records nothing', () => {
+ expect(
+ processStartToken({ startedAtMs: WRITTEN_AT_MS, clock: 'boot-derived', startTicks: 7_311 }),
+ ).toBeUndefined();
+ expect(processStartToken(bootDerivedStart(0))).toBeUndefined();
+ });
+
+ test('no reading records nothing', () => {
+ expect(processStartToken(null)).toBeUndefined();
+ });
+});
+
+describe('parseProcessStartToken', () => {
+ test.each([
+ { name: 'a boot pair', value: { bootId: 'boot-a', startTicks: 7_311 } },
+ { name: 'a boot pair at tick zero', value: { bootId: 'boot-a', startTicks: 0 } },
+ { name: 'a creation time', value: { startedAtMs: WRITTEN_AT_MS } },
+ { name: 'a creation time at the epoch', value: { startedAtMs: 0 } },
+ ])('accepts $name', ({ value }) => {
+ expect(parseProcessStartToken(value)).toEqual(value);
+ });
+
+ test.each([
+ { name: 'null', value: null },
+ { name: 'a string', value: '{"startedAtMs":1}' },
+ { name: 'a number', value: WRITTEN_AT_MS },
+ { name: 'an array', value: [WRITTEN_AT_MS] },
+ { name: 'an empty object', value: {} },
+ { name: 'an empty boot id', value: { bootId: '', startTicks: 1 } },
+ { name: 'an overlong boot id', value: { bootId: 'b'.repeat(65), startTicks: 1 } },
+ { name: 'a non-string boot id', value: { bootId: 42, startTicks: 1 } },
+ { name: 'negative ticks', value: { bootId: 'boot-a', startTicks: -1 } },
+ { name: 'fractional ticks', value: { bootId: 'boot-a', startTicks: 1.5 } },
+ { name: 'a boot id without ticks', value: { bootId: 'boot-a' } },
+ { name: 'ticks without a boot id', value: { startTicks: 1 } },
+ { name: 'negative milliseconds', value: { startedAtMs: -1 } },
+ { name: 'unsafe milliseconds', value: { startedAtMs: 2 ** 53 } },
+ { name: 'string milliseconds', value: { startedAtMs: String(WRITTEN_AT_MS) } },
+ { name: 'only unknown fields', value: { pid: 1, creationTicks: 9 } },
+ ])('rejects $name', ({ value }) => {
+ expect(parseProcessStartToken(value)).toBeUndefined();
+ });
+
+ test.each([
+ {
+ name: 'a creation time with an extra field',
+ value: { startedAtMs: WRITTEN_AT_MS, pid: 1 },
+ token: { startedAtMs: WRITTEN_AT_MS },
+ },
+ {
+ name: 'a boot pair with an extra field',
+ value: { bootId: 'boot-a', startTicks: 7_311, namespace: 'pid:[4026531836]' },
+ token: { bootId: 'boot-a', startTicks: 7_311 },
+ },
+ {
+ name: 'both shapes at once, preferring the boot pair',
+ value: { startedAtMs: WRITTEN_AT_MS, bootId: 'boot-a', startTicks: 1 },
+ token: { bootId: 'boot-a', startTicks: 1 },
+ },
+ {
+ name: 'both shapes with an invalid boot pair, falling back to the creation time',
+ value: { startedAtMs: WRITTEN_AT_MS, bootId: '', startTicks: 1 },
+ token: { startedAtMs: WRITTEN_AT_MS },
+ },
+ ])('reads $name and drops what it does not know', ({ value, token }) => {
+ expect(parseProcessStartToken(value)).toEqual(token);
+ });
+
+ test('every token a writer records parses back to itself', () => {
+ const readings: ProcessStart[] = [
+ creationStart(0),
+ { startedAtMs: WRITTEN_AT_MS, clock: 'boot-derived', startTicks: 0, bootId: 'boot-a' },
+ {
+ startedAtMs: WRITTEN_AT_MS,
+ clock: 'boot-derived',
+ startTicks: 7_311,
+ bootId: 'b'.repeat(64),
+ },
+ ];
+ for (const reading of readings) {
+ const token = processStartToken(reading);
+ expect(token).toBeDefined();
+ expect(parseProcessStartToken(token)).toEqual(token);
+ }
+ });
+});
+
+const FIXTURE_PID = 4242;
+const FIXTURE_START_TICKS = 98_765_432;
+const FIXTURE_BTIME_SECONDS = 1_760_000_000;
+const FIXTURE_BOOT_ID = '6f1c2a3b-0000-4000-8000-0123456789ab';
+const FIXTURE_STAT_WITH_AWKWARD_COMM = `${FIXTURE_PID} (w) x (y) z) S 1 ${FIXTURE_PID} ${FIXTURE_PID} 0 -1 4194560 120 0 0 0 3 1 0 0 20 0 1 0 ${FIXTURE_START_TICKS} 1060864 220 18446744073709551615 1 1 0 0 0 0 0 0 0 0 0 0 17 3 0 0 0 0 0\n`;
+const FIXTURE_PROC_STAT = `cpu 10 0 10 1000 0 0 0 0 0 0\ncpu0 10 0 10 1000 0 0 0 0 0 0\nintr 1 0\nctxt 123\nbtime ${FIXTURE_BTIME_SECONDS}\nprocesses 99\nprocs_running 1\n`;
+
+function procFiles(files: Record<string, string>): (path: string) => string {
+ return (path) => {
+ const content = files[path];
+ if (content === undefined) {
+ throw Object.assign(new Error(`ENOENT: no such file or directory, open '${path}'`), {
+ code: 'ENOENT',
+ });
+ }
+ return content;
+ };
+}
+
+function linuxProcFor(pid: number, overrides: Record<string, string | undefined> = {}) {
+ const files: Record<string, string> = {
+ [`/proc/${pid}/stat`]: FIXTURE_STAT_WITH_AWKWARD_COMM,
+ '/proc/stat': FIXTURE_PROC_STAT,
+ '/proc/sys/kernel/random/boot_id': `${FIXTURE_BOOT_ID}\n`,
+ };
+ for (const [path, content] of Object.entries(overrides)) {
+ if (content === undefined) delete files[path];
+ else files[path] = content;
+ }
+ return procFiles(files);
+}
+
+describe('readProcessStart on Linux', () => {
+ test('reads start ticks after the last parenthesis of a comm containing spaces and parentheses', () => {
+ expect(
+ readProcessStart(FIXTURE_PID, { platform: 'linux', readFile: linuxProcFor(FIXTURE_PID) }),
+ ).toEqual({
+ startedAtMs: FIXTURE_BTIME_SECONDS * 1000 + FIXTURE_START_TICKS * 10,
+ clock: 'boot-derived',
+ startTicks: FIXTURE_START_TICKS,
+ bootId: FIXTURE_BOOT_ID,
+ });
+ });
+
+ test('a host whose boot id cannot be read yields a reading that records no token', () => {
+ const start = readProcessStart(FIXTURE_PID, {
+ platform: 'linux',
+ readFile: linuxProcFor(FIXTURE_PID, { '/proc/sys/kernel/random/boot_id': undefined }),
+ });
+ expect(start).toEqual({
+ startedAtMs: FIXTURE_BTIME_SECONDS * 1000 + FIXTURE_START_TICKS * 10,
+ clock: 'boot-derived',
+ startTicks: FIXTURE_START_TICKS,
+ });
+ expect(processStartToken(start)).toBeUndefined();
+ });
+
+ test('a pid without a /proc entry has no reading', () => {
+ expect(
+ readProcessStart(FIXTURE_PID + 1, {
+ platform: 'linux',
+ readFile: linuxProcFor(FIXTURE_PID),
+ }),
+ ).toBeNull();
+ });
+
+ test('a truncated stat line has no reading', () => {
+ expect(
+ readProcessStart(FIXTURE_PID, {
+ platform: 'linux',
+ readFile: linuxProcFor(FIXTURE_PID, {
+ [`/proc/${FIXTURE_PID}/stat`]: `${FIXTURE_PID} (w) x (y) z) S 1 ${FIXTURE_PID}\n`,
+ }),
+ }),
+ ).toBeNull();
+ });
+
+ test('a /proc/stat without btime has no reading', () => {
+ expect(
+ readProcessStart(FIXTURE_PID, {
+ platform: 'linux',
+ readFile: linuxProcFor(FIXTURE_PID, { '/proc/stat': 'cpu 10 0 10 1000\nctxt 123\n' }),
+ }),
+ ).toBeNull();
+ });
+
+ test.each([0, -1, 1.5, Number.NaN, 0x1_0000_0000])(
+ 'pid %s is not a process id and has no reading',
+ (pid) => {
+ const readFile = (path: string) =>
+ path.endsWith('/stat') && path !== '/proc/stat'
+ ? FIXTURE_STAT_WITH_AWKWARD_COMM
+ : linuxProcFor(FIXTURE_PID)(path);
+ expect(readProcessStart(pid, { platform: 'linux', readFile })).toBeNull();
+ },
+ );
+});
+
+describe('the registered native process-start reader', () => {
+ const unregisters: Array<() => void> = [];
+
+ afterEach(() => {
+ for (const unregister of unregisters.splice(0)) unregister();
+ });
+
+ function register(read: (pid: number) => number | null): void {
+ unregisters.push(registerNativeProcessStartReader(read));
+ }
+
+ test('is the Windows reading for the pid it is asked about', () => {
+ const askedPids: number[] = [];
+ register((pid) => {
+ askedPids.push(pid);
+ return 1_700_000_000_123;
+ });
+ expect(readProcessStart(FIXTURE_PID, { platform: 'win32' })).toEqual({
+ startedAtMs: 1_700_000_000_123,
+ clock: 'creation',
+ });
+ expect(askedPids).toEqual([FIXTURE_PID]);
+ });
+
+ test('is not consulted on Linux or on an unsupported platform', () => {
+ const askedPids: number[] = [];
+ register((pid) => {
+ askedPids.push(pid);
+ return 1_700_000_000_123;
+ });
+ expect(
+ readProcessStart(FIXTURE_PID, { platform: 'linux', readFile: linuxProcFor(FIXTURE_PID) }),
+ ).toMatchObject({ clock: 'boot-derived', startTicks: FIXTURE_START_TICKS });
+ expect(readProcessStart(FIXTURE_PID, { platform: 'aix' })).toBeNull();
+ expect(askedPids).toEqual([]);
+ });
+
+ test('leaves Windows without a reading once unregistered', () => {
+ const unregister = registerNativeProcessStartReader(() => 1_700_000_000_123);
+ expect(readProcessStart(FIXTURE_PID, { platform: 'win32' })).not.toBeNull();
+ unregister();
+ expect(readProcessStart(FIXTURE_PID, { platform: 'win32' })).toBeNull();
+ });
+
+ test('a reader that cannot identify the process leaves Windows without a reading', () => {
+ register(() => null);
+ expect(readProcessStart(FIXTURE_PID, { platform: 'win32' })).toBeNull();
+ });
+
+ test('gives way to a reader passed for the read', () => {
+ register(() => 1_700_000_000_123);
+ expect(
+ readProcessStart(FIXTURE_PID, { platform: 'win32', readNative: () => 1_700_000_000_456 }),
+ ).toEqual({ startedAtMs: 1_700_000_000_456, clock: 'creation' });
+ });
+
+ test('warns once while Windows has no reader to ask', () => {
+ registerNativeProcessStartReader(() => null)();
+ const warn = vi.spyOn(getLogger('process-identity'), 'warn');
+ try {
+ expect(readProcessStart(FIXTURE_PID, { platform: 'win32' })).toBeNull();
+ expect(readProcessStart(FIXTURE_PID + 1, { platform: 'win32' })).toBeNull();
+ expect(warn).toHaveBeenCalledTimes(1);
+ expect(warn).toHaveBeenCalledWith(
+ { event: 'process-start-reader-missing' },
+ expect.any(String),
+ );
+ } finally {
+ warn.mockRestore();
+ }
+ });
+
+ test('warns again when a withdrawn reader leaves Windows without one', () => {
+ registerNativeProcessStartReader(() => null)();
+ const warn = vi.spyOn(getLogger('process-identity'), 'warn');
+ try {
+ expect(readProcessStart(FIXTURE_PID, { platform: 'win32' })).toBeNull();
+ const unregister = registerNativeProcessStartReader(() => 1_700_000_000_123);
+ expect(readProcessStart(FIXTURE_PID, { platform: 'win32' })).not.toBeNull();
+ unregister();
+ expect(readProcessStart(FIXTURE_PID, { platform: 'win32' })).toBeNull();
+ expect(warn).toHaveBeenCalledTimes(2);
+ } finally {
+ warn.mockRestore();
+ }
+ });
+});
+
+describe('readPsProcessStart', () => {
+ test('queries ps in the C locale at UTC and parses lstart as UTC', () => {
+ const calls: Array<{
+ command: string;
+ args: string[];
+ options: ExecFileSyncOptionsWithStringEncoding;
+ }> = [];
+ const startedAtMs = readPsProcessStart(FIXTURE_PID, {
+ env: { PATH: '/usr/bin', LC_ALL: 'de_DE.UTF-8', TZ: 'Europe/Berlin' },
+ run: (command, args, options) => {
+ calls.push({ command, args, options });
+ return 'Tue Oct 13 17:27:40 2026\n';
+ },
+ });
+ expect(startedAtMs).toBe(Date.UTC(2026, 9, 13, 17, 27, 40));
+ expect(calls).toHaveLength(1);
+ expect(calls[0]).toMatchObject({
+ command: 'ps',
+ args: ['-p', String(FIXTURE_PID), '-o', 'lstart='],
+ options: {
+ encoding: 'utf8',
+ timeout: 5000,
+ windowsHide: true,
+ env: { PATH: '/usr/bin', LC_ALL: 'C', TZ: 'UTC0' },
+ },
+ });
+ });
+
+ test.each([
+ ' PID TTY\n',
+ '',
+ 'permission denied',
+ 'Tue Sep 8 20:21:34 2026\nTue Sep 8 20:21:34 2026',
+ ])('output %j that is not one lstart date has no reading', (output) => {
+ expect(readPsProcessStart(FIXTURE_PID, { run: () => output })).toBeNull();
+ });
+
+ test('a ps that cannot run or times out has no reading', () => {
+ expect(
+ readPsProcessStart(FIXTURE_PID, {
+ run: () => {
+ throw Object.assign(new Error('spawnSync ps ETIMEDOUT'), { code: 'ETIMEDOUT' });
+ },
+ }),
+ ).toBeNull();
+ });
+
+ test('a ps that exits unsuccessfully has no reading', () => {
+ expect(
+ readPsProcessStart(FIXTURE_PID, {
+ run: (_command, _args, options) =>
+ execFileSync(process.execPath, ['-e', 'process.exit(1)'], options),
+ }),
+ ).toBeNull();
+ });
+
+ test.each([0, -1, Number.NaN, 12.5, 0x1_0000_0000])(
+ 'an invalid pid %j has no reading and runs nothing',
+ (pid) => {
+ const commands: string[] = [];
+ expect(
+ readPsProcessStart(pid, {
+ run: (command) => {
+ commands.push(command);
+ return 'Tue Oct 13 17:27:40 2026\n';
+ },
+ }),
+ ).toBeNull();
+ expect(commands).toEqual([]);
+ },
+ );
+});
+
+const SECOND_TRUNCATION_PLUS_ONE_TICK_MS = 1_010;
+
+describe.skipIf(process.platform === 'win32')('readProcessStart on this host', () => {
+ const owned: OwnedProcess[] = [];
+
+ afterEach(async () => {
+ for (const holder of owned.splice(0)) await stopOwnedProcess(holder);
+ });
+
+ async function startHolder(): Promise<OwnedProcess> {
+ const holder = await startOwnedProcess();
+ owned.push(holder);
+ return holder;
+ }
+
+ test('reads this process the way the operating system records it', () => {
+ expect(readProcessStart(process.pid)).toEqual(readProcessStartFromOs(process.pid));
+ });
+
+ test('reads a child inside the window in which it was spawned', async () => {
+ const holder = await startHolder();
+ const start = readProcessStart(holder.pid);
+ expect(start).toEqual(readProcessStartFromOs(holder.pid));
+ expect(start?.startedAtMs).toBeGreaterThanOrEqual(
+ holder.spawnRequestedAtMs - SECOND_TRUNCATION_PLUS_ONE_TICK_MS,
+ );
+ expect(start?.startedAtMs).toBeLessThanOrEqual(holder.readyAtMs);
+ });
+
+ test('classifies a child holding a lock written before it started as replaced', async () => {
+ const holder = await startHolder();
+ expect(
+ classifyLockHolder({
+ pid: holder.pid,
+ startedAt: lockWrittenBeforeProcessStarted(holder),
+ machineId: getMachineId(),
+ }),
+ ).toBe('replaced');
+ });
+
+ test('classifies a child holding the lock it wrote as the writer', async () => {
+ const holder = await startHolder();
+ expect(
+ classifyLockHolder({
+ pid: holder.pid,
+ startedAt: lockWrittenByRunningProcess(holder),
+ machineId: getMachineId(),
+ }),
+ ).toBe('writer');
+ });
+
+ test('classifies a child as the writer when its recorded start matches, whatever startedAt says', async () => {
+ const holder = await startHolder();
+ const recorded = processStartTokenFromOs(holder.pid);
+ expect(
+ classifyLockHolder({
+ pid: holder.pid,
+ startedAt: lockWrittenBeforeProcessStarted(holder),
+ machineId: getMachineId(),
+ processStart: recorded,
+ }),
+ ).toBe('writer');
+ });
+});
diff --git a/packages/server/src/process-identity.ts b/packages/server/src/process-identity.ts
new file mode 100644
index 000000000..7b47e093e
--- /dev/null
+++ b/packages/server/src/process-identity.ts
@@ -0,0 +1,445 @@
+import { type ExecFileSyncOptionsWithStringEncoding, execFileSync } from 'node:child_process';
+import { readFileSync } from 'node:fs';
+import { hostname } from 'node:os';
+import {
+ LOCAL_OP_PIPE_STDIO_OPTIONS,
+ withHiddenWindowsConsole,
+} from './child-process-windows-hide.ts';
+import { getLogger } from './logger.ts';
+import { getMachineId } from './machine-id.ts';
+import { isProcessAlive } from './process-alive.ts';
+
+const log = getLogger('process-identity');
+
+export interface ProcessStart {
+ startedAtMs: number;
+ clock: 'creation' | 'boot-derived';
+ startTicks?: number;
+ bootId?: string;
+}
+
+export type ProcessStartToken = { bootId: string; startTicks: number } | { startedAtMs: number };
+
+export type ProcessStartReader = (pid: number) => ProcessStart | null;
+
+export type LockHolderVerdict = 'writer' | 'replaced' | 'gone' | 'unverified';
+
+export type UnverifiedLockHolderCause =
+ | 'process-start-unreadable'
+ | 'process-start-incomparable'
+ | 'process-start-mismatch'
+ | 'process-started-after-lock'
+ | 'recorded-start-in-future'
+ | 'writer-identity-unrecorded'
+ | 'foreign-host'
+ | 'unverified-owner';
+
+export type ProcessStartCause = Exclude<
+ UnverifiedLockHolderCause,
+ 'foreign-host' | 'unverified-owner'
+>;
+
+type LockHolderComparison =
+ | { readonly verdict: 'writer' | 'replaced' }
+ | { readonly verdict: 'unverified'; readonly cause: ProcessStartCause };
+
+export type LockHolderAssessment =
+ | { readonly verdict: 'writer' | 'replaced' | 'gone' }
+ | { readonly verdict: 'unverified'; readonly cause: UnverifiedLockHolderCause };
+
+export interface LockOwnerFields {
+ readonly machineId?: unknown;
+ readonly hostname?: unknown;
+}
+
+export interface LockHolderIdentity {
+ pid: number;
+ startedAt?: string;
+ processStart?: unknown;
+ machineId?: string;
+ hostname?: string;
+}
+
+export type NativeProcessStartReader = (pid: number) => number | null;
+
+type PsRunner = (
+ command: string,
+ args: string[],
+ options: ExecFileSyncOptionsWithStringEncoding,
+) => string;
+
+export interface PsProcessStartOptions {
+ run?: PsRunner;
+ env?: NodeJS.ProcessEnv;
+}
+
+export interface ReadProcessStartOptions {
+ platform?: NodeJS.Platform;
+ readFile?: (path: string) => string;
+ readNative?: NativeProcessStartReader;
+}
+
+export interface RecentProcessStartReaderOptions {
+ maxAgeMs: number;
+ read?: ProcessStartReader;
+ now?: () => number;
+}
+
+export interface ClassifyLockHolderOptions {
+ isAlive?: (pid: number) => boolean;
+ readProcessStart?: ProcessStartReader;
+ sameMachine?: boolean;
+ formatRecordsMachineId?: boolean;
+ refuseHolderStartedAfterLock?: boolean;
+}
+
+const TOKEN_TOLERANCE_MS = 1000;
+const LEGACY_MARGIN_CREATION_MS = 10_000;
+const LEGACY_MARGIN_BOOT_DERIVED_MS = 300_000;
+
+const MAX_PROCESS_ID = 0xffff_ffff;
+const MAX_BOOT_ID_LENGTH = 64;
+const MS_PER_USER_HZ_TICK = 10;
+const PROC_STAT_START_TICKS_INDEX_AFTER_COMM = 19;
+const BOOT_ID_PATH = '/proc/sys/kernel/random/boot_id';
+const PS_PROCESS_START_QUERY_TIMEOUT_MS = 5000;
+const PS_LSTART_PATTERN = /^\w{3}\s+\w{3}\s+\d{1,2}\s+\d{2}:\d{2}:\d{2}\s+\d{4}$/;
+
+const NATIVE_PROCESS_START_READER: unique symbol = Symbol.for(
+ '@inkeep/open-knowledge/native-process-start-reader',
+);
+
+const nativeReaderHost = globalThis as typeof globalThis & {
+ [NATIVE_PROCESS_START_READER]?: NativeProcessStartReader;
+};
+
+let ownProcessStart: ProcessStart | undefined;
+let ownBootId: string | undefined;
+let missingReaderWarned = false;
+
+function isReadableProcessId(pid: number): boolean {
+ return Number.isSafeInteger(pid) && pid > 0 && pid <= MAX_PROCESS_ID;
+}
+
+function isNonNegativeSafeInteger(value: unknown): value is number {
+ return typeof value === 'number' && Number.isSafeInteger(value) && value >= 0;
+}
+
+function isBootId(value: unknown): value is string {
+ return typeof value === 'string' && value.length > 0 && value.length <= MAX_BOOT_ID_LENGTH;
+}
+
+function readFileUtf8(path: string): string {
+ return readFileSync(path, 'utf8');
+}
+
+export function readPsProcessStart(
+ pid: number,
+ options: PsProcessStartOptions = {},
+): number | null {
+ if (!isReadableProcessId(pid)) return null;
+ const env = options.env ?? process.env;
+ const run = options.run ?? execFileSync;
+ const commandOptions: ExecFileSyncOptionsWithStringEncoding = withHiddenWindowsConsole({
+ encoding: 'utf8',
+ ...LOCAL_OP_PIPE_STDIO_OPTIONS,
+ timeout: PS_PROCESS_START_QUERY_TIMEOUT_MS,
+ env: { ...env, LC_ALL: 'C', TZ: 'UTC0' },
+ });
+ try {
+ const raw = run('ps', ['-p', String(pid), '-o', 'lstart='], commandOptions).trim();
+ if (!PS_LSTART_PATTERN.test(raw)) return null;
+ const startedAt = Date.parse(`${raw} UTC`);
+ return Number.isFinite(startedAt) ? startedAt : null;
+ } catch {
+ return null;
+ }
+}
+
+export function registerNativeProcessStartReader(read: NativeProcessStartReader): () => void {
+ nativeReaderHost[NATIVE_PROCESS_START_READER] = read;
+ return () => {
+ if (nativeReaderHost[NATIVE_PROCESS_START_READER] === read) {
+ delete nativeReaderHost[NATIVE_PROCESS_START_READER];
+ missingReaderWarned = false;
+ }
+ };
+}
+
+function parseProcStatStartTicks(stat: string): number | null {
+ const commEnd = stat.lastIndexOf(')');
+ if (commEnd < 0) return null;
+ const fields = stat
+ .slice(commEnd + 1)
+ .trim()
+ .split(/\s+/);
+ const raw = fields[PROC_STAT_START_TICKS_INDEX_AFTER_COMM];
+ if (raw === undefined || !/^\d+$/.test(raw)) return null;
+ const ticks = Number(raw);
+ return Number.isSafeInteger(ticks) ? ticks : null;
+}
+
+function parseBootTimeSeconds(stat: string): number | null {
+ const match = /^btime\s+(\d+)\s*$/m.exec(stat);
+ if (!match?.[1]) return null;
+ const seconds = Number(match[1]);
+ return Number.isSafeInteger(seconds) ? seconds : null;
+}
+
+function readBootId(readFile: (path: string) => string): string | undefined {
+ const memoize = readFile === readFileUtf8;
+ if (memoize && ownBootId !== undefined) return ownBootId;
+ let bootId: string;
+ try {
+ bootId = readFile(BOOT_ID_PATH).trim();
+ } catch {
+ return undefined;
+ }
+ if (!isBootId(bootId)) return undefined;
+ if (memoize) ownBootId = bootId;
+ return bootId;
+}
+
+function readLinuxProcessStart(
+ pid: number,
+ readFile: (path: string) => string,
+): ProcessStart | null {
+ const startTicks = parseProcStatStartTicks(readFile(`/proc/${pid}/stat`));
+ if (startTicks === null) return null;
+ const bootSeconds = parseBootTimeSeconds(readFile('/proc/stat'));
+ if (bootSeconds === null) return null;
+ const start: ProcessStart = {
+ startedAtMs: bootSeconds * 1000 + startTicks * MS_PER_USER_HZ_TICK,
+ clock: 'boot-derived',
+ startTicks,
+ };
+ const bootId = readBootId(readFile);
+ return bootId === undefined ? start : { ...start, bootId };
+}
+
+function readWindowsProcessStart(
+ pid: number,
+ readNative: NativeProcessStartReader | undefined,
+): ProcessStart | null {
+ const read = readNative ?? nativeReaderHost[NATIVE_PROCESS_START_READER];
+ if (read === undefined) {
+ if (!missingReaderWarned) {
+ missingReaderWarned = true;
+ log.warn(
+ { event: 'process-start-reader-missing' },
+ '[process-identity] no Windows process-start reader is registered, so lock holders cannot be verified; later reads are not recorded again while it stays missing',
+ );
+ }
+ return null;
+ }
+ const startedAtMs = read(pid);
+ return isNonNegativeSafeInteger(startedAtMs) ? { startedAtMs, clock: 'creation' } : null;
+}
+
+function readUncachedProcessStart(
+ pid: number,
+ platform: NodeJS.Platform,
+ readFile: (path: string) => string,
+ readNative: NativeProcessStartReader | undefined,
+): ProcessStart | null {
+ try {
+ if (platform === 'linux') return readLinuxProcessStart(pid, readFile);
+ if (platform === 'darwin') {
+ const startedAtMs = readPsProcessStart(pid);
+ return startedAtMs === null ? null : { startedAtMs, clock: 'creation' };
+ }
+ if (platform === 'win32') return readWindowsProcessStart(pid, readNative);
+ return null;
+ } catch {
+ return null;
+ }
+}
+
+export function readProcessStart(
+ pid: number,
+ options: ReadProcessStartOptions = {},
+): ProcessStart | null {
+ if (!isReadableProcessId(pid)) return null;
+ const usesHost =
+ options.platform === undefined &&
+ options.readFile === undefined &&
+ options.readNative === undefined;
+ if (usesHost && pid === process.pid && ownProcessStart !== undefined) return ownProcessStart;
+ const start = readUncachedProcessStart(
+ pid,
+ options.platform ?? process.platform,
+ options.readFile ?? readFileUtf8,
+ options.readNative,
+ );
+ if (usesHost && pid === process.pid && start !== null) ownProcessStart = start;
+ return start;
+}
+
+export function createRecentProcessStartReader(
+ options: RecentProcessStartReaderOptions,
+): ProcessStartReader {
+ const read = options.read ?? readProcessStart;
+ const now = options.now ?? (() => performance.now());
+ const failedReadHoldMs = Math.max(options.maxAgeMs, PS_PROCESS_START_QUERY_TIMEOUT_MS);
+ const recent = new Map<number, { expiresAtMs: number; start: ProcessStart | null }>();
+ return (pid) => {
+ const at = now();
+ for (const [cachedPid, entry] of recent) {
+ if (at >= entry.expiresAtMs) recent.delete(cachedPid);
+ }
+ const cached = recent.get(pid);
+ if (cached !== undefined) return cached.start;
+ const start = read(pid);
+ const holdMs = start === null ? failedReadHoldMs : options.maxAgeMs;
+ recent.set(pid, { expiresAtMs: now() + holdMs, start });
+ return start;
+ };
+}
+
+export function processStartToken(
+ start: ProcessStart | null | undefined,
+): ProcessStartToken | undefined {
+ if (!start) return undefined;
+ if (start.bootId !== undefined && start.startTicks !== undefined) {
+ return { bootId: start.bootId, startTicks: start.startTicks };
+ }
+ if (start.clock === 'creation') return { startedAtMs: start.startedAtMs };
+ return undefined;
+}
+
+export function parseProcessStartToken(value: unknown): ProcessStartToken | undefined {
+ if (!value || typeof value !== 'object' || Array.isArray(value)) return undefined;
+ const { bootId, startTicks, startedAtMs } = value as Record<string, unknown>;
+ if (isBootId(bootId) && isNonNegativeSafeInteger(startTicks)) return { bootId, startTicks };
+ if (isNonNegativeSafeInteger(startedAtMs)) return { startedAtMs };
+ return undefined;
+}
+
+export function recordsLockOwner(lock: LockOwnerFields): boolean {
+ return typeof lock.machineId === 'string' || typeof lock.hostname === 'string';
+}
+
+export function isLockFromThisMachine(
+ lock: LockOwnerFields,
+ local: { readonly machineId?: string; readonly host?: string } = {},
+): boolean {
+ if (typeof lock.machineId === 'string') {
+ return lock.machineId === (local.machineId ?? getMachineId());
+ }
+ return typeof lock.hostname === 'string' && lock.hostname === (local.host ?? hostname());
+}
+
+const INCOMPARABLE: LockHolderComparison = {
+ verdict: 'unverified',
+ cause: 'process-start-incomparable',
+};
+const MISMATCH: LockHolderComparison = { verdict: 'unverified', cause: 'process-start-mismatch' };
+const RECORDED_IN_FUTURE: LockHolderComparison = {
+ verdict: 'unverified',
+ cause: 'recorded-start-in-future',
+};
+const WRITER_IDENTITY_UNRECORDED: LockHolderComparison = {
+ verdict: 'unverified',
+ cause: 'writer-identity-unrecorded',
+};
+const STARTED_AFTER_LOCK: LockHolderComparison = {
+ verdict: 'unverified',
+ cause: 'process-started-after-lock',
+};
+
+function compareWithStartedAt(
+ startedAt: string | undefined,
+ live: ProcessStart,
+ refuseHolderStartedAfterLock: boolean,
+): LockHolderComparison {
+ const writtenAtMs = typeof startedAt === 'string' ? Date.parse(startedAt) : Number.NaN;
+ if (!Number.isFinite(writtenAtMs)) return INCOMPARABLE;
+ const margin =
+ live.clock === 'creation' ? LEGACY_MARGIN_CREATION_MS : LEGACY_MARGIN_BOOT_DERIVED_MS;
+ if (writtenAtMs - Date.now() > margin) return RECORDED_IN_FUTURE;
+ const startedAfterLockMs = live.startedAtMs - writtenAtMs;
+ if (startedAfterLockMs > margin) return { verdict: 'replaced' };
+ if (refuseHolderStartedAfterLock && startedAfterLockMs > 0) return STARTED_AFTER_LOCK;
+ return { verdict: 'writer' };
+}
+
+function compareLockHolderStart(
+ lock: LockHolderIdentity,
+ live: ProcessStart | null,
+ refuseHolderStartedAfterLock: boolean,
+): LockHolderComparison {
+ if (live === null) return { verdict: 'unverified', cause: 'process-start-unreadable' };
+ const token = parseProcessStartToken(lock.processStart);
+ if (token === undefined) {
+ return compareWithStartedAt(lock.startedAt, live, refuseHolderStartedAfterLock);
+ }
+ if ('bootId' in token) {
+ if (live.bootId === undefined || live.startTicks === undefined) return INCOMPARABLE;
+ if (token.bootId !== live.bootId) return { verdict: 'replaced' };
+ if (live.startTicks === token.startTicks) return { verdict: 'writer' };
+ return live.startTicks > token.startTicks ? { verdict: 'replaced' } : MISMATCH;
+ }
+ if (live.clock !== 'creation') return INCOMPARABLE;
+ const drift = live.startedAtMs - token.startedAtMs;
+ if (Math.abs(drift) <= TOKEN_TOLERANCE_MS) return { verdict: 'writer' };
+ return drift > 0 ? { verdict: 'replaced' } : MISMATCH;
+}
+
+function recordsProcessStart(lock: LockHolderIdentity): boolean {
+ return parseProcessStartToken(lock.processStart) !== undefined;
+}
+
+export function assessLockHolder(
+ lock: LockHolderIdentity,
+ options: ClassifyLockHolderOptions = {},
+): LockHolderAssessment {
+ const isAlive = options.isAlive ?? isProcessAlive;
+ if (!isAlive(lock.pid)) return { verdict: 'gone' };
+ const sameMachine = options.sameMachine ?? isLockFromThisMachine(lock);
+ if (!sameMachine && !recordsLockOwner(lock))
+ return { verdict: 'unverified', cause: 'unverified-owner' };
+ const read = options.readProcessStart ?? readProcessStart;
+ const start = read(lock.pid);
+ if (start === null && !isAlive(lock.pid)) return { verdict: 'gone' };
+ const comparison = compareLockHolderStart(
+ lock,
+ start,
+ options.refuseHolderStartedAfterLock ?? false,
+ );
+ if (!sameMachine) {
+ return comparison.verdict === 'writer' && recordsProcessStart(lock)
+ ? comparison
+ : { verdict: 'unverified', cause: 'foreign-host' };
+ }
+ if (
+ comparison.verdict === 'writer' &&
+ (options.formatRecordsMachineId ?? true) &&
+ !recordsProcessStart(lock) &&
+ typeof lock.machineId !== 'string'
+ ) {
+ return WRITER_IDENTITY_UNRECORDED;
+ }
+ return comparison;
+}
+
+export function verdictHoldsLock(verdict: LockHolderVerdict): boolean {
+ switch (verdict) {
+ case 'writer':
+ case 'unverified':
+ return true;
+ case 'replaced':
+ case 'gone':
+ return false;
+ default: {
+ const exhaustive: never = verdict;
+ return exhaustive;
+ }
+ }
+}
+
+export function classifyLockHolder(
+ lock: LockHolderIdentity,
+ options: ClassifyLockHolderOptions = {},
+): LockHolderVerdict {
+ return assessLockHolder(lock, options).verdict;
+}
diff --git a/packages/server/src/process-lock-acquire-fs.test.ts b/packages/server/src/process-lock-acquire-fs.test.ts
new file mode 100644
index 000000000..92b75c750
--- /dev/null
+++ b/packages/server/src/process-lock-acquire-fs.test.ts
@@ -0,0 +1,326 @@
+import { mkdirSync, readFileSync, symlinkSync, unlinkSync, writeFileSync } from 'node:fs';
+import { hostname } from 'node:os';
+import { resolve } from 'node:path';
+import { LOCAL_DIR } from '@inkeep/open-knowledge-core';
+import { afterEach, beforeEach, describe, expect, test, vi } from 'vitest';
+import {
+ legacyServerLock,
+ lockWrittenByRunningProcess,
+ type OwnedProcess,
+ startOwnedProcess,
+ stopOwnedProcess,
+} from '../../../test-support/owned-process.test-helper.ts';
+import { createTempDirFactory } from '../../../test-support/temp-dir.test-helper.ts';
+import { getLogger } from './logger.ts';
+import { getMachineId } from './machine-id.ts';
+import {
+ acquireProcessLock,
+ type LockName,
+ lockFilePath,
+ ProcessLockCollisionError,
+ type ProcessLockMetadata,
+} from './process-lock.ts';
+
+vi.mock('node:fs', async (importOriginal) => {
+ const actual = await importOriginal<typeof import('node:fs')>();
+ return {
+ ...actual,
+ readFileSync: vi.fn(actual.readFileSync),
+ unlinkSync: vi.fn(actual.unlinkSync),
+ };
+});
+
+const actualFs = await vi.importActual<typeof import('node:fs')>('node:fs');
+
+const LOCK_NAME: LockName = 'server';
+const tempDir = createTempDirFactory(afterEach);
+const owned: OwnedProcess[] = [];
+const hostPlatform = Object.getOwnPropertyDescriptor(process, 'platform');
+
+let lockDir: string;
+let lockPath: string;
+
+beforeEach(() => {
+ lockDir = resolve(tempDir('ok-process-lock-acquire-fs-'), '.ok', LOCAL_DIR);
+ lockPath = lockFilePath(lockDir, LOCK_NAME);
+ mkdirSync(lockDir, { recursive: true });
+});
+
+afterEach(async () => {
+ if (hostPlatform) Object.defineProperty(process, 'platform', hostPlatform);
+ vi.mocked(readFileSync).mockImplementation(actualFs.readFileSync);
+ vi.mocked(unlinkSync).mockImplementation(actualFs.unlinkSync);
+ for (const holder of owned.splice(0)) await stopOwnedProcess(holder);
+});
+
+function runAsPlatform(platform: NodeJS.Platform): void {
+ Object.defineProperty(process, 'platform', { value: platform, configurable: true });
+}
+
+function errnoError(code: string, syscall: string): NodeJS.ErrnoException {
+ return Object.assign(new Error(`${code}: simulated failure, ${syscall} '${lockPath}'`), {
+ code,
+ syscall,
+ });
+}
+
+function lockUnlinkFails(code: string, { times }: { times: number }): void {
+ let failures = 0;
+ vi.mocked(unlinkSync).mockImplementation((path) => {
+ if (path === lockPath && failures < times) {
+ failures += 1;
+ throw errnoError(code, 'unlink');
+ }
+ actualFs.unlinkSync(path);
+ });
+}
+
+function lockReadFailsOnce(code: string): void {
+ let failed = false;
+ vi.mocked(readFileSync).mockImplementation(((
+ path: Parameters<typeof readFileSync>[0],
+ options,
+ ) => {
+ if (path === lockPath && !failed) {
+ failed = true;
+ throw errnoError(code, 'open');
+ }
+ return actualFs.readFileSync(path, options);
+ }) as typeof readFileSync);
+}
+
+function writeExitedHolderLock(): string {
+ const text = JSON.stringify({
+ pid: 99_990_201,
+ hostname: hostname(),
+ machineId: getMachineId(),
+ port: 1234,
+ startedAt: new Date().toISOString(),
+ worktreeRoot: '/exited',
+ });
+ writeFileSync(lockPath, text);
+ return text;
+}
+
+async function liveHolderLockText(): Promise<string> {
+ const holder = await startOwnedProcess();
+ owned.push(holder);
+ return JSON.stringify(
+ legacyServerLock({
+ pid: holder.pid,
+ startedAt: lockWrittenByRunningProcess(holder),
+ machineId: getMachineId(),
+ worktreeRoot: '/live',
+ }),
+ );
+}
+
+function acquire() {
+ return acquireProcessLock({
+ lockName: LOCK_NAME,
+ lockDir,
+ metadata: { port: 0, worktreeRoot: '/new/boot' },
+ });
+}
+
+function lockOnDisk(): ProcessLockMetadata {
+ return JSON.parse(actualFs.readFileSync(lockPath, 'utf-8'));
+}
+
+describe('a stale lock whose removal fails', () => {
+ test.each(['EBUSY', 'EPERM', 'EACCES'])(
+ 'on Windows a transient %s is retried and the lock is replaced',
+ (code) => {
+ writeExitedHolderLock();
+ lockUnlinkFails(code, { times: 1 });
+ runAsPlatform('win32');
+
+ acquire();
+
+ expect(lockOnDisk().pid).toBe(process.pid);
+ },
+ );
+
+ test('on Windows a removal that keeps failing reports the last removal error', () => {
+ const text = writeExitedHolderLock();
+ lockUnlinkFails('EBUSY', { times: Number.POSITIVE_INFINITY });
+ runAsPlatform('win32');
+
+ expect(acquire).toThrow(
+ `[server-lock] Could not remove the stale lock at ${lockPath}: EBUSY: simulated failure, unlink '${lockPath}'. Another program has the lock file open; close it or wait, then retry.`,
+ );
+ expect(actualFs.readFileSync(lockPath, 'utf-8')).toBe(text);
+ });
+
+ test('on Windows a sharing violation that clears within 50 ms no longer fails the start', () => {
+ writeExitedHolderLock();
+ let firstFailureAt: number | undefined;
+ vi.mocked(unlinkSync).mockImplementation((path) => {
+ if (path === lockPath) {
+ firstFailureAt ??= Date.now();
+ if (Date.now() - firstFailureAt < 50) throw errnoError('EBUSY', 'unlink');
+ }
+ actualFs.unlinkSync(path);
+ });
+ runAsPlatform('win32');
+
+ acquire();
+
+ expect(lockOnDisk().pid).toBe(process.pid);
+ });
+
+ test('on Windows a removal that succeeds on the last attempt is followed by one more create', () => {
+ writeExitedHolderLock();
+ lockUnlinkFails('EBUSY', { times: 2 });
+ runAsPlatform('win32');
+
+ acquire();
+
+ expect(lockOnDisk().pid).toBe(process.pid);
+ });
+
+ test('on Windows each retried failure is logged with its errno and attempt', () => {
+ writeExitedHolderLock();
+ lockUnlinkFails('EPERM', { times: 2 });
+ runAsPlatform('win32');
+ const warn = vi.spyOn(getLogger('process-lock'), 'warn');
+
+ try {
+ acquire();
+
+ const retries = warn.mock.calls.filter(([fields]) => 'attempt' in Object(fields));
+ expect(retries).toEqual([
+ [
+ { lockPath, code: 'EPERM', attempt: 1 },
+ `[server-lock] Could not remove the stale lock at ${lockPath} (EPERM); retrying, attempt 1 of 3`,
+ ],
+ [
+ { lockPath, code: 'EPERM', attempt: 2 },
+ `[server-lock] Could not remove the stale lock at ${lockPath} (EPERM); retrying, attempt 2 of 3`,
+ ],
+ ]);
+ } finally {
+ warn.mockRestore();
+ }
+ });
+
+ test.each([
+ ['win32', 'EROFS', 'Restore file and parent-directory access, then retry.'],
+ ['darwin', 'EBUSY', 'Another program has the lock file open; close it or wait, then retry.'],
+ ['linux', 'EPERM', 'Restore file and parent-directory access, then retry.'],
+ ['linux', 'EISDIR', 'Resolve that error, then retry.'],
+ ] as const)('on %s a %s failure aborts without a retry', (platform, code, remedy) => {
+ const text = writeExitedHolderLock();
+ lockUnlinkFails(code, { times: 1 });
+ runAsPlatform(platform);
+
+ expect(acquire).toThrow(
+ `[server-lock] Could not remove the stale lock at ${lockPath}: ${code}: simulated failure, unlink '${lockPath}'. ${remedy}`,
+ );
+ expect(actualFs.readFileSync(lockPath, 'utf-8')).toBe(text);
+ });
+});
+
+describe('a lock acquire cannot read', () => {
+ test('on Windows a transient read failure is retried and a live holder still collides', async () => {
+ const text = await liveHolderLockText();
+ writeFileSync(lockPath, text);
+ lockReadFailsOnce('EPERM');
+ runAsPlatform('win32');
+
+ expect(acquire).toThrow(ProcessLockCollisionError);
+ expect(actualFs.readFileSync(lockPath, 'utf-8')).toBe(text);
+ });
+});
+
+describe('a lock path that holds no readable file', () => {
+ test.runIf(process.platform !== 'win32')(
+ 'a dangling symbolic link is removed and replaced without creating its target',
+ () => {
+ const target = resolve(lockDir, 'moved-away.lock');
+ symlinkSync(target, lockPath);
+
+ acquire();
+
+ expect(actualFs.lstatSync(lockPath).isSymbolicLink()).toBe(false);
+ expect(lockOnDisk().pid).toBe(process.pid);
+ expect(actualFs.existsSync(target)).toBe(false);
+ },
+ );
+
+ test.runIf(process.platform !== 'win32')(
+ 'a dangling symbolic link that cannot be removed fails naming the link and is kept',
+ () => {
+ const target = resolve(lockDir, 'moved-away.lock');
+ symlinkSync(target, lockPath);
+ lockUnlinkFails('EACCES', { times: Number.POSITIVE_INFINITY });
+ runAsPlatform('darwin');
+
+ expect(acquire).toThrow(
+ `[server-lock] Could not remove the broken symbolic link at ${lockPath}: EACCES: simulated failure, unlink '${lockPath}'. Restore file and parent-directory access, then retry.`,
+ );
+ expect(actualFs.lstatSync(lockPath).isSymbolicLink()).toBe(true);
+ expect(actualFs.existsSync(target)).toBe(false);
+ },
+ );
+
+ test('a lock that disappears before it is read is retried as a race', () => {
+ writeExitedHolderLock();
+ let vanished = false;
+ vi.mocked(readFileSync).mockImplementation(((
+ path: Parameters<typeof readFileSync>[0],
+ options,
+ ) => {
+ if (path === lockPath && !vanished) {
+ vanished = true;
+ actualFs.unlinkSync(lockPath);
+ }
+ return actualFs.readFileSync(path, options);
+ }) as typeof readFileSync);
+
+ acquire();
+
+ expect(lockOnDisk().pid).toBe(process.pid);
+ });
+
+ test('a lock a successor writes after a read found nothing is kept and collides', async () => {
+ const successorText = await liveHolderLockText();
+ writeExitedHolderLock();
+ let vanished = false;
+ vi.mocked(readFileSync).mockImplementation(((
+ path: Parameters<typeof readFileSync>[0],
+ options,
+ ) => {
+ if (path === lockPath && !vanished) {
+ vanished = true;
+ actualFs.unlinkSync(lockPath);
+ writeFileSync(lockPath, successorText);
+ throw errnoError('ENOENT', 'open');
+ }
+ return actualFs.readFileSync(path, options);
+ }) as typeof readFileSync);
+
+ expect(acquire).toThrow(ProcessLockCollisionError);
+ expect(actualFs.readFileSync(lockPath, 'utf-8')).toBe(successorText);
+ });
+});
+
+test('a corrupt lock replaced by a live server before its removal is kept and collides', async () => {
+ const successorText = await liveHolderLockText();
+ writeFileSync(lockPath, '{ not json');
+ let replaced = false;
+ vi.mocked(readFileSync).mockImplementation(((
+ path: Parameters<typeof readFileSync>[0],
+ options,
+ ) => {
+ const text = actualFs.readFileSync(path, options);
+ if (path === lockPath && !replaced) {
+ replaced = true;
+ writeFileSync(lockPath, successorText);
+ }
+ return text;
+ }) as typeof readFileSync);
+
+ expect(acquire).toThrow(ProcessLockCollisionError);
+ expect(actualFs.readFileSync(lockPath, 'utf-8')).toBe(successorText);
+});
diff --git a/packages/server/src/process-lock-holder-identity.test.ts b/packages/server/src/process-lock-holder-identity.test.ts
new file mode 100644
index 000000000..b28f1ab0f
--- /dev/null
+++ b/packages/server/src/process-lock-holder-identity.test.ts
@@ -0,0 +1,215 @@
+import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs';
+import { hostname } from 'node:os';
+import { resolve } from 'node:path';
+import { LOCAL_DIR } from '@inkeep/open-knowledge-core';
+import { afterEach, beforeEach, describe, expect, test, vi } from 'vitest';
+import {
+ processStartTokenFromOs,
+ processStartTokenOffsetFromOs,
+} from '../../../test-support/os-process-start.test-helper.ts';
+import {
+ legacyServerLock,
+ lockWrittenBeforeProcessStarted,
+ lockWrittenByRunningProcess,
+ type OwnedProcess,
+ ownedProcessHasExited,
+ startOwnedProcess,
+ stopOwnedProcess,
+} from '../../../test-support/owned-process.test-helper.ts';
+import { createTempDirFactory } from '../../../test-support/temp-dir.test-helper.ts';
+import { getLogger } from './logger.ts';
+import { getMachineId } from './machine-id.ts';
+import {
+ acquireProcessLock,
+ type LockName,
+ lockFilePath,
+ ProcessLockCollisionError,
+ type ProcessLockHandle,
+ type ProcessLockMetadata,
+ readProcessLock,
+ readProcessLockDetailed,
+} from './process-lock.ts';
+
+const LOCK_NAME: LockName = 'server';
+const tempDir = createTempDirFactory(afterEach);
+const owned: OwnedProcess[] = [];
+
+let lockDir: string;
+let lockPath: string;
+
+beforeEach(() => {
+ lockDir = resolve(tempDir('ok-process-lock-holder-'), '.ok', LOCAL_DIR);
+ lockPath = lockFilePath(lockDir, LOCK_NAME);
+});
+
+afterEach(async () => {
+ for (const holder of owned.splice(0)) await stopOwnedProcess(holder);
+});
+
+async function startHolder(): Promise<OwnedProcess> {
+ const holder = await startOwnedProcess();
+ owned.push(holder);
+ return holder;
+}
+
+function writeLock(lock: Record<string, unknown>): void {
+ mkdirSync(lockDir, { recursive: true });
+ writeFileSync(lockPath, JSON.stringify(lock), 'utf-8');
+}
+
+function lockNaming(
+ holder: OwnedProcess,
+ overrides: Record<string, unknown> = {},
+): Record<string, unknown> {
+ return {
+ ...legacyServerLock({
+ pid: holder.pid,
+ startedAt: lockWrittenBeforeProcessStarted(holder),
+ machineId: getMachineId(),
+ worktreeRoot: '/previous/boot',
+ }),
+ ...overrides,
+ };
+}
+
+function acquireForNewServer(): ProcessLockHandle {
+ return acquireProcessLock({
+ lockName: LOCK_NAME,
+ lockDir,
+ metadata: { port: 0, worktreeRoot: '/new/boot' },
+ });
+}
+
+describe.skipIf(process.platform === 'win32')(
+ 'a live lock whose recorded process start carries a field this release does not know',
+ () => {
+ test('readProcessLock still matches the holder and returns the token without that field', async () => {
+ const holder = await startHolder();
+ const processStart = processStartTokenFromOs(holder.pid);
+ writeLock(lockNaming(holder, { processStart: { ...processStart, namespace: 'pid:[1]' } }));
+
+ expect(readProcessLock({ lockName: LOCK_NAME, lockDir })?.processStart).toEqual(processStart);
+ });
+
+ test('readProcessLock drops a process start it cannot read', async () => {
+ const holder = await startHolder();
+ writeLock(
+ lockNaming(holder, {
+ startedAt: lockWrittenByRunningProcess(holder),
+ processStart: { startedAtMs: 'soon' },
+ }),
+ );
+
+ const lock = readProcessLock({ lockName: LOCK_NAME, lockDir });
+ expect(lock?.pid).toBe(holder.pid);
+ expect(lock).not.toHaveProperty('processStart');
+ });
+ },
+);
+
+describe.skipIf(process.platform === 'win32')(
+ 'a lock whose recorded process start matches its holder, though its startedAt is 36 h older',
+ () => {
+ test('acquireProcessLock still collides', async () => {
+ const holder = await startHolder();
+ writeLock(lockNaming(holder, { processStart: processStartTokenFromOs(holder.pid) }));
+
+ expect(acquireForNewServer).toThrow(ProcessLockCollisionError);
+ const md: ProcessLockMetadata = JSON.parse(readFileSync(lockPath, 'utf-8'));
+ expect(md.pid).toBe(holder.pid);
+ });
+
+ test('readProcessLock returns the live lock', async () => {
+ const holder = await startHolder();
+ writeLock(lockNaming(holder, { processStart: processStartTokenFromOs(holder.pid) }));
+
+ expect(readProcessLock({ lockName: LOCK_NAME, lockDir })).toMatchObject({ pid: holder.pid });
+ });
+
+ test('readProcessLockDetailed reports the lock live, carrying its process start', async () => {
+ const holder = await startHolder();
+ const processStart = processStartTokenFromOs(holder.pid);
+ writeLock(lockNaming(holder, { processStart }));
+
+ expect(readProcessLockDetailed({ lockName: LOCK_NAME, lockDir })).toMatchObject({
+ status: 'live',
+ lock: { pid: holder.pid, processStart },
+ });
+ });
+ },
+);
+
+describe.skipIf(process.platform === 'win32')(
+ 'a lock whose holder started after it was written',
+ () => {
+ test('readProcessLock reports no live server and leaves the lock on disk', async () => {
+ const holder = await startHolder();
+ writeLock(lockNaming(holder));
+
+ expect(readProcessLock({ lockName: LOCK_NAME, lockDir })).toBeNull();
+ expect(existsSync(lockPath)).toBe(true);
+ });
+
+ test('readProcessLockDetailed reports the lock stale and leaves it on disk', async () => {
+ const holder = await startHolder();
+ writeLock(lockNaming(holder));
+
+ expect(readProcessLockDetailed({ lockName: LOCK_NAME, lockDir })).toMatchObject({
+ status: 'stale',
+ });
+ expect(existsSync(lockPath)).toBe(true);
+ });
+
+ test('acquireProcessLock still collides when the lock was written on another machine', async () => {
+ const holder = await startHolder();
+ writeLock(
+ lockNaming(holder, {
+ machineId: 'some-other-machine-id',
+ hostname: `${hostname()}-elsewhere`,
+ }),
+ );
+ const warn = vi.spyOn(getLogger('process-lock'), 'warn');
+
+ try {
+ expect(acquireForNewServer).toThrow(ProcessLockCollisionError);
+ expect(warn).toHaveBeenCalledWith(
+ expect.objectContaining({
+ lockPid: holder.pid,
+ holder: 'unverified',
+ cause: 'foreign-host',
+ }),
+ expect.stringContaining(`Could not confirm that pid=${holder.pid}`),
+ );
+ } finally {
+ warn.mockRestore();
+ }
+ const md: ProcessLockMetadata = JSON.parse(readFileSync(lockPath, 'utf-8'));
+ expect(md.pid).toBe(holder.pid);
+ expect(ownedProcessHasExited(holder)).toBe(false);
+ });
+ },
+);
+
+describe.skipIf(process.platform === 'win32')(
+ 'a lock whose recorded process start is later than its holder started',
+ () => {
+ test('readProcessLock still returns the live lock', async () => {
+ const holder = await startHolder();
+ writeLock(lockNaming(holder, { processStart: processStartTokenOffsetFromOs(holder.pid, 5) }));
+
+ expect(readProcessLock({ lockName: LOCK_NAME, lockDir })).toMatchObject({
+ pid: holder.pid,
+ });
+ });
+
+ test('readProcessLockDetailed reports the lock live', async () => {
+ const holder = await startHolder();
+ writeLock(lockNaming(holder, { processStart: processStartTokenOffsetFromOs(holder.pid, 5) }));
+
+ expect(readProcessLockDetailed({ lockName: LOCK_NAME, lockDir })).toMatchObject({
+ status: 'live',
+ lock: { pid: holder.pid },
+ });
+ });
+ },
+);
diff --git a/packages/server/src/process-lock-read-cache.test.ts b/packages/server/src/process-lock-read-cache.test.ts
new file mode 100644
index 000000000..aa26fdf1d
--- /dev/null
+++ b/packages/server/src/process-lock-read-cache.test.ts
@@ -0,0 +1,88 @@
+import { mkdirSync, writeFileSync } from 'node:fs';
+import { resolve } from 'node:path';
+import { LOCAL_DIR } from '@inkeep/open-knowledge-core';
+import { afterEach, beforeEach, describe, expect, test, vi } from 'vitest';
+import { legacyServerLock } from '../../../test-support/owned-process.test-helper.ts';
+import { createTempDirFactory } from '../../../test-support/temp-dir.test-helper.ts';
+import { getMachineId } from './machine-id.ts';
+import type { ProcessStart } from './process-identity.ts';
+import {
+ type LockName,
+ lockFilePath,
+ readProcessLock,
+ readProcessLockDetailed,
+} from './process-lock.ts';
+
+const startReads = vi.hoisted(() => ({
+ pids: [] as number[],
+ nowMs: 0,
+ holderStart: null as ProcessStart | null,
+}));
+
+vi.mock('./process-identity.ts', async (importOriginal) => {
+ const actual = await importOriginal<typeof import('./process-identity.ts')>();
+ return {
+ ...actual,
+ createRecentProcessStartReader: (
+ options: Parameters<typeof actual.createRecentProcessStartReader>[0],
+ ) =>
+ actual.createRecentProcessStartReader({
+ ...options,
+ now: () => startReads.nowMs,
+ read: (pid) => {
+ startReads.pids.push(pid);
+ return startReads.holderStart;
+ },
+ }),
+ };
+});
+
+const LOCK_NAME: LockName = 'server';
+const LOCK_READ_MAX_AGE_MS = 1_000;
+const tempDir = createTempDirFactory(afterEach);
+
+let lockDir: string;
+
+beforeEach(() => {
+ startReads.pids = [];
+ startReads.nowMs += 60_000;
+ const startedAt = new Date();
+ startReads.holderStart = { startedAtMs: startedAt.getTime(), clock: 'creation' };
+ lockDir = resolve(tempDir('ok-process-lock-read-cache-'), '.ok', LOCAL_DIR);
+ mkdirSync(lockDir, { recursive: true });
+ writeFileSync(
+ lockFilePath(lockDir, LOCK_NAME),
+ JSON.stringify(
+ legacyServerLock({
+ pid: process.pid,
+ startedAt: startedAt.toISOString(),
+ machineId: getMachineId(),
+ worktreeRoot: lockDir,
+ }),
+ ),
+ 'utf-8',
+ );
+});
+
+describe('reading a live server lock repeatedly', () => {
+ test('readProcessLock reads the holder start once inside the maximum age', () => {
+ expect(readProcessLock({ lockName: LOCK_NAME, lockDir })?.pid).toBe(process.pid);
+ startReads.nowMs += LOCK_READ_MAX_AGE_MS - 1;
+ expect(readProcessLock({ lockName: LOCK_NAME, lockDir })?.pid).toBe(process.pid);
+ expect(startReads.pids).toEqual([process.pid]);
+ });
+
+ test('readProcessLockDetailed reads the holder start once inside the maximum age', () => {
+ expect(readProcessLockDetailed({ lockName: LOCK_NAME, lockDir }).status).toBe('live');
+ startReads.nowMs += LOCK_READ_MAX_AGE_MS - 1;
+ expect(readProcessLockDetailed({ lockName: LOCK_NAME, lockDir }).status).toBe('live');
+ expect(startReads.pids).toEqual([process.pid]);
+ });
+
+ test('reads the holder start again once the reading reaches the maximum age', () => {
+ readProcessLock({ lockName: LOCK_NAME, lockDir });
+ startReads.nowMs += LOCK_READ_MAX_AGE_MS;
+ readProcessLockDetailed({ lockName: LOCK_NAME, lockDir });
+ expect(startReads.pids).toEqual([process.pid, process.pid]);
+ });
+});
diff --git a/packages/server/src/process-lock-successor-race.test.ts b/packages/server/src/process-lock-successor-race.test.ts
new file mode 100644
index 000000000..7c6ac98cb
--- /dev/null
+++ b/packages/server/src/process-lock-successor-race.test.ts
@@ -0,0 +1,172 @@
+import { mkdirSync, readFileSync, writeFileSync } from 'node:fs';
+import { resolve } from 'node:path';
+import { LOCAL_DIR } from '@inkeep/open-knowledge-core';
+import { afterEach, beforeEach, expect, test, vi } from 'vitest';
+import {
+ legacyServerLock,
+ lockWrittenByRunningProcess,
+ type OwnedProcess,
+ startOwnedProcess,
+ stopOwnedProcess,
+} from '../../../test-support/owned-process.test-helper.ts';
+import { createTempDirFactory } from '../../../test-support/temp-dir.test-helper.ts';
+import { getMachineId } from './machine-id.ts';
+import { isProcessAlive } from './process-alive.ts';
+import {
+ acquireProcessLock,
+ type LockName,
+ lockFilePath,
+ ProcessLockCollisionError,
+ readProcessLock,
+ readProcessLockDetailed,
+} from './process-lock.ts';
+
+vi.mock('./process-alive.ts', async (importOriginal) => {
+ const actual = await importOriginal<typeof import('./process-alive.ts')>();
+ return { ...actual, isProcessAlive: vi.fn(actual.isProcessAlive) };
+});
+
+const { isProcessAlive: actualIsProcessAlive } =
+ await vi.importActual<typeof import('./process-alive.ts')>('./process-alive.ts');
+
+const LOCK_NAME: LockName = 'server';
+const tempDir = createTempDirFactory(afterEach);
+const owned: OwnedProcess[] = [];
+
+let lockDir: string;
+let lockPath: string;
+
+beforeEach(() => {
+ lockDir = resolve(tempDir('ok-process-lock-successor-'), '.ok', LOCAL_DIR);
+ lockPath = lockFilePath(lockDir, LOCK_NAME);
+ mkdirSync(lockDir, { recursive: true });
+});
+
+afterEach(async () => {
+ vi.mocked(isProcessAlive).mockImplementation(actualIsProcessAlive);
+ for (const holder of owned.splice(0)) await stopOwnedProcess(holder);
+});
+
+async function writeSuccessorLockText(): Promise<string> {
+ const successor = await startOwnedProcess();
+ owned.push(successor);
+ return JSON.stringify(
+ legacyServerLock({
+ pid: successor.pid,
+ startedAt: lockWrittenByRunningProcess(successor),
+ machineId: getMachineId(),
+ worktreeRoot: '/successor',
+ }),
+ );
+}
+
+function writePredecessorLock(pid: number): void {
+ writeFileSync(
+ lockPath,
+ JSON.stringify(
+ legacyServerLock({
+ pid,
+ startedAt: new Date().toISOString(),
+ machineId: getMachineId(),
+ worktreeRoot: '/predecessor',
+ }),
+ ),
+ );
+}
+
+function samePidSuccessorLockText(pid: number): string {
+ return JSON.stringify(
+ legacyServerLock({
+ pid,
+ startedAt: '2026-01-01T00:00:00.000Z',
+ machineId: getMachineId(),
+ worktreeRoot: '/successor',
+ }),
+ );
+}
+
+function predecessorExitsWhileItsStartIsRead(
+ pid: number,
+ successorLockText: string,
+ { successorRunsUnderSamePid = false } = {},
+): void {
+ let checks = 0;
+ vi.mocked(isProcessAlive).mockImplementation((candidate) => {
+ if (candidate !== pid) return actualIsProcessAlive(candidate);
+ checks += 1;
+ if (checks === 1) return true;
+ if (checks === 2) {
+ writeFileSync(lockPath, successorLockText);
+ return false;
+ }
+ return successorRunsUnderSamePid;
+ });
+}
+
+test('readProcessLock keeps a lock a successor wrote while the exited holder was assessed', async () => {
+ const successorLockText = await writeSuccessorLockText();
+ writePredecessorLock(99_990_101);
+ predecessorExitsWhileItsStartIsRead(99_990_101, successorLockText);
+
+ expect(readProcessLock({ lockName: LOCK_NAME, lockDir })).toBeNull();
+ expect(readFileSync(lockPath, 'utf-8')).toBe(successorLockText);
+});
+
+test('readProcessLockDetailed keeps a lock a successor wrote while the exited holder was assessed', async () => {
+ const successorLockText = await writeSuccessorLockText();
+ writePredecessorLock(99_990_102);
+ predecessorExitsWhileItsStartIsRead(99_990_102, successorLockText);
+
+ expect(readProcessLockDetailed({ lockName: LOCK_NAME, lockDir }).status).toBe('stale');
+ expect(readFileSync(lockPath, 'utf-8')).toBe(successorLockText);
+});
+
+test('acquireProcessLock collides with a successor that took the lock while the exited holder was assessed', async () => {
+ const successorLockText = await writeSuccessorLockText();
+ writePredecessorLock(99_990_103);
+ predecessorExitsWhileItsStartIsRead(99_990_103, successorLockText);
+
+ expect(() =>
+ acquireProcessLock({
+ lockName: LOCK_NAME,
+ lockDir,
+ metadata: { port: 0, worktreeRoot: '/new/boot' },
+ }),
+ ).toThrow(ProcessLockCollisionError);
+ expect(readFileSync(lockPath, 'utf-8')).toBe(successorLockText);
+});
+
+test('readProcessLock keeps a lock a successor under the same pid wrote while the exited holder was assessed', () => {
+ const successorLockText = samePidSuccessorLockText(99_990_104);
+ writePredecessorLock(99_990_104);
+ predecessorExitsWhileItsStartIsRead(99_990_104, successorLockText);
+
+ expect(readProcessLock({ lockName: LOCK_NAME, lockDir })).toBeNull();
+ expect(readFileSync(lockPath, 'utf-8')).toBe(successorLockText);
+});
+
+test('readProcessLockDetailed keeps a lock a successor under the same pid wrote while the exited holder was assessed', () => {
+ const successorLockText = samePidSuccessorLockText(99_990_105);
+ writePredecessorLock(99_990_105);
+ predecessorExitsWhileItsStartIsRead(99_990_105, successorLockText);
+
+ expect(readProcessLockDetailed({ lockName: LOCK_NAME, lockDir }).status).toBe('stale');
+ expect(readFileSync(lockPath, 'utf-8')).toBe(successorLockText);
+});
+
+test('acquireProcessLock collides with a successor under the same pid that took the lock while the exited holder was assessed', () => {
+ const successorLockText = samePidSuccessorLockText(99_990_106);
+ writePredecessorLock(99_990_106);
+ predecessorExitsWhileItsStartIsRead(99_990_106, successorLockText, {
+ successorRunsUnderSamePid: true,
+ });
+
+ expect(() =>
+ acquireProcessLock({
+ lockName: LOCK_NAME,
+ lockDir,
+ metadata: { port: 0, worktreeRoot: '/new/boot' },
+ }),
+ ).toThrow(ProcessLockCollisionError);
+ expect(readFileSync(lockPath, 'utf-8')).toBe(successorLockText);
+});
diff --git a/packages/server/src/process-lock.test.ts b/packages/server/src/process-lock.test.ts
index 8eec4e0bd..756d4f674 100644
--- a/packages/server/src/process-lock.test.ts
+++ b/packages/server/src/process-lock.test.ts
@@ -1,9 +1,20 @@
-import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs';
+import { chmodSync, existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs';
import { mkdtemp, rm } from 'node:fs/promises';
import { hostname, tmpdir } from 'node:os';
import { resolve } from 'node:path';
import { LOCAL_DIR } from '@inkeep/open-knowledge-core';
import { afterEach, beforeEach, describe, expect, test, vi } from 'vitest';
+import { processStartTokenFromOs } from '../../../test-support/os-process-start.test-helper.ts';
+import {
+ legacyServerLock,
+ lockWrittenBeforeProcessStarted,
+ lockWrittenByRunningProcess,
+ type OwnedProcess,
+ ownedProcessHasExited,
+ startOwnedProcess,
+ stopOwnedProcess,
+} from '../../../test-support/owned-process.test-helper.ts';
+import { getLogger } from './logger';
import { getMachineId } from './machine-id';
import {
acquireProcessLock,
@@ -12,6 +23,7 @@ import {
lockFilePath,
markProcessLockDraining,
ProcessLockCollisionError,
+ type ProcessLockHandle,
type ProcessLockMetadata,
readProcessLock,
readProcessLockDetailed,
@@ -118,6 +130,32 @@ describe('acquireProcessLock', () => {
expect(md.worktreeRoot).toBe('/new');
});
+ test('records the replaced holder as lockPid, apart from the logger pid binding', () => {
+ const stale: ProcessLockMetadata = {
+ pid: 99999999,
+ hostname: hostname(),
+ port: 1234,
+ startedAt: new Date().toISOString(),
+ worktreeRoot: '/old',
+ };
+ mkdirSync(lockDir, { recursive: true });
+ writeFileSync(lockPath, JSON.stringify(stale), 'utf-8');
+ const warn = vi.spyOn(getLogger('process-lock'), 'warn');
+ try {
+ acquireProcessLock({
+ lockName: LOCK_NAME,
+ lockDir,
+ metadata: { port: 3000, worktreeRoot: '/new' },
+ });
+ expect(warn).toHaveBeenCalledWith(
+ { lockPid: 99999999, host: hostname(), holder: 'gone' },
+ expect.stringContaining('Stale lock detected (pid=99999999'),
+ );
+ } finally {
+ warn.mockRestore();
+ }
+ });
+
test('throws ProcessLockCollisionError when lock owner is alive', () => {
acquireProcessLock({
lockName: LOCK_NAME,
@@ -294,6 +332,119 @@ describe('acquireProcessLock', () => {
});
});
+describe('a lock that cannot be removed from a read-only lock directory', () => {
+ const canRestrictWrites = process.platform !== 'win32' && process.getuid?.() !== 0;
+
+ function writeLockInReadOnlyDir(text: string): void {
+ mkdirSync(lockDir, { recursive: true });
+ writeFileSync(lockPath, text, 'utf-8');
+ chmodSync(lockDir, 0o500);
+ }
+
+ afterEach(() => {
+ if (existsSync(lockDir)) chmodSync(lockDir, 0o700);
+ });
+
+ const exitedHolderLock = JSON.stringify({
+ pid: 99999999,
+ hostname: hostname(),
+ machineId: getMachineId(),
+ port: 1234,
+ startedAt: new Date().toISOString(),
+ worktreeRoot: '/old',
+ });
+
+ test.runIf(canRestrictWrites)(
+ 'acquireProcessLock names the stale lock and the unlink error instead of reporting contention',
+ () => {
+ writeLockInReadOnlyDir(exitedHolderLock);
+
+ expect(() =>
+ acquireProcessLock({
+ lockName: LOCK_NAME,
+ lockDir,
+ metadata: { port: 0, worktreeRoot: '/new' },
+ }),
+ ).toThrow(
+ `[server-lock] Could not remove the stale lock at ${lockPath}: EACCES: permission denied, unlink '${lockPath}'. Restore file and parent-directory access, then retry.`,
+ );
+ expect(readFileSync(lockPath, 'utf-8')).toBe(exitedHolderLock);
+ },
+ );
+
+ test.runIf(canRestrictWrites)(
+ 'acquireProcessLock names the corrupt lock and the unlink error instead of reporting contention',
+ () => {
+ writeLockInReadOnlyDir('{ not json');
+
+ expect(() =>
+ acquireProcessLock({
+ lockName: LOCK_NAME,
+ lockDir,
+ metadata: { port: 0, worktreeRoot: '/new' },
+ }),
+ ).toThrow(
+ `[server-lock] Could not remove the corrupt lock at ${lockPath}: EACCES: permission denied, unlink '${lockPath}'. Restore file and parent-directory access, then retry.`,
+ );
+ expect(readFileSync(lockPath, 'utf-8')).toBe('{ not json');
+ },
+ );
+
+ test.runIf(canRestrictWrites)('readProcessLock still reports no live server', () => {
+ writeLockInReadOnlyDir(exitedHolderLock);
+
+ expect(readProcessLock({ lockName: LOCK_NAME, lockDir })).toBeNull();
+ expect(readFileSync(lockPath, 'utf-8')).toBe(exitedHolderLock);
+ });
+
+ test.runIf(canRestrictWrites)('readProcessLockDetailed still reports the lock stale', () => {
+ writeLockInReadOnlyDir(exitedHolderLock);
+
+ expect(readProcessLockDetailed({ lockName: LOCK_NAME, lockDir })).toMatchObject({
+ status: 'stale',
+ lock: { pid: 99999999 },
+ });
+ expect(readFileSync(lockPath, 'utf-8')).toBe(exitedHolderLock);
+ });
+});
+
+describe('a lock acquire cannot read', () => {
+ const canRestrictReads = process.platform !== 'win32' && process.getuid?.() !== 0;
+
+ afterEach(() => {
+ if (existsSync(lockPath)) chmodSync(lockPath, 0o600);
+ });
+
+ test.runIf(canRestrictReads)(
+ 'acquireProcessLock keeps an unreadable lock naming a live process and reports the read error',
+ () => {
+ const text = JSON.stringify({
+ pid: aliveForeignPid(),
+ hostname: hostname(),
+ machineId: getMachineId(),
+ port: 1234,
+ startedAt: new Date().toISOString(),
+ worktreeRoot: '/live',
+ });
+ mkdirSync(lockDir, { recursive: true });
+ writeFileSync(lockPath, text, 'utf-8');
+ chmodSync(lockPath, 0o000);
+
+ expect(() =>
+ acquireProcessLock({
+ lockName: LOCK_NAME,
+ lockDir,
+ metadata: { port: 0, worktreeRoot: '/new' },
+ }),
+ ).toThrow(
+ `[server-lock] Could not read the lock at ${lockPath}: EACCES: permission denied, open '${lockPath}'. Restore file and parent-directory access, then retry.`,
+ );
+ chmodSync(lockPath, 0o600);
+ expect(readFileSync(lockPath, 'utf-8')).toBe(text);
+ },
+ );
+});
+
describe('one-URL contract (url field + lockBaseUrl)', () => {
test('acquire omits url by default; updatePort(port, url) writes both in agreement', () => {
const handle = acquireProcessLock({
@@ -1323,3 +1474,173 @@ describe('waitForProcessLockDrain', () => {
expect(outcome).toBe('timeout');
});
});
+
+describe('a lock whose pid is still running', () => {
+ const owned: OwnedProcess[] = [];
+
+ afterEach(async () => {
+ for (const holder of owned.splice(0)) await stopOwnedProcess(holder);
+ });
+
+ async function startHolder(): Promise<OwnedProcess> {
+ const holder = await startOwnedProcess();
+ owned.push(holder);
+ return holder;
+ }
+
+ function writeLegacyLock(holder: OwnedProcess, startedAt: string): void {
+ mkdirSync(lockDir, { recursive: true });
+ writeFileSync(
+ lockPath,
+ JSON.stringify(
+ legacyServerLock({
+ pid: holder.pid,
+ startedAt,
+ machineId: getMachineId(),
+ worktreeRoot: '/previous/boot',
+ }),
+ ),
+ 'utf-8',
+ );
+ }
+
+ function acquireForNewServer(): ProcessLockHandle {
+ return acquireProcessLock({
+ lockName: LOCK_NAME,
+ lockDir,
+ metadata: { port: 0, worktreeRoot: '/new/boot', kind: 'interactive' },
+ });
+ }
+
+ describe('by a process that started after the lock was written (pid reused)', () => {
+ test('acquireProcessLock replaces the lock instead of colliding', async () => {
+ const holder = await startHolder();
+ writeLegacyLock(holder, lockWrittenBeforeProcessStarted(holder));
+
+ expect(acquireForNewServer).not.toThrow();
+ const md: ProcessLockMetadata = JSON.parse(readFileSync(lockPath, 'utf-8'));
+ expect(md.pid).toBe(process.pid);
+ expect(md.worktreeRoot).toBe('/new/boot');
+ expect(ownedProcessHasExited(holder)).toBe(false);
+ });
+
+ test('readProcessLock reports no live server', async () => {
+ const holder = await startHolder();
+ writeLegacyLock(holder, lockWrittenBeforeProcessStarted(holder));
+
+ expect(readProcessLock({ lockName: LOCK_NAME, lockDir })).toBeNull();
+ });
+
+ test('readProcessLockDetailed reports the lock stale', async () => {
+ const holder = await startHolder();
+ writeLegacyLock(holder, lockWrittenBeforeProcessStarted(holder));
+
+ expect(readProcessLockDetailed({ lockName: LOCK_NAME, lockDir })).toMatchObject({
+ status: 'stale',
+ lock: { pid: holder.pid },
+ });
+ });
+ });
+
+ describe('by the process that wrote it', () => {
+ test('acquireProcessLock still collides', async () => {
+ const holder = await startHolder();
+ writeLegacyLock(holder, lockWrittenByRunningProcess(holder));
+
+ expect(acquireForNewServer).toThrow(ProcessLockCollisionError);
+ const md: ProcessLockMetadata = JSON.parse(readFileSync(lockPath, 'utf-8'));
+ expect(md.pid).toBe(holder.pid);
+ });
+
+ test('readProcessLock returns the live lock', async () => {
+ const holder = await startHolder();
+ writeLegacyLock(holder, lockWrittenByRunningProcess(holder));
+
+ expect(readProcessLock({ lockName: LOCK_NAME, lockDir })).toMatchObject({ pid: holder.pid });
+ });
+
+ test('readProcessLockDetailed reports the lock live', async () => {
+ const holder = await startHolder();
+ writeLegacyLock(holder, lockWrittenByRunningProcess(holder));
+
+ expect(readProcessLockDetailed({ lockName: LOCK_NAME, lockDir })).toMatchObject({
+ status: 'live',
+ lock: { pid: holder.pid },
+ });
+ });
+ });
+
+ describe('by a process that does not match the recorded process start', () => {
+ const BOOT_ID_PATH = '/proc/sys/kernel/random/boot_id';
+ const PREVIOUS_BOOT_ID = '00000000-0000-4000-8000-000000000000';
+
+ function processStartOfPreviousWriter(holder: OwnedProcess): Record<string, number | string> {
+ if (process.platform === 'linux') {
+ expect(existsSync(BOOT_ID_PATH)).toBe(true);
+ const bootId = readFileSync(BOOT_ID_PATH, 'utf-8').trim();
+ expect(bootId).toMatch(/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/);
+ expect(bootId).not.toBe(PREVIOUS_BOOT_ID);
+ return { bootId: PREVIOUS_BOOT_ID, startTicks: 4_096 };
+ }
+ return { startedAtMs: Date.parse(lockWrittenBeforeProcessStarted(holder)) };
+ }
+
+ function writeLockRecordingPreviousWriter(holder: OwnedProcess): void {
+ mkdirSync(lockDir, { recursive: true });
+ writeFileSync(
+ lockPath,
+ JSON.stringify({
+ ...legacyServerLock({
+ pid: holder.pid,
+ startedAt: lockWrittenByRunningProcess(holder),
+ machineId: getMachineId(),
+ worktreeRoot: '/previous/boot',
+ }),
+ processStart: processStartOfPreviousWriter(holder),
+ }),
+ 'utf-8',
+ );
+ }
+
+ test('acquireProcessLock replaces the lock instead of colliding', async () => {
+ const holder = await startHolder();
+ writeLockRecordingPreviousWriter(holder);
+
+ expect(acquireForNewServer).not.toThrow();
+ const md: ProcessLockMetadata = JSON.parse(readFileSync(lockPath, 'utf-8'));
+ expect(md.pid).toBe(process.pid);
+ expect(md.worktreeRoot).toBe('/new/boot');
+ expect(ownedProcessHasExited(holder)).toBe(false);
+ });
+
+ test('readProcessLock reports no live server', async () => {
+ const holder = await startHolder();
+ writeLockRecordingPreviousWriter(holder);
+
+ expect(readProcessLock({ lockName: LOCK_NAME, lockDir })).toBeNull();
+ });
+
+ test('readProcessLockDetailed reports the lock stale', async () => {
+ const holder = await startHolder();
+ writeLockRecordingPreviousWriter(holder);
+
+ expect(readProcessLockDetailed({ lockName: LOCK_NAME, lockDir })).toMatchObject({
+ status: 'stale',
+ lock: { pid: holder.pid },
+ });
+ });
+ });
+});
+
+describe('the process start a lock records for its writer', () => {
+ test('acquireProcessLock records the start the operating system holds for the writer', () => {
+ acquireProcessLock({
+ lockName: LOCK_NAME,
+ lockDir,
+ metadata: { port: 0, worktreeRoot: '/wt' },
+ });
+
+ const md: ProcessLockMetadata = JSON.parse(readFileSync(lockPath, 'utf-8'));
+ expect(md.processStart).toEqual(processStartTokenFromOs(process.pid));
+ });
+});
diff --git a/packages/server/src/process-lock.ts b/packages/server/src/process-lock.ts
index 95bf213c4..3415fbf08 100644
--- a/packages/server/src/process-lock.ts
+++ b/packages/server/src/process-lock.ts
@@ -1,6 +1,7 @@
import {
closeSync,
existsSync,
+ lstatSync,
mkdirSync,
openSync,
readFileSync,
@@ -14,7 +15,19 @@ import { resolveDesktopProductName } from '@inkeep/open-knowledge-core';
import { errnoCode } from './http/handler-utils.ts';
import { getLogger } from './logger.ts';
import { getMachineId } from './machine-id.ts';
-import { isProcessAlive, isValidLockPid } from './process-alive.ts';
+import { isValidLockPid } from './process-alive.ts';
+import {
+ assessLockHolder,
+ classifyLockHolder,
+ createRecentProcessStartReader,
+ isLockFromThisMachine,
+ type ProcessStartToken,
+ parseProcessStartToken,
+ processStartToken,
+ readProcessStart,
+ verdictHoldsLock,
+} from './process-identity.ts';
+import { sleepSyncMs } from './sleep-sync.ts';
import { PROTOCOL_VERSION, RUNTIME_VERSION } from './version-constants.ts';
const log = getLogger('process-lock');
@@ -31,6 +44,7 @@ export interface ProcessLockMetadata {
startedAt: string;
worktreeRoot: string;
machineId?: string;
+ processStart?: ProcessStartToken;
draining?: boolean;
kind?: LockKind;
parentPid?: number;
@@ -63,6 +77,40 @@ export class ProcessLockCollisionError extends Error {
}
}
+export const LOCK_FILE_ACCESS_REMEDY = 'Restore file and parent-directory access, then retry.';
+const LOCK_FILE_BUSY_REMEDY =
+ 'Another program has the lock file open; close it or wait, then retry.';
+const LOCK_FILE_OTHER_REMEDY = 'Resolve that error, then retry.';
+const LOCK_FILE_ACCESS_CODES = new Set(['EACCES', 'EPERM', 'EROFS']);
+
+function lockFileRemedy(cause: unknown): string {
+ const code = errnoCode(cause);
+ if (code === 'EBUSY') return LOCK_FILE_BUSY_REMEDY;
+ if (code !== undefined && LOCK_FILE_ACCESS_CODES.has(code)) return LOCK_FILE_ACCESS_REMEDY;
+ return LOCK_FILE_OTHER_REMEDY;
+}
+
+type LockFileFailure = 'read' | 'stale' | 'corrupt' | 'dangling-link';
+
+const LOCK_FILE_FAILURE_ACTIONS: Record<LockFileFailure, string> = {
+ read: 'read the lock',
+ stale: 'remove the stale lock',
+ corrupt: 'remove the corrupt lock',
+ 'dangling-link': 'remove the broken symbolic link',
+};
+
+export class ProcessLockFileError extends Error {
+ readonly lockPath: string;
+ constructor(logPrefix: string, failure: LockFileFailure, lockPath: string, cause: unknown) {
+ super(
+ `${logPrefix} Could not ${LOCK_FILE_FAILURE_ACTIONS[failure]} at ${lockPath}: ${cause instanceof Error ? cause.message : String(cause)}. ${lockFileRemedy(cause)}`,
+ { cause },
+ );
+ this.name = 'ProcessLockFileError';
+ this.lockPath = lockPath;
+ }
+}
+
export function lockFilePath(lockDir: string, lockName: LockName): string {
return resolve(lockDir, `${lockName}.lock`);
}
@@ -84,10 +132,14 @@ function dropActiveLockRef(lockPath: string): boolean {
}
function isSameMachine(existing: ProcessLockMetadata): boolean {
- if (typeof existing.machineId === 'string') return existing.machineId === getMachineId();
- return existing.hostname === hostname();
+ return isLockFromThisMachine(existing);
}
+const LOCK_READ_HOLDER_START_MAX_AGE_MS = 1_000;
+const readRecentHolderStart = createRecentProcessStartReader({
+ maxAgeMs: LOCK_READ_HOLDER_START_MAX_AGE_MS,
+});
+
const exitUnlinkPaths = new Set<string>();
let exitUnlinkHandlerRegistered = false;
@@ -114,18 +166,57 @@ function registerExitUnlink(lockPath: string): void {
});
}
-function parseLock(lockPath: string, logPrefix: string): ProcessLockMetadata | null {
+export function toProcessLockMetadata(record: { pid: number }): ProcessLockMetadata {
+ const { processStart, ...rest } = record as { pid: number; processStart?: unknown };
+ const token = parseProcessStartToken(processStart);
+ return { ...rest, ...(token !== undefined && { processStart: token }) } as ProcessLockMetadata;
+}
+
+function parseLockText(text: string): ProcessLockMetadata | null {
+ let parsed: unknown;
try {
- const parsed = JSON.parse(readFileSync(lockPath, 'utf-8'));
- if (parsed && typeof parsed === 'object' && isValidLockPid((parsed as { pid?: unknown }).pid)) {
- return parsed as ProcessLockMetadata;
- }
- log.warn({ lockPath }, `${logPrefix} Corrupt lock file at ${lockPath} — replacing`);
- return null;
+ parsed = JSON.parse(text);
} catch {
- log.warn({ lockPath }, `${logPrefix} Corrupt lock file at ${lockPath} — replacing`);
return null;
}
+ if (parsed && typeof parsed === 'object' && isValidLockPid((parsed as { pid?: unknown }).pid)) {
+ return toProcessLockMetadata(parsed as { pid: number });
+ }
+ return null;
+}
+
+export type LockRemoval = 'removed' | 'changed' | 'missing';
+
+export function removeLockIfUnchanged(lockPath: string, assessedText: string): LockRemoval {
+ try {
+ if (readFileSync(lockPath, 'utf-8') !== assessedText) return 'changed';
+ unlinkSync(lockPath);
+ return 'removed';
+ } catch (err) {
+ if (errnoCode(err) === 'ENOENT') return 'missing';
+ throw err;
+ }
+}
+
+const WIN32_TRANSIENT_LOCK_FILE_CODES = new Set(['EBUSY', 'EPERM', 'EACCES']);
+const WIN32_TRANSIENT_RETRY_PAUSE_MS = 100;
+
+function isTransientLockFileError(err: unknown): boolean {
+ const code = errnoCode(err);
+ return (
+ process.platform === 'win32' && code !== undefined && WIN32_TRANSIENT_LOCK_FILE_CODES.has(code)
+ );
+}
+
+function removeExitedHolderLock(lockName: LockName, lockPath: string, assessedText: string): void {
+ try {
+ removeLockIfUnchanged(lockPath, assessedText);
+ } catch (err) {
+ log.debug(
+ { lockPath, err },
+ `[${lockName}-lock] Could not remove the lock at ${lockPath} left by an exited holder`,
+ );
+ }
}
function writerChannel(logPrefix: string): string | undefined {
@@ -164,6 +255,15 @@ export function acquireProcessLock(opts: {
mkdirSync(lockDir, { recursive: true });
const lockPath = lockFilePath(lockDir, lockName);
+ const ownStart = readProcessStart(process.pid);
+ const processStart = processStartToken(ownStart);
+ if (processStart === undefined) {
+ const reason = ownStart === null ? 'start-unreadable' : 'boot-id-unreadable';
+ log.warn(
+ { event: 'process-start-unrecorded', platform: process.platform, reason },
+ `${logPrefix} Writing the lock without a processStart (${reason}); readers fall back to its startedAt margin`,
+ );
+ }
const record: ProcessLockMetadata = {
pid: process.pid,
hostname: hostname(),
@@ -172,6 +272,7 @@ export function acquireProcessLock(opts: {
startedAt: new Date().toISOString(),
worktreeRoot: init.worktreeRoot,
machineId: getMachineId(),
+ ...(processStart !== undefined && { processStart }),
...(init.kind !== undefined && { kind: init.kind }),
...(init.parentPid !== undefined && { parentPid: init.parentPid }),
...(init.capabilities !== undefined && { capabilities: init.capabilities }),
@@ -182,7 +283,49 @@ export function acquireProcessLock(opts: {
const payload = JSON.stringify(record, null, 2);
const MAX_ATTEMPTS = 3;
- for (let attempt = 0; attempt < MAX_ATTEMPTS; attempt++) {
+ const keepTransientFailure = (
+ failure: LockFileFailure,
+ err: unknown,
+ attempt: number,
+ ): ProcessLockFileError => {
+ const error = new ProcessLockFileError(logPrefix, failure, lockPath, err);
+ if (!isTransientLockFileError(err)) throw error;
+ if (attempt < MAX_ATTEMPTS) {
+ const code = errnoCode(err);
+ log.warn(
+ { lockPath, code, attempt },
+ `${logPrefix} Could not ${LOCK_FILE_FAILURE_ACTIONS[failure]} at ${lockPath} (${code}); retrying, attempt ${attempt} of ${MAX_ATTEMPTS}`,
+ );
+ sleepSyncMs(WIN32_TRANSIENT_RETRY_PAUSE_MS * attempt);
+ }
+ return error;
+ };
+ const removeAssessedLock = (
+ kind: 'stale' | 'corrupt',
+ text: string,
+ attempt: number,
+ ): ProcessLockFileError | undefined => {
+ try {
+ removeLockIfUnchanged(lockPath, text);
+ return undefined;
+ } catch (err) {
+ return keepTransientFailure(kind, err, attempt);
+ }
+ };
+ const removeDanglingLink = (attempt: number): ProcessLockFileError | undefined => {
+ try {
+ if (!lstatSync(lockPath).isSymbolicLink()) return undefined;
+ unlinkSync(lockPath);
+ log.warn({ lockPath }, `${logPrefix} Removed a broken symbolic link at ${lockPath}`);
+ return undefined;
+ } catch (err) {
+ if (errnoCode(err) === 'ENOENT') return undefined;
+ return keepTransientFailure('dangling-link', err, attempt);
+ }
+ };
+
+ let lastTransientFailure: ProcessLockFileError | undefined;
+ for (let attempt = 1; ; attempt++) {
if (!existsSync(lockPath)) {
try {
const fd = openSync(lockPath, 'wx', 0o600);
@@ -198,8 +341,21 @@ export function acquireProcessLock(opts: {
if (errnoCode(err) !== 'EEXIST') throw err;
}
}
+ if (attempt > MAX_ATTEMPTS) break;
+ lastTransientFailure = undefined;
- const existing = parseLock(lockPath, logPrefix);
+ let text: string;
+ try {
+ text = readFileSync(lockPath, 'utf-8');
+ } catch (err) {
+ lastTransientFailure =
+ errnoCode(err) === 'ENOENT'
+ ? removeDanglingLink(attempt)
+ : keepTransientFailure('read', err, attempt);
+ continue;
+ }
+
+ const existing = parseLockText(text);
if (existing) {
if (isSameMachine(existing) && existing.pid === process.pid) {
writeFileSync(lockPath, payload, { encoding: 'utf-8', mode: 0o600 });
@@ -207,22 +363,38 @@ export function acquireProcessLock(opts: {
registerExitUnlink(lockPath);
return buildHandle({ lockName, lockDir, lockPath });
}
- if (isProcessAlive(existing.pid)) {
+ const assessment = assessLockHolder(existing);
+ if (verdictHoldsLock(assessment.verdict)) {
+ if (assessment.verdict === 'unverified') {
+ log.warn(
+ {
+ lockPid: existing.pid,
+ host: existing.hostname,
+ holder: assessment.verdict,
+ cause: assessment.cause,
+ },
+ `${logPrefix} Could not confirm that pid=${existing.pid} wrote ${lockPath}; refusing to replace it`,
+ );
+ }
throw new ProcessLockCollisionError(existing, lockPath, lockName);
}
log.warn(
- { pid: existing.pid, host: existing.hostname },
+ { lockPid: existing.pid, host: existing.hostname, holder: assessment.verdict },
`${logPrefix} Stale lock detected (pid=${existing.pid}, host=${existing.hostname}) — replacing`,
);
+ lastTransientFailure = removeAssessedLock('stale', text, attempt);
+ continue;
}
- try {
- unlinkSync(lockPath);
- } catch {}
+ log.warn({ lockPath }, `${logPrefix} Corrupt lock file at ${lockPath} — replacing`);
+ lastTransientFailure = removeAssessedLock('corrupt', text, attempt);
}
- throw new Error(
- `${logPrefix} Failed to acquire ${lockPath} after ${MAX_ATTEMPTS} attempts (concurrent acquire contention).`,
+ throw (
+ lastTransientFailure ??
+ new Error(
+ `${logPrefix} Failed to acquire ${lockPath} after ${MAX_ATTEMPTS} attempts (concurrent acquire contention).`,
+ )
);
}
@@ -307,7 +479,7 @@ export function updateProcessLockPort(opts: {
);
return;
}
- existing = parsed as ProcessLockMetadata;
+ existing = toProcessLockMetadata(parsed);
} catch {
log.warn(
{ lockPath },
@@ -358,7 +530,7 @@ export function markProcessLockDraining(opts: { lockName: LockName; lockDir: str
);
return;
}
- existing = parsed as ProcessLockMetadata;
+ existing = toProcessLockMetadata(parsed);
} catch (err) {
if (errnoCode(err) !== 'ENOENT') {
log.warn(
@@ -394,21 +566,25 @@ export function readProcessLock(opts: {
const lockPath = lockFilePath(lockDir, lockName);
if (!existsSync(lockPath)) return null;
+ let text: string;
let existing: ProcessLockMetadata;
try {
- const parsed = JSON.parse(readFileSync(lockPath, 'utf-8'));
+ text = readFileSync(lockPath, 'utf-8');
+ const parsed = JSON.parse(text);
if (!parsed || typeof parsed !== 'object' || !isValidLockPid((parsed as { pid?: unknown }).pid))
return null;
- existing = { ...parsed, channel: lockChannel(parsed.channel) } as ProcessLockMetadata;
+ existing = toProcessLockMetadata({ ...parsed, channel: lockChannel(parsed.channel) });
} catch {
return null;
}
if (!isSameMachine(existing)) return null;
- if (!isProcessAlive(existing.pid)) {
- try {
- unlinkSync(lockPath);
- } catch {}
+ const holder = classifyLockHolder(existing, {
+ sameMachine: true,
+ readProcessStart: readRecentHolderStart,
+ });
+ if (!verdictHoldsLock(holder)) {
+ if (holder === 'gone') removeExitedHolderLock(lockName, lockPath, text);
return null;
}
@@ -429,9 +605,11 @@ export function readProcessLockDetailed(opts: {
const lockPath = lockFilePath(lockDir, lockName);
if (!existsSync(lockPath)) return { status: 'absent' };
+ let text: string;
let raw: unknown;
try {
- raw = JSON.parse(readFileSync(lockPath, 'utf-8'));
+ text = readFileSync(lockPath, 'utf-8');
+ raw = JSON.parse(text);
} catch {
return { status: 'incompatible', reason: 'corrupt', raw: undefined };
}
@@ -458,6 +636,7 @@ export function readProcessLockDetailed(opts: {
startedAt: r.startedAt,
worktreeRoot: r.worktreeRoot,
machineId: typeof r.machineId === 'string' ? r.machineId : undefined,
+ processStart: parseProcessStartToken(r.processStart),
draining: r.draining === true ? true : undefined,
capabilities:
Array.isArray(r.capabilities) && r.capabilities.every((c) => typeof c === 'string')
@@ -469,10 +648,12 @@ export function readProcessLockDetailed(opts: {
};
if (!isSameMachine(lock)) return { status: 'stale', lock };
- if (!isProcessAlive(lock.pid)) {
- try {
- unlinkSync(lockPath);
- } catch {}
+ const holder = classifyLockHolder(lock, {
+ sameMachine: true,
+ readProcessStart: readRecentHolderStart,
+ });
+ if (!verdictHoldsLock(holder)) {
+ if (holder === 'gone') removeExitedHolderLock(lockName, lockPath, text);
return { status: 'stale', lock };
}
diff --git a/packages/server/src/server-factory.test.ts b/packages/server/src/server-factory.test.ts
index 1986aaffa..844bc7f3f 100644
--- a/packages/server/src/server-factory.test.ts
+++ b/packages/server/src/server-factory.test.ts
@@ -36,6 +36,7 @@ import { stringify as stringifyYaml } from 'yaml';
import * as Y from 'yjs';
import { runningAsRoot } from '../../../test-support/capabilities.test-helper.ts';
import { configureTestGitRepository } from '../../../test-support/configure-git-fixture.test-helper.ts';
+import { processStartTokenFromOs } from '../../../test-support/os-process-start.test-helper.ts';
import { MAX_AGENT_SESSIONS } from './agent-sessions.ts';
import { BacklinkIndex } from './backlink-index.ts';
import { getBootTimings, resetBootTimingsForTest, startBootTimings } from './boot-timings.ts';
@@ -2472,6 +2473,7 @@ describe('createServer() server-lock integration (V0-1)', () => {
expect(md.pid).toBe(process.pid);
expect(md.worktreeRoot).toBe(tmpDir);
expect(md.draining).toBeUndefined();
+ expect(md.processStart).toEqual(processStartTokenFromOs(process.pid));
await server.destroy();
@@ -2479,6 +2481,7 @@ describe('createServer() server-lock integration (V0-1)', () => {
const drained = JSON.parse(readFileSync(lockPath, 'utf-8'));
expect(drained.pid).toBe(process.pid);
expect(drained.draining).toBe(true);
+ expect(drained.processStart).toEqual(md.processStart);
});
test('exposes lockDir on ServerInstance', async () => {
diff --git a/packages/server/src/shadow-lock.test.ts b/packages/server/src/shadow-lock.test.ts
index 50e5adda9..c8b7b5f71 100644
--- a/packages/server/src/shadow-lock.test.ts
+++ b/packages/server/src/shadow-lock.test.ts
@@ -2,7 +2,20 @@ import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs';
import { mkdtemp, rm } from 'node:fs/promises';
import { hostname, tmpdir } from 'node:os';
import { resolve } from 'node:path';
-import { afterEach, beforeEach, describe, expect, test } from 'vitest';
+import { afterEach, beforeEach, describe, expect, test, vi } from 'vitest';
+import {
+ processStartTokenFromOs,
+ processStartTokenOffsetFromOs,
+} from '../../../test-support/os-process-start.test-helper.ts';
+import {
+ lockWrittenBeforeProcessStarted,
+ lockWrittenByRunningProcess,
+ type OwnedProcess,
+ ownedProcessHasExited,
+ startOwnedProcess,
+ stopOwnedProcess,
+} from '../../../test-support/owned-process.test-helper.ts';
+import { getLogger } from './logger.ts';
import { acquireLock, type LockMetadata, releaseLock } from './shadow-lock';
let tmpDir: string;
@@ -52,6 +65,49 @@ describe('acquireLock', () => {
expect(metadata.worktreeRoot).toBe('/new/worktree');
});
+ test('records the replaced holder as lockPid, apart from the logger pid binding', () => {
+ const shadowDir = resolve(tmpDir, 'shadow');
+ mkdirSync(shadowDir, { recursive: true });
+ const staleLock: LockMetadata = {
+ pid: 99999999,
+ hostname: hostname(),
+ startedAt: new Date().toISOString(),
+ worktreeRoot: '/old/worktree',
+ };
+ writeFileSync(resolve(shadowDir, 'lock'), JSON.stringify(staleLock), 'utf-8');
+ const warn = vi.spyOn(getLogger('shadow-lock'), 'warn');
+ try {
+ acquireLock(shadowDir, '/new/worktree');
+ expect(warn).toHaveBeenCalledWith(
+ { lockPid: 99999999, host: hostname(), holder: 'gone' },
+ expect.stringContaining('Stale lock detected (pid=99999999'),
+ );
+ } finally {
+ warn.mockRestore();
+ }
+ });
+
+ test('records an invalid lock pid as lockPid, apart from the logger pid binding', () => {
+ const shadowDir = resolve(tmpDir, 'shadow');
+ mkdirSync(shadowDir, { recursive: true });
+ const lockPath = resolve(shadowDir, 'lock');
+ writeFileSync(
+ lockPath,
+ JSON.stringify({ pid: 0, hostname: hostname(), startedAt: new Date().toISOString() }),
+ 'utf-8',
+ );
+ const warn = vi.spyOn(getLogger('shadow-lock'), 'warn');
+ try {
+ acquireLock(shadowDir, '/new/worktree');
+ expect(warn).toHaveBeenCalledWith(
+ { lockPath, lockPid: '0' },
+ expect.stringContaining('Invalid lock pid (0)'),
+ );
+ } finally {
+ warn.mockRestore();
+ }
+ });
+
test('rejects when lock owner is alive (different process)', () => {
const shadowDir = resolve(tmpDir, 'shadow');
mkdirSync(shadowDir, { recursive: true });
@@ -128,3 +184,114 @@ describe('releaseLock', () => {
expect(existsSync(resolve(shadowDir, 'lock'))).toBe(false);
});
});
+
+describe('acquireLock when the recorded pid is still running', () => {
+ const owned: OwnedProcess[] = [];
+
+ afterEach(async () => {
+ for (const holder of owned.splice(0)) await stopOwnedProcess(holder);
+ });
+
+ async function seedLockHeldBy(startedAt: (holder: OwnedProcess) => string): Promise<{
+ shadowDir: string;
+ holder: OwnedProcess;
+ }> {
+ const holder = await startOwnedProcess();
+ owned.push(holder);
+ const shadowDir = resolve(tmpDir, 'shadow');
+ mkdirSync(shadowDir, { recursive: true });
+ const lock: LockMetadata = {
+ pid: holder.pid,
+ hostname: hostname(),
+ startedAt: startedAt(holder),
+ worktreeRoot: '/previous/boot',
+ };
+ writeFileSync(resolve(shadowDir, 'lock'), JSON.stringify(lock), 'utf-8');
+ return { shadowDir, holder };
+ }
+
+ test('replaces a lock whose pid now belongs to a process started after it was written', async () => {
+ const { shadowDir, holder } = await seedLockHeldBy(lockWrittenBeforeProcessStarted);
+ const warn = vi.spyOn(getLogger('shadow-lock'), 'warn');
+
+ try {
+ expect(() => acquireLock(shadowDir, '/new/boot')).not.toThrow();
+ expect(warn).toHaveBeenCalledWith(
+ expect.objectContaining({ lockPid: holder.pid, holder: 'replaced' }),
+ expect.stringContaining('Stale lock detected'),
+ );
+ } finally {
+ warn.mockRestore();
+ }
+ const metadata: LockMetadata = JSON.parse(readFileSync(resolve(shadowDir, 'lock'), 'utf-8'));
+ expect(metadata.pid).toBe(process.pid);
+ expect(metadata.worktreeRoot).toBe('/new/boot');
+ expect(ownedProcessHasExited(holder)).toBe(false);
+ });
+
+ test('still rejects a lock held by the live process that wrote it, as its confirmed writer', async () => {
+ const { shadowDir, holder } = await seedLockHeldBy(lockWrittenByRunningProcess);
+ const warn = vi.spyOn(getLogger('shadow-lock'), 'warn');
+
+ try {
+ expect(() => acquireLock(shadowDir, '/new/boot')).toThrow(/locked by another writer/);
+ expect(warn).not.toHaveBeenCalled();
+ } finally {
+ warn.mockRestore();
+ }
+ const metadata: LockMetadata = JSON.parse(readFileSync(resolve(shadowDir, 'lock'), 'utf-8'));
+ expect(metadata.pid).toBe(holder.pid);
+ });
+});
+
+describe('the process start a shadow lock records for its writer', () => {
+ test('acquireLock records the start the operating system holds for the writer', () => {
+ const shadowDir = resolve(tmpDir, 'shadow');
+ mkdirSync(shadowDir, { recursive: true });
+
+ const lockPath = acquireLock(shadowDir, '/some/worktree');
+
+ const metadata: LockMetadata = JSON.parse(readFileSync(lockPath, 'utf-8'));
+ expect(metadata.processStart).toEqual(processStartTokenFromOs(process.pid));
+ });
+});
+
+describe.skipIf(process.platform === 'win32')(
+ 'acquireLock when the recorded pid is held by a process it cannot verify',
+ () => {
+ const owned: OwnedProcess[] = [];
+
+ afterEach(async () => {
+ for (const holder of owned.splice(0)) await stopOwnedProcess(holder);
+ });
+
+ test('rejects a lock whose recorded process start is later than its holder started', async () => {
+ const holder = await startOwnedProcess();
+ owned.push(holder);
+ const shadowDir = resolve(tmpDir, 'shadow');
+ mkdirSync(shadowDir, { recursive: true });
+ const lock: LockMetadata = {
+ pid: holder.pid,
+ hostname: hostname(),
+ startedAt: lockWrittenByRunningProcess(holder),
+ worktreeRoot: '/previous/boot',
+ processStart: processStartTokenOffsetFromOs(holder.pid, 5),
+ };
+ writeFileSync(resolve(shadowDir, 'lock'), JSON.stringify(lock), 'utf-8');
+ const warn = vi.spyOn(getLogger('shadow-lock'), 'warn');
+
+ try {
+ expect(() => acquireLock(shadowDir, '/new/boot')).toThrow(/locked by another writer/);
+ expect(warn).toHaveBeenCalledWith(
+ { lockPid: holder.pid, holder: 'unverified', cause: 'process-start-mismatch' },
+ expect.stringContaining(`Could not confirm that pid=${holder.pid}`),
+ );
+ } finally {
+ warn.mockRestore();
+ }
+ const metadata: LockMetadata = JSON.parse(readFileSync(resolve(shadowDir, 'lock'), 'utf-8'));
+ expect(metadata.pid).toBe(holder.pid);
+ expect(ownedProcessHasExited(holder)).toBe(false);
+ });
+ },
+);
diff --git a/packages/server/src/shadow-lock.ts b/packages/server/src/shadow-lock.ts
index bf14913a9..f48e5f4ab 100644
--- a/packages/server/src/shadow-lock.ts
+++ b/packages/server/src/shadow-lock.ts
@@ -2,7 +2,13 @@ import { existsSync, readFileSync, unlinkSync, writeFileSync } from 'node:fs';
import { hostname } from 'node:os';
import { resolve } from 'node:path';
import { getLogger } from './logger.ts';
-import { isProcessAlive, isValidLockPid } from './process-alive.ts';
+import { isValidLockPid } from './process-alive.ts';
+import {
+ assessLockHolder,
+ processStartToken,
+ readProcessStart,
+ verdictHoldsLock,
+} from './process-identity.ts';
const log = getLogger('shadow-lock');
@@ -11,6 +17,7 @@ export interface LockMetadata {
hostname: string;
startedAt: string;
worktreeRoot: string;
+ processStart?: unknown;
}
export function acquireLock(shadowDir: string, worktreeRoot: string): string {
@@ -26,15 +33,25 @@ export function acquireLock(shadowDir: string, worktreeRoot: string): string {
if (existing && !isValidLockPid(existing.pid)) {
log.warn(
- { lockPath, pid: String(existing.pid) },
+ { lockPath, lockPid: String(existing.pid) },
`Invalid lock pid (${String(existing.pid)}) at ${lockPath} — replacing`,
);
existing = null;
}
if (existing) {
const sameHost = existing.hostname === hostname();
+ const assessment =
+ sameHost && existing.pid !== process.pid
+ ? assessLockHolder(existing, { sameMachine: true, formatRecordsMachineId: false })
+ : undefined;
if (sameHost && existing.pid === process.pid) {
- } else if (sameHost && isProcessAlive(existing.pid)) {
+ } else if (assessment !== undefined && verdictHoldsLock(assessment.verdict)) {
+ if (assessment.verdict === 'unverified') {
+ log.warn(
+ { lockPid: existing.pid, holder: assessment.verdict, cause: assessment.cause },
+ `Could not confirm that pid=${existing.pid} wrote ${lockPath}; refusing to replace it`,
+ );
+ }
throw new Error(
`Shadow repo at ${shadowDir} is locked by another writer ` +
`(pid=${existing.pid}, worktree=${existing.worktreeRoot}, ` +
@@ -43,18 +60,32 @@ export function acquireLock(shadowDir: string, worktreeRoot: string): string {
);
} else {
log.warn(
- { pid: existing.pid, host: existing.hostname },
+ {
+ lockPid: existing.pid,
+ host: existing.hostname,
+ ...(assessment !== undefined && { holder: assessment.verdict }),
+ },
`Stale lock detected (pid=${existing.pid}, host=${existing.hostname}) — replacing`,
);
}
}
}
+ const ownStart = readProcessStart(process.pid);
+ const processStart = processStartToken(ownStart);
+ if (processStart === undefined) {
+ const reason = ownStart === null ? 'start-unreadable' : 'boot-id-unreadable';
+ log.warn(
+ { event: 'process-start-unrecorded', platform: process.platform, reason },
+ `Writing the shadow lock without a processStart (${reason}); readers fall back to its startedAt margin`,
+ );
+ }
const metadata: LockMetadata = {
pid: process.pid,
hostname: hostname(),
startedAt: new Date().toISOString(),
worktreeRoot,
+ ...(processStart !== undefined && { processStart }),
};
writeFileSync(lockPath, JSON.stringify(metadata, null, 2), 'utf-8');
diff --git a/packages/server/src/sleep-sync.ts b/packages/server/src/sleep-sync.ts
new file mode 100644
index 000000000..23d04d038
--- /dev/null
+++ b/packages/server/src/sleep-sync.ts
@@ -0,0 +1,3 @@
+export function sleepSyncMs(ms: number): void {
+ Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, ms);
+}
diff --git a/test-support/os-process-start.test-helper.ts b/test-support/os-process-start.test-helper.ts
new file mode 100644
index 000000000..45286cc92
--- /dev/null
+++ b/test-support/os-process-start.test-helper.ts
@@ -0,0 +1,62 @@
+import { execFileSync } from 'node:child_process';
+import { readFileSync } from 'node:fs';
+
+export interface OsProcessStart {
+ startedAtMs: number;
+ clock: 'creation' | 'boot-derived';
+ startTicks?: number;
+ bootId?: string;
+}
+
+export type OsProcessStartToken = { bootId: string; startTicks: number } | { startedAtMs: number };
+
+export function readProcessStartFromOs(pid: number): OsProcessStart {
+ if (process.platform === 'darwin') {
+ const lstart = execFileSync('ps', ['-p', String(pid), '-o', 'lstart='], {
+ encoding: 'utf8',
+ env: { ...process.env, LC_ALL: 'C', TZ: 'UTC0' },
+ }).trim();
+ return { startedAtMs: Date.parse(`${lstart} UTC`), clock: 'creation' };
+ }
+ if (process.platform === 'linux') {
+ const stat = readFileSync(`/proc/${pid}/stat`, 'utf8');
+ const startTicks = Number(
+ stat
+ .slice(stat.lastIndexOf(')') + 1)
+ .trim()
+ .split(/\s+/)[19],
+ );
+ const btime = Number(/^btime (\d+)$/m.exec(readFileSync('/proc/stat', 'utf8'))?.[1]);
+ const bootId = readFileSync('/proc/sys/kernel/random/boot_id', 'utf8').trim();
+ return {
+ startedAtMs: btime * 1000 + startTicks * 10,
+ clock: 'boot-derived',
+ startTicks,
+ bootId,
+ };
+ }
+ throw new Error(`no independent process-start source on ${process.platform}`);
+}
+
+export function processStartTokenFromOs(pid: number): OsProcessStartToken {
+ const start = readProcessStartFromOs(pid);
+ if (start.bootId !== undefined && start.startTicks !== undefined) {
+ return { bootId: start.bootId, startTicks: start.startTicks };
+ }
+ return { startedAtMs: start.startedAtMs };
+}
+
+const USER_HZ_TICKS_PER_SECOND = 100;
+
+export function processStartTokenOffsetFromOs(
+ pid: number,
+ offsetSeconds: number,
+): OsProcessStartToken {
+ const token = processStartTokenFromOs(pid);
+ return 'bootId' in token
+ ? {
+ bootId: token.bootId,
+ startTicks: token.startTicks + offsetSeconds * USER_HZ_TICKS_PER_SECOND,
+ }
+ : { startedAtMs: token.startedAtMs + offsetSeconds * 1000 };
+}
diff --git a/test-support/owned-process.test-helper.ts b/test-support/owned-process.test-helper.ts
new file mode 100644
index 000000000..3957897ed
--- /dev/null
+++ b/test-support/owned-process.test-helper.ts
@@ -0,0 +1,83 @@
+import { type ChildProcess, spawn } from 'node:child_process';
+import { once } from 'node:events';
+import { hostname } from 'node:os';
+
+export interface OwnedProcess {
+ readonly child: ChildProcess;
+ readonly pid: number;
+ readonly spawnRequestedAtMs: number;
+ readonly readyAtMs: number;
+}
+
+const OWNED_PROCESS_SOURCE = [
+ "process.on('disconnect', () => process.exit(0));",
+ 'setInterval(() => {}, 1000);',
+ "process.send('ready');",
+].join('\n');
+
+const LOCK_WRITTEN_BEFORE_RESTART_MS = 36 * 60 * 60 * 1000;
+
+export async function startOwnedProcess(): Promise<OwnedProcess> {
+ const spawnRequestedAtMs = Date.now();
+ const child = spawn(process.execPath, ['-e', OWNED_PROCESS_SOURCE], {
+ stdio: ['ignore', 'ignore', 'ignore', 'ipc'],
+ });
+ await new Promise<void>((resolveReady, rejectReady) => {
+ const onExit = (code: number | null, signal: NodeJS.Signals | null) => {
+ rejectReady(new Error(`owned process exited before it was ready (${code ?? signal})`));
+ };
+ child.once('exit', onExit);
+ child.once('error', rejectReady);
+ child.once('message', () => {
+ child.off('exit', onExit);
+ child.off('error', rejectReady);
+ resolveReady();
+ });
+ });
+ const pid = child.pid;
+ if (pid === undefined) throw new Error('owned process has no pid');
+ return { child, pid, spawnRequestedAtMs, readyAtMs: Date.now() };
+}
+
+export async function stopOwnedProcess(owned: OwnedProcess): Promise<void> {
+ const { child } = owned;
+ if (child.exitCode !== null || child.signalCode !== null) return;
+ const exited = once(child, 'exit');
+ child.kill('SIGKILL');
+ await exited;
+}
+
+export function ownedProcessHasExited(owned: OwnedProcess): boolean {
+ return owned.child.exitCode !== null || owned.child.signalCode !== null;
+}
+
+export function lockWrittenBeforeProcessStarted(owned: OwnedProcess): string {
+ return new Date(owned.spawnRequestedAtMs - LOCK_WRITTEN_BEFORE_RESTART_MS).toISOString();
+}
+
+export function lockWrittenByRunningProcess(owned: OwnedProcess): string {
+ return new Date(owned.readyAtMs).toISOString();
+}
+
+export function legacyServerLock(args: {
+ pid: number;
+ startedAt: string;
+ machineId: string;
+ worktreeRoot: string;
+ port?: number;
+}): Record<string, unknown> {
+ const port = args.port ?? 47_931;
+ return {
+ pid: args.pid,
+ hostname: hostname(),
+ port,
+ url: `http://127.0.0.1:${port}`,
+ startedAt: args.startedAt,
+ worktreeRoot: args.worktreeRoot,
+ machineId: args.machineId,
+ kind: 'interactive',
+ capabilities: ['http', 'ws', 'ui'],
+ protocolVersion: 2,
+ runtimeVersion: '0.79.13',
+ };
+}