Skip to content

Beautiful, AI-native markdown IDE and LLM wiki

fix(desktop): tell a frozen main thread from a dead one (PRD-8439) (#4080)

miles-kt-inkeepb89470f7bb3b
diff --git a/.changeset/main-thread-watchdog.md b/.changeset/main-thread-watchdog.md
new file mode 100644
index 000000000..549b7853b
--- /dev/null
+++ b/.changeset/main-thread-watchdog.md
@@ -0,0 +1,9 @@
+---
+"@inkeep/open-knowledge": patch
+---
+
+Crash reports now include evidence of prolonged main-thread freezes.
+
+Until now the "previous session ended without a clean quit" report carried one liveness fact, the last time the main thread's own 60-second heartbeat ran. A main thread that hangs stops that heartbeat exactly the way a crash does, so "the app died at 11:29" and "the app froze at 11:29 and you force-quit it four hours later" produced identical reports and pointed at different bugs.
+
+The desktop now runs a watchdog on a worker thread that records main-thread pings every five seconds. At the next launch, the desktop log reports whether the last successful sample observed a block of at least 15 seconds, together with the measured duration, threshold, sample time and evidence state. A `died` verdict means that sample was below the threshold; it does not rule out a shorter or recovered block, or a block after the watchdog stopped writing. Missing or unusable evidence yields no verdict. The worker re-baselines after a whole-process pause so sleep does not count as a hang. These diagnostics do not change whether the app offers a crash report.
diff --git a/docs/content/reference/what-open-knowledge-writes.mdx b/docs/content/reference/what-open-knowledge-writes.mdx
index 4bb431000..86130a955 100644
--- a/docs/content/reference/what-open-knowledge-writes.mdx
+++ b/docs/content/reference/what-open-knowledge-writes.mdx
@@ -48,6 +48,7 @@ On every platform the app writes app-level state and logs on launch. The table b
| `~/Library/Application Support/OpenKnowledge/Crashpad/` | Native crash minidumps, written by the OS crash handler (Electron's Crashpad) if the app ever crashes. Local-only: the crash reporter runs with `uploadToServer: false`, so no dump is ever uploaded automatically | Outside-project (home dir) |
| `~/Library/Application Support/OpenKnowledge/bug-report-dirty-shutdown.json` | Dirty-shutdown sentinel — written on **every** launch, refreshed as it runs, and removed on a clean quit. The next launch reads it to tell an app crash (prompts you) from the endings that are not a crash: a reboot, an OS shutdown, or dying asleep (all suppressed and logged instead), an update install shutting the app down to replace its files (logged either way, and suppressed in almost every case: a shutdown the app had no chance to record, or one it cannot date because the previous session left no heartbeat, can still prompt so you are able to report it), and a death the app can date to more than a week ago, which is too stale to report usefully — the diagnostics a report would gather describe the day you sent it rather than the day it happened. A fresh crash dump still overrides those, the same as it does for a reboot, a suspend or an OS shutdown — except in the first 30 seconds after the app committed to quitting for an update install, where a dump is part of that same shutdown rather than a crash you could act on. A session that left neither a heartbeat nor a crash dump cannot be dated, so it is never treated as stale | Outside-project (home dir) |
| `~/Library/Application Support/OpenKnowledge/bug-report-crash-acks.json` | Records which crash-report invitations you already answered or dismissed, so the same crash never re-prompts. Answering is what retires a crash: an invitation you simply never saw — because it went unanswered for a day, or the window it was waiting on never appeared — is dropped rather than answered, so that crash can be offered again the next time you open the app | Outside-project (home dir) |
+| `~/Library/Application Support/OpenKnowledge/bug-report-main-thread-liveness.json` | Main-thread liveness witness. A background worker rewrites it every five seconds while the app runs. The next launch can identify a main-thread block that had lasted at least 15 seconds at the last successful sample. A sample below that threshold does not rule out a shorter or recovered block, or a block after the worker stopped writing. Read only at the next launch; left in place on a clean quit and overwritten on the next run, through a `.tmp-` sibling during each write. The verdict lands in the desktop log a bug report carries; it does not change whether the app prompts you | Outside-project (home dir) |
The app registers the `openknowledge://` URL scheme (for deep links) and checks for updates — on launch and periodically while it runs — against the OpenKnowledge update service at `openknowledge.ai/updates`, which redirects to the GitHub release asset and counts the update per version and channel (falling back to GitHub directly if it's unreachable). On macOS and Windows an update on your channel downloads in the background and installs on the next quit; on Linux — where there is no apt/rpm repository — the app downloads the new deb/rpm from the release feed and asks you to authorize the install (a `pkexec` password prompt) each time. You can also check on demand from **Check for updates…** in the app and Help menus. The channel (stable or beta) is fixed by the build you installed. Reinstall a stable build for any supported platform from `openknowledge.ai/download`. The `/download/beta` shortcut serves the latest Apple Silicon macOS beta; Windows and Linux beta installers are available from [GitHub Releases](https://github.com/inkeep/open-knowledge/releases). Override the feed with `OK_UPDATER_FEED_URL`.
diff --git a/packages/desktop/src/main/crash-detection.test.ts b/packages/desktop/src/main/crash-detection.test.ts
index 955534926..66d22ed63 100644
--- a/packages/desktop/src/main/crash-detection.test.ts
+++ b/packages/desktop/src/main/crash-detection.test.ts
@@ -25,6 +25,7 @@ import {
STALE_CRASH_AFTER_MS,
startLocalCrashReporter,
} from './crash-detection.ts';
+import type { WatchdogRead } from './main-thread-watchdog.ts';
import { buildMinidump } from './minidump.test-helper.ts';
import { type AppState, emptyState } from './state-store.ts';
@@ -61,6 +62,9 @@ interface Rig {
setBootSessionUuid(uuid: string | null): void;
setInstallInFlight(inFlight: InstallInFlight | null): void;
setAppVersion(version: string): void;
+ setWatchdogRead(read: WatchdogRead): void;
+ watchdogStarts: string[];
+ watchdogStops(): number;
tick(): Date;
advance(ms: number): void;
nowMs(): number;
@@ -76,6 +80,9 @@ function makeRig(): Rig {
let rendererAvailable = true;
let bootSessionUuid: string | null = 'boot-epoch-a';
let installInFlight: InstallInFlight | null = null;
+ let watchdogRead: WatchdogRead = { kind: 'absent' };
+ const watchdogStarts: string[] = [];
+ let watchdogStops = 0;
let clockMs = Date.parse('2026-07-10T00:00:00.000Z');
const appBundleRoot = join(dir, 'Applications', 'OpenKnowledge.app');
const ownModules = [
@@ -123,6 +130,11 @@ function makeRig(): Rig {
setAppVersion(version: string) {
rig.deps.appVersion = version;
},
+ setWatchdogRead(read: WatchdogRead) {
+ watchdogRead = read;
+ },
+ watchdogStarts,
+ watchdogStops: () => watchdogStops,
tick() {
clockMs += 10_000;
return new Date(clockMs);
@@ -151,6 +163,18 @@ function makeRig(): Rig {
},
currentBootSessionUuid: () => bootSessionUuid,
installInFlight: () => installInFlight,
+ mainThreadWatchdog: {
+ readPrevious: () => watchdogRead,
+ start: (bootId: string) => {
+ watchdogStarts.push(bootId);
+ watchdogRead = { kind: 'absent' };
+ return {
+ stop: () => {
+ watchdogStops += 1;
+ },
+ };
+ },
+ },
logger: {
info: (payload) => {
infos.push(payload);
@@ -2968,3 +2992,175 @@ describe('staleness bounds on the crash reporter', () => {
expect(bootInvite(armed).crashedAt).toBe(dumpAt.toISOString());
});
});
+
+describe('whether the previous session died or hung', () => {
+ function captureLines(rig: Rig): Array<Record<string, unknown>> {
+ const lines: Array<Record<string, unknown>> = [];
+ rig.deps.logger = {
+ info: (payload: Record<string, unknown>) => {
+ lines.push(payload);
+ },
+ warn: (payload: Record<string, unknown>) => {
+ lines.push(payload);
+ rig.warnings.push(payload);
+ },
+ };
+ return lines;
+ }
+
+ function witnessFor(
+ rig: Rig,
+ blockedForMs: number,
+ bootId = readSentinel(rig).bootId ?? '',
+ writtenAtMs = rig.nowMs(),
+ ) {
+ rig.setWatchdogRead({
+ kind: 'record',
+ record: {
+ schemaVersion: 1,
+ bootId,
+ writtenAt: new Date(writtenAtMs).toISOString(),
+ tickMs: 5_000,
+ blockedForMs,
+ mainTicksObserved: 4,
+ },
+ });
+ }
+
+ function bootBreadcrumb(lines: Array<Record<string, unknown>>) {
+ const line = lines.find((l) => l.event === 'crash-detection.boot');
+ expect(line).toBeDefined();
+ return line as Record<string, unknown>;
+ }
+
+ test('the previous witness is read before this session starts its own', () => {
+ const rig = makeRig();
+ createCrashDetection(rig.deps).detectBootCrash();
+ witnessFor(rig, 4_000);
+ const lines = captureLines(rig);
+
+ createCrashDetection(rig.deps).detectBootCrash();
+
+ expect(bootBreadcrumb(lines).livenessEvidence).toBe('matched');
+ });
+
+ test('a witness that fell behind the last heartbeat cannot speak to the death', () => {
+ const rig = makeRig();
+ const dying = createCrashDetection(rig.deps);
+ dying.detectBootCrash();
+ rig.advance(5 * 60_000);
+ dying.noteAlive();
+ const lastAliveMs = Date.parse(readSentinel(rig).lastAliveAt ?? '');
+ witnessFor(rig, 4_000, undefined, lastAliveMs - 60_000);
+ const lines = captureLines(rig);
+
+ createCrashDetection(rig.deps).detectBootCrash();
+
+ const breadcrumb = bootBreadcrumb(lines);
+ expect(breadcrumb.livenessVerdict).toBeNull();
+ expect(breadcrumb.livenessEvidence).toBe('stale-witness');
+ });
+
+ test('a witness that outlived a silent main thread reports a hang and its span', () => {
+ const rig = makeRig();
+ createCrashDetection(rig.deps).detectBootCrash();
+ witnessFor(rig, 14_320_118);
+ const lines = captureLines(rig);
+
+ createCrashDetection(rig.deps).detectBootCrash();
+
+ const breadcrumb = bootBreadcrumb(lines);
+ expect(breadcrumb.livenessVerdict).toBe('blocked');
+ expect(breadcrumb.mainThreadBlockedForMs).toBe(14_320_118);
+ expect(breadcrumb.mainThreadStallThresholdMs).toBe(15_000);
+ expect(breadcrumb.livenessEvidence).toBe('matched');
+ expect(breadcrumb.livenessWitnessAt).toBeTruthy();
+ });
+
+ test('a witness below the stall threshold reports died', () => {
+ const rig = makeRig();
+ createCrashDetection(rig.deps).detectBootCrash();
+ witnessFor(rig, 4_000);
+ const lines = captureLines(rig);
+
+ createCrashDetection(rig.deps).detectBootCrash();
+
+ const breadcrumb = bootBreadcrumb(lines);
+ expect(breadcrumb.livenessVerdict).toBe('died');
+ expect(breadcrumb.mainThreadBlockedForMs).toBe(4_000);
+ expect(breadcrumb.mainThreadStallThresholdMs).toBe(15_000);
+ expect(breadcrumb.livenessEvidence).toBe('matched');
+ });
+
+ test('no witness file spells the verdict out as unknown', () => {
+ const rig = makeRig();
+ createCrashDetection(rig.deps).detectBootCrash();
+ const lines = captureLines(rig);
+
+ createCrashDetection(rig.deps).detectBootCrash();
+
+ const breadcrumb = bootBreadcrumb(lines);
+ expect(breadcrumb.livenessVerdict).toBeNull();
+ expect(breadcrumb.mainThreadBlockedForMs).toBeNull();
+ expect(breadcrumb.livenessWitnessAt).toBeNull();
+ expect(breadcrumb.livenessEvidence).toBe('absent');
+ });
+
+ test('a witness left by an older session is not evidence about this death', () => {
+ const rig = makeRig();
+ createCrashDetection(rig.deps).detectBootCrash();
+ witnessFor(rig, 14_320_118, 'a-much-older-boot');
+ const lines = captureLines(rig);
+
+ createCrashDetection(rig.deps).detectBootCrash();
+
+ const breadcrumb = bootBreadcrumb(lines);
+ expect(breadcrumb.livenessVerdict).toBeNull();
+ expect(breadcrumb.mainThreadBlockedForMs).toBeNull();
+ expect(breadcrumb.livenessEvidence).toBe('boot-mismatch');
+ });
+
+ test('a suppressed machine-level death still carries the verdict', () => {
+ const rig = makeRig();
+ const sessionA = createCrashDetection(rig.deps);
+ sessionA.detectBootCrash();
+ sessionA.noteOsShutdown();
+ witnessFor(rig, 90_000);
+ const lines = captureLines(rig);
+
+ expect(createCrashDetection(rig.deps).detectBootCrash()).toBeNull();
+
+ const breadcrumb = lines.find((l) => l.event === 'crash-detection.machine-level-death');
+ expect(breadcrumb?.livenessVerdict).toBe('blocked');
+ expect(breadcrumb?.mainThreadBlockedForMs).toBe(90_000);
+ expect(breadcrumb?.livenessEvidence).toBe('matched');
+ });
+
+ test('the witness for this session starts under the bootId this boot armed', () => {
+ const rig = makeRig();
+ createCrashDetection(rig.deps).detectBootCrash();
+
+ expect(rig.watchdogStarts).toEqual([readSentinel(rig).bootId]);
+ });
+
+ test('a clean quit stops the witness before the sentinel goes', () => {
+ const rig = makeRig();
+ const detection = createCrashDetection(rig.deps);
+ detection.detectBootCrash();
+ expect(rig.watchdogStops()).toBe(0);
+
+ detection.markCleanQuit();
+
+ expect(rig.watchdogStops()).toBe(1);
+ });
+
+ test('detecting twice retires the first witness rather than leaving two running', () => {
+ const rig = makeRig();
+ const detection = createCrashDetection(rig.deps);
+ detection.detectBootCrash();
+ detection.detectBootCrash();
+
+ expect(rig.watchdogStarts).toHaveLength(2);
+ expect(rig.watchdogStops()).toBe(1);
+ });
+});
diff --git a/packages/desktop/src/main/crash-detection.ts b/packages/desktop/src/main/crash-detection.ts
index 6e280e979..b76ef81cf 100644
--- a/packages/desktop/src/main/crash-detection.ts
+++ b/packages/desktop/src/main/crash-detection.ts
@@ -10,6 +10,13 @@ import {
import { dirname, join } from 'node:path';
import type { OkBugReportCrashDetectedEvent } from '@inkeep/open-knowledge-core';
import { asReportableAppVersion } from './crashed-app-version.ts';
+import {
+ classifyPreviousLiveness,
+ isFileMissingError,
+ livenessLogFields,
+ type MainThreadWatchdog,
+ type MainThreadWatchdogHandle,
+} from './main-thread-watchdog.ts';
import {
classifyMinidumpCrashKind,
classifyMinidumpOwnership,
@@ -103,6 +110,7 @@ export interface CrashDetectionDeps {
now(): Date;
currentBootSessionUuid(): string | null;
installInFlight?(span: { deathFromMs: number; deathToMs: number }): InstallInFlight | null;
+ mainThreadWatchdog: Pick<MainThreadWatchdog, 'readPrevious' | 'start'>;
logger: CrashLogger;
}
@@ -143,12 +151,6 @@ export function startLocalCrashReporter(reporter: {
reporter.start({ uploadToServer: false });
}
-function isFileMissingError(err: unknown): boolean {
- return (
- typeof err === 'object' && err !== null && (err as NodeJS.ErrnoException).code === 'ENOENT'
- );
-}
-
function epochMsOrNull(iso: string | null): number | null {
if (iso === null) return null;
const ms = Date.parse(iso);
@@ -224,6 +226,7 @@ export function createCrashDetection(deps: CrashDetectionDeps): CrashDetection {
let sentinel: SentinelState | null = null;
let cleanQuitMarked = false;
+ let watchdog: MainThreadWatchdogHandle | null = null;
type SentinelWriteContext = 'arm' | 'alive' | 'os-shutdown' | 'suspend' | 'resume';
@@ -495,6 +498,14 @@ export function createCrashDetection(deps: CrashDetectionDeps): CrashDetection {
? installInFlight.handoffAt
: null;
+ const livenessFields = livenessLogFields(
+ classifyPreviousLiveness(
+ deps.mainThreadWatchdog.readPrevious(),
+ prevBootId,
+ epochMsOrNull(prevLastAliveAt),
+ ),
+ );
+
const freshDumps: ClassifiedDump[] = freshMinidumpEntries().map((entry) => {
const ownership = classifyDump(entry.path);
return {
@@ -651,6 +662,7 @@ export function createCrashDetection(deps: CrashDetectionDeps): CrashDetection {
suspendedAt: prevSuspendedAt,
pendingOsShutdownAt: prevPendingOsShutdownAt,
osShutdownReasons: prevOsShutdownReasons,
+ ...livenessFields,
};
if (reason === 'os-shutdown') {
deps.logger.warn(
@@ -714,6 +726,7 @@ export function createCrashDetection(deps: CrashDetectionDeps): CrashDetection {
}
: {}),
detectingAppVersion: deps.appVersion,
+ ...livenessFields,
},
'previous session ended uncleanly — arming report invitation',
);
@@ -734,12 +747,16 @@ export function createCrashDetection(deps: CrashDetectionDeps): CrashDetection {
...(bootSessionUuid !== null ? { bootSessionUuid } : {}),
};
writeSentinel('arm');
+ watchdog?.stop();
+ watchdog = deps.mainThreadWatchdog.start(sentinel.bootId);
return armed;
},
markCleanQuit(): void {
cleanQuitMarked = true;
+ watchdog?.stop();
+ watchdog = null;
try {
rmSync(deps.sentinelPath, { force: true });
} catch (err) {
diff --git a/packages/desktop/src/main/index.ts b/packages/desktop/src/main/index.ts
index 91d6ac92c..421686e52 100644
--- a/packages/desktop/src/main/index.ts
+++ b/packages/desktop/src/main/index.ts
@@ -317,6 +317,7 @@ import {
runManualInstallFallbackDialog,
} from './linux-install-fallback.ts';
import { createMenuTranslator, resolveMenuCatalogDir } from './main-i18n.ts';
+import { createMainThreadWatchdog } from './main-thread-watchdog.ts';
import {
checkAndRepairMcpWiringOnStartup,
type McpStartupRepairResult,
@@ -5752,6 +5753,10 @@ function bootPrimaryInstance(): void {
span,
bootStateSnapshot.versionPendingInstallStagedAt,
),
+ mainThreadWatchdog: createMainThreadWatchdog({
+ path: join(app.getPath('userData'), 'bug-report-main-thread-liveness.json'),
+ logger: getLogger('main-thread-watchdog'),
+ }),
logger: getLogger('crash-detection'),
});
crashDetection.detectBootCrash();
diff --git a/packages/desktop/src/main/ipc/bug-report.test.ts b/packages/desktop/src/main/ipc/bug-report.test.ts
index 627909c15..56b6c57ad 100644
--- a/packages/desktop/src/main/ipc/bug-report.test.ts
+++ b/packages/desktop/src/main/ipc/bug-report.test.ts
@@ -2282,6 +2282,10 @@ describe('handleBugReportCrashAck', () => {
return new Date(clockMs);
},
currentBootSessionUuid: () => 'boot-epoch-test',
+ mainThreadWatchdog: {
+ readPrevious: () => ({ kind: 'absent' }) as const,
+ start: () => ({ stop: () => {} }),
+ },
logger: { info: () => {}, warn: () => {} },
};
const seedMinidump = (relPath: string): void => {
diff --git a/packages/desktop/src/main/main-thread-watchdog.test.ts b/packages/desktop/src/main/main-thread-watchdog.test.ts
new file mode 100644
index 000000000..b66ca026f
--- /dev/null
+++ b/packages/desktop/src/main/main-thread-watchdog.test.ts
@@ -0,0 +1,411 @@
+import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join, resolve } from 'node:path';
+import { setTimeout as delay } from 'node:timers/promises';
+import { afterEach, describe, expect, test } from 'vitest';
+import {
+ classifyPreviousLiveness,
+ computeWatchdogTick,
+ createMainThreadWatchdog,
+ livenessLogFields,
+ type MainThreadWatchdogHandle,
+ parseWatchdogRecord,
+ STALL_THRESHOLD_TICKS,
+ stallThresholdMs,
+ WATCHDOG_TICK_MS,
+ type WatchdogRecord,
+ type WatchdogTickState,
+} from './main-thread-watchdog.ts';
+
+const tmpDirs: string[] = [];
+const handles: MainThreadWatchdogHandle[] = [];
+
+afterEach(() => {
+ for (const handle of handles.splice(0)) handle.stop();
+ for (const dir of tmpDirs.splice(0)) {
+ rmSync(dir, { recursive: true, force: true });
+ }
+});
+
+function makeDir(): string {
+ const dir = mkdtempSync(resolve(tmpdir(), 'ok-main-thread-watchdog-'));
+ tmpDirs.push(dir);
+ return dir;
+}
+
+function record(overrides: Partial<WatchdogRecord> = {}): WatchdogRecord {
+ return {
+ schemaVersion: 1,
+ bootId: 'boot-1',
+ writtenAt: '2026-07-10T00:00:00.000Z',
+ tickMs: WATCHDOG_TICK_MS,
+ blockedForMs: 0,
+ mainTicksObserved: 12,
+ ...overrides,
+ };
+}
+
+describe('classifyPreviousLiveness', () => {
+ test('a witness that kept hearing the main thread says the session died', () => {
+ const liveness = classifyPreviousLiveness(
+ { kind: 'record', record: record({ blockedForMs: 4_000 }) },
+ 'boot-1',
+ null,
+ );
+
+ expect(liveness).toEqual({
+ kind: 'died',
+ blockedForMs: 4_000,
+ stallThresholdMs: 15_000,
+ writtenAt: '2026-07-10T00:00:00.000Z',
+ });
+ });
+
+ test('a witness that outlived a silent main thread says it was blocked', () => {
+ const liveness = classifyPreviousLiveness(
+ { kind: 'record', record: record({ blockedForMs: 14_320_118 }) },
+ 'boot-1',
+ null,
+ );
+
+ expect(liveness).toEqual({
+ kind: 'blocked',
+ blockedForMs: 14_320_118,
+ stallThresholdMs: 15_000,
+ writtenAt: '2026-07-10T00:00:00.000Z',
+ });
+ });
+
+ test('the stall threshold is three ticks and is inclusive', () => {
+ expect(stallThresholdMs(record())).toBe(15_000);
+ expect(
+ classifyPreviousLiveness(
+ { kind: 'record', record: record({ blockedForMs: 15_000 }) },
+ 'boot-1',
+ null,
+ ).kind,
+ ).toBe('blocked');
+ expect(
+ classifyPreviousLiveness(
+ { kind: 'record', record: record({ blockedForMs: 14_999 }) },
+ 'boot-1',
+ null,
+ ).kind,
+ ).toBe('died');
+ });
+
+ test('the threshold tracks the tick cadence the record was written at', () => {
+ expect(stallThresholdMs(record({ tickMs: 20 }))).toBe(60);
+ });
+
+ test('a missing or torn file is no evidence rather than a verdict', () => {
+ expect(classifyPreviousLiveness({ kind: 'absent' }, 'boot-1', null)).toEqual({
+ kind: 'no-evidence',
+ why: 'absent',
+ });
+ expect(classifyPreviousLiveness({ kind: 'unreadable' }, 'boot-1', null)).toEqual({
+ kind: 'no-evidence',
+ why: 'unreadable',
+ });
+ });
+
+ test('a record from another session is no evidence about this one', () => {
+ expect(
+ classifyPreviousLiveness(
+ { kind: 'record', record: record({ blockedForMs: 60_000 }) },
+ 'boot-2',
+ null,
+ ),
+ ).toEqual({ kind: 'no-evidence', why: 'boot-mismatch' });
+ });
+
+ test('an unidentifiable previous session cannot claim the record', () => {
+ expect(
+ classifyPreviousLiveness(
+ { kind: 'record', record: record({ blockedForMs: 60_000 }) },
+ null,
+ null,
+ ),
+ ).toEqual({ kind: 'no-evidence', why: 'no-previous-boot' });
+ });
+
+ test('a record that never heard the main thread cannot vouch for a death', () => {
+ expect(
+ classifyPreviousLiveness(
+ { kind: 'record', record: record({ blockedForMs: 4_000, mainTicksObserved: 0 }) },
+ 'boot-1',
+ null,
+ ),
+ ).toEqual({ kind: 'no-evidence', why: 'unwitnessed-main' });
+ });
+
+ test('a record that never heard the main thread can still prove a hang', () => {
+ expect(
+ classifyPreviousLiveness(
+ { kind: 'record', record: record({ blockedForMs: 60_000, mainTicksObserved: 0 }) },
+ 'boot-1',
+ null,
+ ),
+ ).toEqual({
+ kind: 'blocked',
+ blockedForMs: 60_000,
+ stallThresholdMs: 15_000,
+ writtenAt: record().writtenAt,
+ });
+ });
+
+ test('a witness that stopped writing long before the last heartbeat is not trusted', () => {
+ const writtenAt = '2026-07-10T00:00:00.000Z';
+ const lastAliveMs = Date.parse(writtenAt) + 20_000;
+
+ expect(
+ classifyPreviousLiveness(
+ { kind: 'record', record: record({ blockedForMs: 4_000, writtenAt }) },
+ 'boot-1',
+ lastAliveMs,
+ ),
+ ).toEqual({ kind: 'no-evidence', why: 'stale-witness' });
+ });
+
+ test('a heartbeat that only slightly outran the witness still yields a verdict', () => {
+ const writtenAt = '2026-07-10T00:00:00.000Z';
+ const lastAliveMs = Date.parse(writtenAt) + 3_000;
+
+ expect(
+ classifyPreviousLiveness(
+ { kind: 'record', record: record({ blockedForMs: 4_000, writtenAt }) },
+ 'boot-1',
+ lastAliveMs,
+ ).kind,
+ ).toBe('died');
+ });
+});
+
+describe('computeWatchdogTick', () => {
+ function tickState(overrides: Partial<WatchdogTickState> = {}): WatchdogTickState {
+ return { lastMainAt: 1_000, lastTickAt: 90, mainTicksObserved: 5, ...overrides };
+ }
+
+ test('reports the gap since the last ping when the worker kept its own schedule', () => {
+ const state = tickState();
+
+ const rec = computeWatchdogTick(state, 5_000, 100, 20, STALL_THRESHOLD_TICKS, 'boot-x', 'W');
+
+ expect(rec.blockedForMs).toBe(4_000);
+ expect(rec.mainTicksObserved).toBe(5);
+ expect(rec.bootId).toBe('boot-x');
+ expect(rec.tickMs).toBe(20);
+ expect(state.lastTickAt).toBe(100);
+ });
+
+ test('re-baselines instead of blaming the main thread when the worker itself was descheduled', () => {
+ const state = tickState({ lastTickAt: 0 });
+
+ const rec = computeWatchdogTick(
+ state,
+ 2_000,
+ 30_000_000,
+ 20,
+ STALL_THRESHOLD_TICKS,
+ 'boot-x',
+ 'W',
+ );
+
+ expect(rec.blockedForMs).toBe(0);
+ expect(rec.mainTicksObserved).toBe(0);
+ expect(state.lastMainAt).toBe(2_000);
+ });
+});
+
+describe('parseWatchdogRecord', () => {
+ test('a written record round-trips', () => {
+ const written = record({ blockedForMs: 7, mainTicksObserved: 3 });
+
+ expect(parseWatchdogRecord(`${JSON.stringify(written)}\n`)).toEqual(written);
+ });
+
+ test('every truncation of a record reads as unparseable without throwing', () => {
+ const raw = JSON.stringify(record());
+
+ for (let end = 0; end < raw.length; end += 1) {
+ expect(parseWatchdogRecord(raw.slice(0, end)), `truncated at ${end}`).toBeNull();
+ }
+ });
+
+ test('a record from an unknown schema version is rejected', () => {
+ expect(parseWatchdogRecord(JSON.stringify({ ...record(), schemaVersion: 2 }))).toBeNull();
+ });
+
+ test('a record without a usable bootId is rejected', () => {
+ expect(parseWatchdogRecord(JSON.stringify({ ...record(), bootId: undefined }))).toBeNull();
+ expect(parseWatchdogRecord(JSON.stringify({ ...record(), bootId: '' }))).toBeNull();
+ expect(parseWatchdogRecord(JSON.stringify({ ...record(), bootId: 7 }))).toBeNull();
+ });
+
+ test('a record whose timestamp is not a date is rejected', () => {
+ expect(
+ parseWatchdogRecord(JSON.stringify({ ...record(), writtenAt: 'not-a-date' })),
+ ).toBeNull();
+ });
+
+ test('non-finite and negative durations are rejected', () => {
+ expect(
+ parseWatchdogRecord(
+ '{"schemaVersion":1,"bootId":"b","writtenAt":"2026-07-10T00:00:00.000Z","tickMs":1e999,"blockedForMs":0,"mainTicksObserved":0}',
+ ),
+ ).toBeNull();
+ expect(parseWatchdogRecord(JSON.stringify({ ...record(), blockedForMs: -1 }))).toBeNull();
+ expect(
+ parseWatchdogRecord(JSON.stringify({ ...record(), mainTicksObserved: null })),
+ ).toBeNull();
+ });
+
+ test('a non-positive tick cadence is rejected so a stall threshold of zero cannot invert the verdict', () => {
+ expect(parseWatchdogRecord(JSON.stringify({ ...record(), tickMs: 0 }))).toBeNull();
+ expect(parseWatchdogRecord(JSON.stringify({ ...record(), tickMs: -5 }))).toBeNull();
+ });
+
+ test('a JSON value that is not an object is rejected', () => {
+ expect(parseWatchdogRecord('null')).toBeNull();
+ expect(parseWatchdogRecord('[]')).toBeNull();
+ });
+});
+
+describe('livenessLogFields', () => {
+ test('a verdict names the witness that carried it', () => {
+ expect(
+ livenessLogFields({
+ kind: 'blocked',
+ blockedForMs: 90_000,
+ stallThresholdMs: 15_000,
+ writtenAt: 'W',
+ }),
+ ).toStrictEqual({
+ livenessVerdict: 'blocked',
+ mainThreadBlockedForMs: 90_000,
+ mainThreadStallThresholdMs: 15_000,
+ livenessWitnessAt: 'W',
+ livenessEvidence: 'matched',
+ });
+ expect(
+ livenessLogFields({
+ kind: 'died',
+ blockedForMs: 12,
+ stallThresholdMs: 15_000,
+ writtenAt: 'W',
+ }),
+ ).toStrictEqual({
+ livenessVerdict: 'died',
+ mainThreadBlockedForMs: 12,
+ mainThreadStallThresholdMs: 15_000,
+ livenessWitnessAt: 'W',
+ livenessEvidence: 'matched',
+ });
+ });
+
+ test('absent evidence spells its nulls out rather than omitting the keys', () => {
+ for (const why of [
+ 'absent',
+ 'unreadable',
+ 'no-previous-boot',
+ 'boot-mismatch',
+ 'stale-witness',
+ 'unwitnessed-main',
+ ] as const) {
+ const fields = livenessLogFields({ kind: 'no-evidence', why });
+
+ expect(Object.keys(fields).sort()).toEqual([
+ 'livenessEvidence',
+ 'livenessVerdict',
+ 'livenessWitnessAt',
+ 'mainThreadBlockedForMs',
+ 'mainThreadStallThresholdMs',
+ ]);
+ expect(fields).toStrictEqual({
+ livenessVerdict: null,
+ mainThreadBlockedForMs: null,
+ mainThreadStallThresholdMs: null,
+ livenessWitnessAt: null,
+ livenessEvidence: why,
+ });
+ }
+ });
+});
+
+describe('the worker witness', () => {
+ test('it records the running session and the pings it heard from main', async () => {
+ const path = join(makeDir(), 'nested', 'watchdog.json');
+ const warnings: Record<string, unknown>[] = [];
+ const watchdog = createMainThreadWatchdog({
+ path,
+ logger: {
+ warn: (payload) => {
+ warnings.push(payload);
+ },
+ },
+ tickMs: 20,
+ });
+
+ expect(watchdog.readPrevious()).toEqual({ kind: 'absent' });
+
+ const handle = watchdog.start('boot-x');
+ handles.push(handle);
+
+ let written: WatchdogRecord | null = null;
+ const deadline = Date.now() + 10_000;
+ while (written === null && Date.now() < deadline) {
+ await delay(20);
+ try {
+ written = parseWatchdogRecord(readFileSync(path, 'utf8'));
+ } catch {
+ written = null;
+ }
+ if (written !== null && written.mainTicksObserved < 1) written = null;
+ }
+
+ expect(written).not.toBeNull();
+ expect(written?.bootId).toBe('boot-x');
+ expect(written?.tickMs).toBe(20);
+ expect(written?.mainTicksObserved).toBeGreaterThanOrEqual(1);
+ expect(written?.blockedForMs).toBeLessThan(1_000);
+ expect(Number.isFinite(Date.parse(written?.writtenAt ?? ''))).toBe(true);
+
+ expect(watchdog.readPrevious()).toEqual({ kind: 'record', record: written });
+
+ handle.stop();
+ handle.stop();
+
+ await delay(200);
+
+ expect(warnings).toEqual([]);
+ });
+
+ test('a witness file it cannot write warns once and keeps trying', async () => {
+ const parentAsFile = join(makeDir(), 'not-a-directory');
+ writeFileSync(parentAsFile, '');
+ const warnings: Record<string, unknown>[] = [];
+ const watchdog = createMainThreadWatchdog({
+ path: join(parentAsFile, 'watchdog.json'),
+ logger: {
+ warn: (payload) => {
+ warnings.push(payload);
+ },
+ },
+ tickMs: 20,
+ });
+
+ const handle = watchdog.start('boot-y');
+ handles.push(handle);
+ await delay(400);
+ handle.stop();
+
+ expect(warnings.map((w) => w.event)).toEqual(['main-thread-watchdog.write-failed']);
+ });
+
+ test('an unreadable file reads as unreadable rather than absent', () => {
+ const dir = makeDir();
+ const watchdog = createMainThreadWatchdog({ path: dir, logger: { warn: () => {} } });
+
+ expect(watchdog.readPrevious()).toEqual({ kind: 'unreadable' });
+ });
+});
diff --git a/packages/desktop/src/main/main-thread-watchdog.ts b/packages/desktop/src/main/main-thread-watchdog.ts
new file mode 100644
index 000000000..ff6dbc024
--- /dev/null
+++ b/packages/desktop/src/main/main-thread-watchdog.ts
@@ -0,0 +1,340 @@
+import { readFileSync } from 'node:fs';
+import { Worker } from 'node:worker_threads';
+
+export const WATCHDOG_TICK_MS = 5_000;
+
+export const STALL_THRESHOLD_TICKS = 3;
+
+export interface WatchdogRecord {
+ readonly schemaVersion: 1;
+ readonly bootId: string;
+ readonly writtenAt: string;
+ readonly tickMs: number;
+ readonly blockedForMs: number;
+ readonly mainTicksObserved: number;
+}
+
+export interface WatchdogTickState {
+ lastMainAt: number;
+ lastTickAt: number;
+ mainTicksObserved: number;
+}
+
+export type WatchdogRead =
+ | { kind: 'record'; record: WatchdogRecord }
+ | { kind: 'absent' }
+ | { kind: 'unreadable' };
+
+export type PreviousSessionLiveness =
+ | { kind: 'died'; blockedForMs: number; stallThresholdMs: number; writtenAt: string }
+ | { kind: 'blocked'; blockedForMs: number; stallThresholdMs: number; writtenAt: string }
+ | {
+ kind: 'no-evidence';
+ why:
+ | 'absent'
+ | 'unreadable'
+ | 'no-previous-boot'
+ | 'boot-mismatch'
+ | 'stale-witness'
+ | 'unwitnessed-main';
+ };
+
+export interface LivenessLogFields {
+ livenessVerdict: 'died' | 'blocked' | null;
+ mainThreadBlockedForMs: number | null;
+ mainThreadStallThresholdMs: number | null;
+ livenessWitnessAt: string | null;
+ livenessEvidence:
+ | 'matched'
+ | 'absent'
+ | 'unreadable'
+ | 'no-previous-boot'
+ | 'boot-mismatch'
+ | 'stale-witness'
+ | 'unwitnessed-main';
+}
+
+export interface MainThreadWatchdogHandle {
+ stop(): void;
+}
+
+export interface MainThreadWatchdog {
+ readPrevious(): WatchdogRead;
+ start(bootId: string): MainThreadWatchdogHandle;
+}
+
+interface WatchdogLogger {
+ warn(payload: Record<string, unknown>, msg: string): void;
+}
+
+export function computeWatchdogTick(
+ state: WatchdogTickState,
+ nowPerf: number,
+ nowWall: number,
+ tickMs: number,
+ stallThresholdTicks: number,
+ bootId: string,
+ writtenAt: string,
+): WatchdogRecord {
+ if (nowWall - state.lastTickAt > tickMs * stallThresholdTicks) {
+ state.lastMainAt = nowPerf;
+ state.mainTicksObserved = 0;
+ }
+ state.lastTickAt = nowWall;
+ return {
+ schemaVersion: 1,
+ bootId,
+ writtenAt,
+ tickMs,
+ blockedForMs: Math.round(nowPerf - state.lastMainAt),
+ mainTicksObserved: state.mainTicksObserved,
+ };
+}
+
+/* STOP: computeWatchdogTick is serialized into WORKER_SOURCE through toString(), so it must stay
+ closure-free and import-free. WORKER_SOURCE is an eval string rather than a file-backed worker
+ entry because electron-builder.yml unpacks node-pty's worker on the recorded ground that
+ worker_threads cannot load JS from inside app.asar; a file-backed entry has to prove it loads
+ from a packaged build before it replaces this. */
+const WORKER_SOURCE = `
+const { mkdirSync, renameSync, unlinkSync, writeFileSync } = require('node:fs');
+const { dirname } = require('node:path');
+const { parentPort, workerData } = require('node:worker_threads');
+
+const { path, bootId, tickMs } = workerData;
+const computeTick = ${computeWatchdogTick.toString()};
+const state = {
+ lastMainAt: performance.now(),
+ lastTickAt: Date.now(),
+ mainTicksObserved: 0,
+};
+let reportedFailure = false;
+
+parentPort.on('message', () => {
+ state.lastMainAt = performance.now();
+ state.mainTicksObserved += 1;
+});
+
+function writeWitness() {
+ const record = computeTick(
+ state,
+ performance.now(),
+ Date.now(),
+ tickMs,
+ ${STALL_THRESHOLD_TICKS},
+ bootId,
+ new Date().toISOString(),
+ );
+ const tmpPath = path + '.tmp-' + process.pid + '-' + Date.now();
+ try {
+ mkdirSync(dirname(path), { recursive: true });
+ writeFileSync(tmpPath, JSON.stringify(record) + '\\n');
+ renameSync(tmpPath, path);
+ } catch (err) {
+ try {
+ unlinkSync(tmpPath);
+ } catch {}
+ if (reportedFailure) return;
+ reportedFailure = true;
+ parentPort.postMessage({ type: 'write-failed', code: errorCode(err) });
+ }
+}
+
+writeWitness();
+setInterval(writeWitness, tickMs);
+
+function errorCode(err) {
+ return err && typeof err.code === 'string' ? err.code : 'unknown';
+}
+`;
+
+export function isFileMissingError(err: unknown): boolean {
+ return (
+ typeof err === 'object' && err !== null && (err as NodeJS.ErrnoException).code === 'ENOENT'
+ );
+}
+
+function watchdogFailureFields(cause: unknown): Record<string, unknown> {
+ return cause instanceof Error ? { err: cause } : { exitCode: cause };
+}
+
+function isNonNegativeFinite(value: unknown): value is number {
+ return typeof value === 'number' && Number.isFinite(value) && value >= 0;
+}
+
+function isPositiveFinite(value: unknown): value is number {
+ return typeof value === 'number' && Number.isFinite(value) && value > 0;
+}
+
+export function parseWatchdogRecord(raw: string): WatchdogRecord | null {
+ let parsed: unknown;
+ try {
+ parsed = JSON.parse(raw);
+ } catch {
+ return null;
+ }
+ if (typeof parsed !== 'object' || parsed === null || Array.isArray(parsed)) return null;
+ const p = parsed as Record<string, unknown>;
+ if (p.schemaVersion !== 1) return null;
+ if (typeof p.bootId !== 'string' || p.bootId === '') return null;
+ if (typeof p.writtenAt !== 'string' || !Number.isFinite(Date.parse(p.writtenAt))) return null;
+ if (!isPositiveFinite(p.tickMs)) return null;
+ if (!isNonNegativeFinite(p.blockedForMs)) return null;
+ if (!isNonNegativeFinite(p.mainTicksObserved)) return null;
+ return {
+ schemaVersion: 1,
+ bootId: p.bootId,
+ writtenAt: p.writtenAt,
+ tickMs: p.tickMs,
+ blockedForMs: p.blockedForMs,
+ mainTicksObserved: p.mainTicksObserved,
+ };
+}
+
+export function stallThresholdMs(record: WatchdogRecord): number {
+ return record.tickMs * STALL_THRESHOLD_TICKS;
+}
+
+export function classifyPreviousLiveness(
+ read: WatchdogRead,
+ prevBootId: string | null,
+ prevLastAliveAtMs: number | null,
+): PreviousSessionLiveness {
+ if (read.kind === 'absent') return { kind: 'no-evidence', why: 'absent' };
+ if (read.kind === 'unreadable') return { kind: 'no-evidence', why: 'unreadable' };
+ const { record } = read;
+ if (prevBootId === null) return { kind: 'no-evidence', why: 'no-previous-boot' };
+ if (record.bootId !== prevBootId) return { kind: 'no-evidence', why: 'boot-mismatch' };
+ if (
+ prevLastAliveAtMs !== null &&
+ prevLastAliveAtMs - Date.parse(record.writtenAt) > stallThresholdMs(record)
+ ) {
+ return { kind: 'no-evidence', why: 'stale-witness' };
+ }
+ const threshold = stallThresholdMs(record);
+ const blocked = record.blockedForMs >= threshold;
+ if (!blocked && record.mainTicksObserved === 0) {
+ return { kind: 'no-evidence', why: 'unwitnessed-main' };
+ }
+ return {
+ kind: blocked ? 'blocked' : 'died',
+ blockedForMs: record.blockedForMs,
+ stallThresholdMs: threshold,
+ writtenAt: record.writtenAt,
+ };
+}
+
+export function livenessLogFields(liveness: PreviousSessionLiveness): LivenessLogFields {
+ if (liveness.kind === 'no-evidence') {
+ return {
+ livenessVerdict: null,
+ mainThreadBlockedForMs: null,
+ mainThreadStallThresholdMs: null,
+ livenessWitnessAt: null,
+ livenessEvidence: liveness.why,
+ };
+ }
+ return {
+ livenessVerdict: liveness.kind,
+ mainThreadBlockedForMs: liveness.blockedForMs,
+ mainThreadStallThresholdMs: liveness.stallThresholdMs,
+ livenessWitnessAt: liveness.writtenAt,
+ livenessEvidence: 'matched',
+ };
+}
+
+export function createMainThreadWatchdog(opts: {
+ path: string;
+ logger: WatchdogLogger;
+ tickMs?: number;
+}): MainThreadWatchdog {
+ const tickMs = opts.tickMs ?? WATCHDOG_TICK_MS;
+
+ return {
+ readPrevious(): WatchdogRead {
+ let raw: string;
+ try {
+ raw = readFileSync(opts.path, 'utf8');
+ } catch (err) {
+ if (isFileMissingError(err)) return { kind: 'absent' };
+ opts.logger.warn(
+ { event: 'main-thread-watchdog.read-failed', ...watchdogFailureFields(err) },
+ 'the previous session witness file could not be read',
+ );
+ return { kind: 'unreadable' };
+ }
+ const record = parseWatchdogRecord(raw);
+ if (record === null) {
+ opts.logger.warn(
+ { event: 'main-thread-watchdog.read-failed', bytes: raw.length },
+ 'the previous session witness file did not parse as a record',
+ );
+ return { kind: 'unreadable' };
+ }
+ return { kind: 'record', record };
+ },
+
+ start(bootId: string): MainThreadWatchdogHandle {
+ let worker: Worker;
+ try {
+ worker = new Worker(WORKER_SOURCE, {
+ eval: true,
+ workerData: { path: opts.path, bootId, tickMs },
+ });
+ } catch (err) {
+ opts.logger.warn(
+ { event: 'main-thread-watchdog.spawn-failed', ...watchdogFailureFields(err) },
+ 'could not start the main-thread watchdog — a hang will not be told apart from a death',
+ );
+ return { stop() {} };
+ }
+ worker.unref();
+ const ping = setInterval(() => {
+ worker.postMessage(1);
+ }, tickMs);
+ ping.unref();
+
+ let stopped = false;
+ let warnedWorkerFailed = false;
+ const warnWorkerFailed = (cause: Error | number): void => {
+ if (stopped || warnedWorkerFailed) return;
+ warnedWorkerFailed = true;
+ opts.logger.warn(
+ { event: 'main-thread-watchdog.worker-failed', ...watchdogFailureFields(cause) },
+ 'the main-thread watchdog stopped witnessing this session',
+ );
+ };
+ worker.on('error', (err) => {
+ warnWorkerFailed(err);
+ });
+ worker.on('exit', (code) => {
+ clearInterval(ping);
+ if (code !== 0) warnWorkerFailed(code);
+ });
+
+ let warnedWriteFailed = false;
+ worker.on('message', (message: unknown) => {
+ if (typeof message !== 'object' || message === null) return;
+ const { type, code } = message as { type?: unknown; code?: unknown };
+ if (type !== 'write-failed' || warnedWriteFailed) return;
+ warnedWriteFailed = true;
+ opts.logger.warn(
+ {
+ event: 'main-thread-watchdog.write-failed',
+ code: typeof code === 'string' ? code : 'unknown',
+ },
+ 'the main-thread watchdog could not write its witness file',
+ );
+ });
+
+ return {
+ stop() {
+ if (stopped) return;
+ stopped = true;
+ clearInterval(ping);
+ void worker.terminate();
+ },
+ };
+ },
+ };
+}
diff --git a/packages/desktop/tests/smoke/_helpers/platform-gate.ts b/packages/desktop/tests/smoke/_helpers/platform-gate.ts
index 62cf7719b..f3b58563d 100644
--- a/packages/desktop/tests/smoke/_helpers/platform-gate.ts
+++ b/packages/desktop/tests/smoke/_helpers/platform-gate.ts
@@ -32,6 +32,7 @@ export const SPEC_PLATFORM_GATES = {
'create-new-project.e2e.ts': ['!PLATFORM_SUPPORTED'],
'deep-link.e2e.ts': ['!DARWIN'],
'external-link.e2e.ts': ['!DARWIN'],
+ 'liveness-watchdog.e2e.ts': ['!PLATFORM_SUPPORTED'],
'mcp-wiring.e2e.ts': ['!PLATFORM_SUPPORTED', 'WINDOWS', "WINDOWS || TARGET.mode === 'packaged'"],
'navigator-close-on-open.e2e.ts': ['!PLATFORM_SUPPORTED'],
'navigator-return.e2e.ts': ['!PLATFORM_SUPPORTED'],
diff --git a/packages/desktop/tests/smoke/liveness-watchdog.e2e.ts b/packages/desktop/tests/smoke/liveness-watchdog.e2e.ts
new file mode 100644
index 000000000..9df3f07fc
--- /dev/null
+++ b/packages/desktop/tests/smoke/liveness-watchdog.e2e.ts
@@ -0,0 +1,157 @@
+import { spawnSync } from 'node:child_process';
+import { existsSync, mkdtempSync, readdirSync, readFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { type ElectronApplication, _electron as electron } from '@playwright/test';
+import { desktopLaunchOptions, resolveDesktopTarget } from './_helpers/launch-desktop';
+import {
+ homeEnv,
+ PLATFORM_SKIP_REASON,
+ PLATFORM_SUPPORTED,
+ SMOKE_ENABLED,
+ userDataDirFor,
+} from './_helpers/platform-gate';
+import { expect, test } from './_helpers/smoke-test';
+
+const TARGET = resolveDesktopTarget();
+
+const STALL_THRESHOLD_MS = 15_000;
+const FREEZE_MS = 90_000;
+
+type BootBreadcrumb = Record<string, unknown> & { event: 'crash-detection.boot'; time: string };
+
+interface WitnessRecord {
+ bootId: string;
+ blockedForMs: number;
+ mainTicksObserved: number;
+}
+
+function readBootBreadcrumbs(tmpHome: string): BootBreadcrumb[] {
+ const logsDir = join(tmpHome, '.ok', 'logs');
+ if (!existsSync(logsDir)) return [];
+ const lines = readdirSync(logsDir)
+ .filter((name) => name.startsWith('desktop.') && name.endsWith('.log'))
+ .flatMap((name) => readFileSync(join(logsDir, name), 'utf8').split('\n'));
+ const out: BootBreadcrumb[] = [];
+ for (const line of lines) {
+ if (!line.includes('crash-detection.boot')) continue;
+ try {
+ const parsed = JSON.parse(line) as Record<string, unknown>;
+ if (parsed.event === 'crash-detection.boot') out.push(parsed as BootBreadcrumb);
+ } catch {}
+ }
+ return out.sort((a, b) => Date.parse(a.time) - Date.parse(b.time));
+}
+
+function readWitness(tmpHome: string): WitnessRecord | null {
+ const path = join(userDataDirFor(tmpHome), 'bug-report-main-thread-liveness.json');
+ try {
+ return JSON.parse(readFileSync(path, 'utf8')) as WitnessRecord;
+ } catch {
+ return null;
+ }
+}
+
+async function launchIsolated(tmpHome: string): Promise<ElectronApplication> {
+ const app = await electron.launch(
+ desktopLaunchOptions({
+ target: TARGET,
+ args: [`--user-data-dir=${userDataDirFor(tmpHome)}`],
+ env: homeEnv(tmpHome),
+ timeout: 30_000,
+ }),
+ );
+ await app.firstWindow({ timeout: 20_000 });
+ return app;
+}
+
+async function waitForWitnessPings(tmpHome: string, minPings: number): Promise<void> {
+ await expect
+ .poll(() => readWitness(tmpHome)?.mainTicksObserved ?? -1, { timeout: 30_000 })
+ .toBeGreaterThanOrEqual(minPings);
+}
+
+function killProcessTree(app: ElectronApplication): Promise<void> {
+ const proc = app.process();
+ const exited = new Promise<void>((resolve) => proc.once('exit', () => resolve()));
+ if (process.platform === 'win32') {
+ spawnSync('taskkill', ['/pid', String(proc.pid), '/T', '/F'], { stdio: 'ignore' });
+ } else {
+ proc.kill('SIGKILL');
+ }
+ return exited;
+}
+
+function freezeMainThread(app: ElectronApplication, ms: number): { settled: () => boolean } {
+ let settled = false;
+ const markSettled = (): void => {
+ settled = true;
+ };
+ void app
+ .evaluate((_electron, freezeMs) => {
+ const end = Date.now() + freezeMs;
+ while (Date.now() < end) {}
+ }, ms)
+ .then(markSettled, markSettled);
+ return { settled: () => settled };
+}
+
+async function relaunchAndReadBreadcrumb(
+ tmpHome: string,
+ captureStderrFor: (app: ElectronApplication, opts?: { cleanupDirs?: readonly string[] }) => void,
+): Promise<BootBreadcrumb> {
+ const app = await launchIsolated(tmpHome);
+ captureStderrFor(app, { cleanupDirs: [tmpHome] });
+ let breadcrumb: BootBreadcrumb | undefined;
+ await expect(async () => {
+ breadcrumb = readBootBreadcrumbs(tmpHome).at(-1);
+ expect(breadcrumb).toBeDefined();
+ }).toPass({ timeout: 20_000 });
+ if (breadcrumb === undefined)
+ throw new Error('no crash-detection.boot breadcrumb after relaunch');
+ return breadcrumb;
+}
+
+test.describe('liveness watchdog separates a frozen main thread from a dead one (PRD-8439)', () => {
+ test.skip(!SMOKE_ENABLED, 'Set OK_DESKTOP_E2E_SMOKE=1 to run Electron smoke tests.');
+ test.skip(!PLATFORM_SUPPORTED, PLATFORM_SKIP_REASON);
+ test.skip(!TARGET.exists, TARGET.missingReason);
+
+ test('a main thread frozen past the stall threshold, then killed, boots as blocked', async ({
+ captureStderrFor,
+ }) => {
+ test.setTimeout(200_000);
+ const tmpHome = mkdtempSync(join(tmpdir(), 'ok-liveness-frozen-'));
+ const app = await launchIsolated(tmpHome);
+ captureStderrFor(app, { cleanupDirs: [tmpHome] });
+ await waitForWitnessPings(tmpHome, 1);
+
+ const freeze = freezeMainThread(app, FREEZE_MS);
+ await expect
+ .poll(() => readWitness(tmpHome)?.blockedForMs ?? 0, { timeout: 60_000 })
+ .toBeGreaterThanOrEqual(STALL_THRESHOLD_MS + 5_000);
+ expect(freeze.settled()).toBe(false);
+ await killProcessTree(app);
+
+ const breadcrumb = await relaunchAndReadBreadcrumb(tmpHome, captureStderrFor);
+ console.log('[liveness-watchdog] frozen-then-killed breadcrumb', JSON.stringify(breadcrumb));
+ expect(breadcrumb.dirtyShutdown).toBe(true);
+ expect(breadcrumb.livenessVerdict).toBe('blocked');
+ expect(breadcrumb.mainThreadBlockedForMs).toBeGreaterThanOrEqual(STALL_THRESHOLD_MS);
+ });
+
+ test('a main thread killed while responsive boots as died', async ({ captureStderrFor }) => {
+ test.setTimeout(120_000);
+ const tmpHome = mkdtempSync(join(tmpdir(), 'ok-liveness-died-'));
+ const app = await launchIsolated(tmpHome);
+ captureStderrFor(app, { cleanupDirs: [tmpHome] });
+ await waitForWitnessPings(tmpHome, 2);
+
+ await killProcessTree(app);
+
+ const breadcrumb = await relaunchAndReadBreadcrumb(tmpHome, captureStderrFor);
+ console.log('[liveness-watchdog] died breadcrumb', JSON.stringify(breadcrumb));
+ expect(breadcrumb.dirtyShutdown).toBe(true);
+ expect(breadcrumb.livenessVerdict).toBe('died');
+ });
+});