Build your personal knowledge base with Trilium Notes
TypeScript electronelectron-appknowledge-baseknowledge-graphknowledge-managementknowledge-management-graphlocal-firstnote-managmentnote-takernote-takingnotebooknotesnotes-apppersonal-knowledge-basepersonal-wikiscriptableself-hostedself-hostingwiki
fix(standalone): wait out stale OPFS handles instead of going in-memory
Claudefd58d7e9081c
diff --git a/.claude/skills/developing-standalone/SKILL.md b/.claude/skills/developing-standalone/SKILL.mdindex 7028e6bd3d..03a90b0136 100644--- a/.claude/skills/developing-standalone/SKILL.md+++ b/.claude/skills/developing-standalone/SKILL.md@@ -33,7 +33,7 @@ apps/standalone/1. **`main.ts`** requires a service worker (secure context: HTTPS or localhost — otherwise it throws a message explaining that), registers `./sw.js` with scope `/`, reloads once if the SW installed but doesn't control the page yet, and exposes `window.standaloneApi` (`restore.importBackup`, `backup.downloadDatabase`) — the client's escape hatch for things carrying a whole file, the twin of desktop's `window.electronApi`.2. **Leader election** (`leader_election.ts`): the OPFS SAHPool VFS takes *exclusive* sync-access handles, and `createSyncAccessHandle` exists only in a dedicated worker, so exactly one tab per origin can own the database. A **Web Lock** (`trilium-standalone-db`, held for the tab's lifetime) picks it; only the winner calls `startLocalServerWorker()`; when it closes the browser releases the lock and a waiting tab is promoted. Where Web Locks are missing (some WebViews) the tab assumes it is alone.3. **Service worker** (`sw.ts`): cache-first for hashed assets, network-first otherwise; requests under `LOCAL_API_PREFIXES` (`/bootstrap`, `/api/`, `/sync/`, `/search/`, from `local-bridge.ts`) are forwarded with `LOCAL_FETCH` to the **leader** client, found via a cached id, `WHO_IS_LEADER` / `LEADER_REPLY` probing, and one retry on `NOT_LEADER` (leadership can flip while a request is in flight). Backup downloads stream over a `MessageChannel` from the leader. It must **let `/_trilium_native_http/` fall through** (Android native proxy) and has a defensive guard for `capacitor://`.-4. **The worker** (`local-server-worker.ts`): installs the SAHPool VFS (falls back to an in-memory DB when OPFS is unavailable or in `TRILIUM_INTEGRATION_TEST=memory` builds, where it seeds a fixture), calls `initializeCore({...})` with the `lightweight/*` providers, registers `createConfiguredRouter()` (`BrowserRouter` mimicking Express, path params + query strings), then answers `LOCAL_REQUEST` messages. `DbLock` (`lightweight/db_lock.ts`) serialises `createAsyncRoute` handlers (imports) that hold a transaction across `await`s against the single connection — synchronous routes keep a zero-overhead path. `WS_MESSAGE` replaces the WebSocket in both directions. Anything that escapes after startup is reported as `WORKER_ERROR` for one background task; before startup it means the app never came up and `error-overlay.ts` paints it (the async worker init is invisible to `main.ts`'s try/catch).+4. **The worker** (`local-server-worker.ts`): waits for the previous worker's OPFS access handles to be released (`waitForSahPoolRelease` — a reload races the browser's asynchronous release, and `installOpfsSAHPoolVfs()` reacts to an acquisition failure by *deleting the pool directory*, so the pool must be acquirable before it runs), then installs the SAHPool VFS **once** — failure is a fatal startup error painted by `error-overlay.ts`, never an in-memory fallback (`TRILIUM_INTEGRATION_TEST=memory` builds seed a fixture into the pool, so writes survive reloads within a test). It then calls `initializeCore({...})` with the `lightweight/*` providers, registers `createConfiguredRouter()` (`BrowserRouter` mimicking Express, path params + query strings), then answers `LOCAL_REQUEST` messages. `DbLock` (`lightweight/db_lock.ts`) serialises `createAsyncRoute` handlers (imports) that hold a transaction across `await`s against the single connection — synchronous routes keep a zero-overhead path. `WS_MESSAGE` replaces the WebSocket in both directions. Anything that escapes after startup is reported as `WORKER_ERROR` for one background task; before startup it means the app never came up and `error-overlay.ts` paints it (the async worker init is invisible to `main.ts`'s try/catch).## The provider set (`src/lightweight/`)@@ -83,7 +83,7 @@ The rule: a dynamic import inside core splits a dep out **only if the dep has nopnpm standalone:start # vite devpnpm --filter standalone build # dist/ (needs NODE_OPTIONS=--max-old-space-size=4096, set by the script)pnpm --filter standalone start-prod # build + vite preview on :8888-TRILIUM_INTEGRATION_TEST=memory pnpm --filter standalone build # in-memory DB seeded from a fixture (what e2e uses)+TRILIUM_INTEGRATION_TEST=memory pnpm --filter standalone build # OPFS DB seeded from a fixture (what e2e uses)pnpm --filter standalone e2e # Playwright: multi-tab leader handoff, extra windows```diff --git a/apps/standalone/src/lightweight/sql_provider.spec.ts b/apps/standalone/src/lightweight/sql_provider.spec.tsindex ba841223aa..ceeaf778ee 100644--- a/apps/standalone/src/lightweight/sql_provider.spec.ts+++ b/apps/standalone/src/lightweight/sql_provider.spec.ts@@ -2,7 +2,7 @@ import { getSql } from "@triliumnext/core";import type { Statement } from "@triliumnext/core";import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";-import BrowserSqlProvider from "./sql_provider.js";+import BrowserSqlProvider, { waitForSahPoolRelease } from "./sql_provider.js";// The sqlite-wasm module can only be initialized once per worker (test_setup.ts// already does it for core). Reuse that initialized module rather than calling@@ -539,3 +539,84 @@ describe("BrowserSqlProvider error and defensive paths", () => {expect([...iterStmt.iterate()]).toEqual([undefined]);});});++describe("waitForSahPoolRelease", () => {+ /** A fake OPFS root holding `.opfs-sahpool/.opaque` with the given entries. */+ function fakeRootWith(entries: [string, { kind: string; createSyncAccessHandle?: () => Promise<{ close(): void }> }][]) {+ const opaque = {+ async *entries() {+ yield* entries;+ }+ };+ const pool = {+ getDirectoryHandle: async (name: string) => {+ if (name !== ".opaque") throw new DOMException(name, "NotFoundError");+ return opaque;+ }+ };+ const root = {+ getDirectoryHandle: async (name: string) => {+ if (name !== ".opfs-sahpool") throw new DOMException(name, "NotFoundError");+ return pool;+ }+ };+ return async () => root as unknown as FileSystemDirectoryHandle;+ }++ it("resolves immediately when the pool does not exist yet", async () => {+ const emptyRoot = async () => ({+ getDirectoryHandle: async (name: string) => {+ throw new DOMException(name, "NotFoundError");+ }+ }) as unknown as FileSystemDirectoryHandle;++ await expect(waitForSahPoolRelease({ getRoot: emptyRoot })).resolves.toBe(true);+ });++ it("probes every file, skipping directories, and closes the probe handles", async () => {+ const closed: string[] = [];+ const file = (name: string) => ({+ kind: "file",+ createSyncAccessHandle: async () => ({ close: () => closed.push(name) })+ });+ const getRoot = fakeRootWith([+ ["subdir", { kind: "directory" }],+ ["pool-1", file("pool-1")],+ ["pool-2", file("pool-2")]+ ]);++ await expect(waitForSahPoolRelease({ getRoot })).resolves.toBe(true);+ expect(closed).toEqual(["pool-1", "pool-2"]);+ });++ it("waits for a held file's release, and reports one still held at the deadline", async () => {+ let denials = 3;+ const eventuallyReleased = {+ kind: "file",+ createSyncAccessHandle: async () => {+ if (denials > 0) {+ denials--;+ throw new DOMException("held", "NoModificationAllowedError");+ }+ return { close: () => {} };+ }+ };+ await expect(waitForSahPoolRelease({+ getRoot: fakeRootWith([["pool-1", eventuallyReleased]]),+ pollMs: 1+ })).resolves.toBe(true);+ expect(denials).toBe(0);++ const neverReleased = {+ kind: "file",+ createSyncAccessHandle: async () => {+ throw new DOMException("held", "NoModificationAllowedError");+ }+ };+ await expect(waitForSahPoolRelease({+ getRoot: fakeRootWith([["pool-1", neverReleased]]),+ pollMs: 1,+ deadlineMs: 25+ })).resolves.toBe(false);+ });+});diff --git a/apps/standalone/src/lightweight/sql_provider.ts b/apps/standalone/src/lightweight/sql_provider.tsindex 7aa00c48d2..a7cf6948f9 100644--- a/apps/standalone/src/lightweight/sql_provider.ts+++ b/apps/standalone/src/lightweight/sql_provider.ts@@ -674,3 +674,88 @@ export default class BrowserSqlProvider implements DatabaseProvider {}}}++/**+ * Where `installOpfsSAHPoolVfs()` keeps the pool files under the default options+ * {@link BrowserSqlProvider.installSahPool} uses: `"." + name` under the OPFS root, with the+ * files in its `.opaque` subdirectory.+ */+const SAH_POOL_OPAQUE_PATH = ".opfs-sahpool/.opaque";++export interface SahPoolReleaseWait {+ /** How long to keep probing before reporting the pool as still held. */+ deadlineMs?: number;+ /** Pause between two probe rounds. */+ pollMs?: number;+ /** Stands in for `navigator.storage.getDirectory` in tests. */+ getRoot?: () => Promise<FileSystemDirectoryHandle>;+}++/**+ * Waits until no file of the SAHPool directory is held by another context.+ *+ * A page reload starts a new worker while the browser is still releasing the previous worker's+ * exclusive OPFS access handles, so an immediate {@link BrowserSqlProvider.installSahPool} can+ * find the pool files locked. That failure must not simply be retried:+ * `installOpfsSAHPoolVfs()` reacts to it by deleting the pool directory, database included, so+ * the pool has to be acquirable before it runs at all.+ *+ * Resolves `true` once every pool file accepts a (immediately closed) probe access handle, and+ * without waiting when the pool does not exist yet or the OPFS API is missing — in both cases+ * `installSahPool()` is the one with the authoritative answer. Resolves `false` when a file is+ * still held after `deadlineMs`.+ */+export async function waitForSahPoolRelease({ deadlineMs = 15_000, pollMs = 150, getRoot }: SahPoolReleaseWait = {}): Promise<boolean> {+ if (!getRoot) {+ if (!navigator?.storage?.getDirectory) {+ return true;+ }+ getRoot = () => navigator.storage.getDirectory();+ }++ let dir: FileSystemDirectoryHandle;+ try {+ dir = await getRoot();+ for (const segment of SAH_POOL_OPAQUE_PATH.split("/")) {+ dir = await dir.getDirectoryHandle(segment);+ }+ } catch {+ return true;+ }++ const deadline = Date.now() + deadlineMs;+ let announced = false;+ for (;;) {+ const heldFile = await findHeldPoolFile(dir);+ if (!heldFile) {+ if (announced) {+ console.log("[BrowserSqlProvider] The SAHPool files have been released");+ }+ return true;+ }+ if (!announced) {+ console.log(`[BrowserSqlProvider] Pool file "${heldFile}" is still held, usually by the worker of the page this one replaced — waiting for its release...`);+ announced = true;+ }+ if (Date.now() >= deadline) {+ return false;+ }+ await new Promise(resolve => setTimeout(resolve, pollMs));+ }+}++/** The name of a pool file whose exclusive access handle another context still holds, if any. */+async function findHeldPoolFile(dir: FileSystemDirectoryHandle): Promise<string | null> {+ for await (const [name, handle] of dir.entries()) {+ if (handle.kind !== "file") {+ continue;+ }+ try {+ const probe = await (handle as FileSystemFileHandle).createSyncAccessHandle();+ probe.close();+ } catch {+ return name;+ }+ }+ return null;+}diff --git a/apps/standalone/src/local-server-worker.ts b/apps/standalone/src/local-server-worker.tsindex 1cb8efd0c0..10bd15e8c1 100644--- a/apps/standalone/src/local-server-worker.ts+++ b/apps/standalone/src/local-server-worker.ts@@ -97,6 +97,7 @@ let removeBackupLeftovers: typeof import('./lightweight/backup_provider').removelet StandaloneInAppHelpProvider: typeof import('./lightweight/in_app_help_provider').default;let translationProvider: typeof import('./lightweight/translation_provider').default;let createConfiguredRouter: typeof import('./lightweight/browser_routes').createConfiguredRouter;+let waitForSahPoolRelease: typeof import('./lightweight/sql_provider').waitForSahPoolRelease;// Instance statelet sqlProvider: InstanceType<typeof BrowserSqlProvider> | null = null;@@ -129,11 +130,7 @@ function assertSqliteMagic(buffer: Uint8Array, source: string): void {* Load the test fixture database for integration tests.* Seeds from the fixture if not already present.*/-async function loadTestDatabase(sahPoolAvailable: boolean, dbName: string): Promise<void> {- if (!sahPoolAvailable) {- throw new Error("SAHPool is required for integration tests.");- }-+async function loadTestDatabase(dbName: string): Promise<void> {const poolFiles = sqlProvider!.sahPool!.getFileNames();if (!poolFiles.includes(dbName)) {console.log("[Worker] Integration test mode: seeding fixture database into SAHPool...");@@ -191,6 +188,7 @@ async function loadModules(): Promise<void> {]);BrowserSqlProvider = sqlModule.default;+ waitForSahPoolRelease = sqlModule.waitForSahPoolRelease;WorkerMessagingProvider = messagingModule.default;BrowserExecutionContext = clsModule.default;BrowserCryptoProvider = cryptoModule.default;@@ -231,6 +229,14 @@ async function initialize(): Promise<void> {// First, load all modules dynamicallyawait loadModules();+ // A reload starts this worker while the browser is still releasing the previous+ // worker's exclusive OPFS access handles, so the boot waits for the database pool+ // to be free — before the log service, whose own OPFS file is held the same way.+ const poolReleased = await waitForSahPoolRelease();+ if (!poolReleased) {+ console.warn("[Worker] The SAHPool files are still held by another context; opening the database is expected to fail.");+ }+// Initialize log service as early as possible so subsequent// initialization steps are persisted to the OPFS log file.const logService = new StandaloneLogService();@@ -241,13 +247,20 @@ async function initialize(): Promise<void> {logService.info("[Worker] Initializing SQLite WASM...");await sqlProvider!.initWasm();- // Try to install the SAHPool VFS (preferred: supports WAL, much faster)- let sahPoolAvailable = false;+ // A failure here ends the boot: throwing from initialize() reaches the page as+ // WORKER_ERROR and error-overlay.ts paints it. Opening an in-memory database+ // instead would boot an empty instance whose notes vanish on the next reload.try {await sqlProvider!.installSahPool();- sahPoolAvailable = true;} catch (e) {- logService.info(`[Worker] SAHPool VFS not available, will fall back to in-memory: ${e}`);+ const reason = e instanceof Error ? e.message : String(e);+ throw new Error(+ "The notes database could not be opened, because the browser did not grant "+ + `exclusive access to its persistent storage (OPFS): ${reason}\n\n`+ + "Close any other window running Trilium and reload this page. Your notes are "+ + "not affected. If the browser does not support OPFS sync access handles at "+ + "all, Trilium cannot run in it."+ );}// Integration test mode is baked in at build time via the@@ -267,15 +280,10 @@ async function initialize(): Promise<void> {// Playwright gives each test a fresh BrowserContext, which means a// fresh OPFS — so on the first worker init of a test we seed from// the fixture, and subsequent inits in the same test reuse it.- await loadTestDatabase(sahPoolAvailable, dbName);- } else if (sahPoolAvailable) {- logService.info("[Worker] SAHPool available, loading persistent database (WAL mode)...");- sqlProvider!.loadFromSahPool(dbName);+ await loadTestDatabase(dbName);} else {- // SAHPool only needs a Worker + OPFS API, so reaching this- // branch means the environment lacks OPFS entirely.- logService.info("[Worker] OPFS not available, using in-memory database (data will not persist)");- sqlProvider!.loadFromMemory();+ logService.info("[Worker] Loading persistent database from SAHPool (WAL mode)...");+ sqlProvider!.loadFromSahPool(dbName);}logService.info("[Worker] Database loaded");