Skip to content

armgit / macro

Public Imported from GitHub
0

Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.

Seanaye/test/cache startup retry (#8921)

Sean Ayeb94c22648792
diff --git a/apps/web/AGENTS.md b/apps/web/AGENTS.md
index 0b32b40b49..fdb72f1d5c 100644
--- a/apps/web/AGENTS.md
+++ b/apps/web/AGENTS.md
@@ -194,7 +194,9 @@ ownership. Cancel, logout, and the ten-minute timeout retire pending callbacks.
even with retained user data; preserve session state on transient failures.
- Treat a repeated worker-ready frame as a coordinator restart: retire the old
engine and re-register in place, preserving durable storage and never replaying
- uncertain enqueues. See `docs/graphql-normalized-cache-plan.md`.
+ uncertain enqueues. Space failed coordinator registrations with bounded,
+ cancelable waits; a missing startup frame is not proof of untouched storage.
+ See `docs/graphql-normalized-cache-plan.md`.
- Reject already-routed optimistic enqueues with missing replies as uncertain on
every owner departure; never forward them through network fallback, because
their durable queue entries may survive the old engine.
diff --git a/apps/web/docs/graphql-normalized-cache-plan.md b/apps/web/docs/graphql-normalized-cache-plan.md
index d71f3ec98e..ba5068f2ea 100644
--- a/apps/web/docs/graphql-normalized-cache-plan.md
+++ b/apps/web/docs/graphql-normalized-cache-plan.md
@@ -303,8 +303,17 @@ Startup phase budgets are relayed to every page, including late joiners: asset
loading gets five minutes, taking the lock gets the retry window plus five
seconds, and database opening gets twenty seconds, plus a five-second page
response grace. Ordinary read deadlines remain ten seconds. Coordinator
-registration gets sixty seconds per attempt, with three attempts; engine recovery
-retains its bounded five retries. Exhausted bootstrap retries preserve the cache
+registration gets sixty seconds per attempt, with three attempts separated by
+100 ms and 200 ms waits. Safari can abort a shared worker's download when its
+creating page leaves, then briefly reuse that failed startup for immediate
+retries. These waits let the browser retire it without changing the worker name,
+URL, or storage scope. Navigation and disposal cancel the wait; no new adapter
+is created by a canceled initialization, even if its timer already fired.
+`graphql_cache.registration_retry` reports the wait duration, retry number,
+triggering error class, and whether the wait completed or was canceled. This is
+separate from a repeated-ready restart and does not make generic transport
+failures storage-safe. Engine recovery retains its bounded five retries.
+Exhausted bootstrap retries preserve the cache
scope only when the coordinator explicitly proves storage was untouched, not from
a page's last-seen phase. Missing/broken transports remain conservatively
uncertain. Pages retry proven untouched bootstrap and owner-lock failures on reconnect,
diff --git a/apps/web/src/lib/graphql-cache/host/registration-retry.test.ts b/apps/web/src/lib/graphql-cache/host/registration-retry.test.ts
new file mode 100644
index 0000000000..e3bce7823f
--- /dev/null
+++ b/apps/web/src/lib/graphql-cache/host/registration-retry.test.ts
@@ -0,0 +1,286 @@
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+import {
+ type CacheRequest,
+ INITIAL_CACHE_REVISION,
+ type WorkerMessage,
+} from '../protocol';
+import type { CacheTelemetryObservation } from '../telemetry';
+import type {
+ CacheCoordinatorPageAdapterOptions,
+ PageAdapterDisposeOptions,
+} from '../worker/coordinator-page-adapter';
+import { CacheBootstrapExhaustedError } from '../worker/startup';
+import { CacheNavigationError } from './navigation-error';
+
+const adapterFactory = vi.hoisted(() => vi.fn());
+vi.mock('../worker/coordinator-page-adapter', () => ({
+ createCacheCoordinatorPageAdapter: adapterFactory,
+}));
+
+import { createWorkerCacheHost } from './worker-host';
+
+class RegistrationAdapter {
+ onmessage: ((event: MessageEvent<WorkerMessage>) => void) | null = null;
+ readonly requests: CacheRequest[] = [];
+ readonly start = vi.fn(async () => {});
+ readonly dispose = vi.fn(
+ async (_options: PageAdapterDisposeOptions = {}) => {}
+ );
+
+ constructor(readonly options: CacheCoordinatorPageAdapterOptions) {}
+
+ postMessage(request: CacheRequest): void {
+ this.requests.push(request);
+ this.onmessage?.(
+ new MessageEvent('message', {
+ data: {
+ id: request.id,
+ ok: true,
+ result: request.kind === 'init' ? null : INITIAL_CACHE_REVISION,
+ },
+ })
+ );
+ }
+}
+
+const scope = 'cache-parking-registration-retry';
+const loadFailure = new Error('SharedWorker transport error');
+let available: boolean;
+let adapters: RegistrationAdapter[];
+let observations: CacheTelemetryObservation[];
+const onInitializationError = vi.fn<(error: Error) => void>();
+let host: ReturnType<typeof createWorkerCacheHost>;
+
+beforeEach(() => {
+ vi.useFakeTimers();
+ localStorage.clear();
+ localStorage.setItem('graphql-cache:scope', scope);
+ vi.stubGlobal('SharedWorker', vi.fn());
+ vi.stubGlobal('Worker', vi.fn());
+ vi.stubGlobal('MessageChannel', vi.fn());
+ vi.stubGlobal('navigator', {
+ storage: { getDirectory: vi.fn() },
+ locks: {
+ request: vi.fn(
+ async (_name: string, _options: LockOptions, callback: () => unknown) =>
+ callback()
+ ),
+ },
+ });
+ available = false;
+ adapters = [];
+ observations = [];
+ onInitializationError.mockReset();
+ adapterFactory.mockReset();
+ adapterFactory.mockImplementation(
+ (options: CacheCoordinatorPageAdapterOptions) => {
+ const adapter = new RegistrationAdapter(options);
+ adapter.start.mockImplementation(async () => {
+ if (available) return;
+ options.onTerminalError?.(loadFailure);
+ throw loadFailure;
+ });
+ adapters.push(adapter);
+ return adapter;
+ }
+ );
+ host = createWorkerCacheHost({
+ scope,
+ onInitializationError,
+ telemetry: {
+ record: (observation) => observations.push(observation),
+ flush: vi.fn(),
+ },
+ });
+});
+
+afterEach(() => {
+ host.dispose();
+ vi.useRealTimers();
+ vi.unstubAllGlobals();
+ vi.restoreAllMocks();
+});
+
+describe('coordinator registration retry', () => {
+ it('recovers when the failed startup becomes reusable between attempts', async () => {
+ const outcome = host.currentRevision().then(
+ (revision) => ({ revision }),
+ (error: unknown) => ({ error })
+ );
+ await vi.advanceTimersByTimeAsync(0);
+ available = true;
+ await vi.advanceTimersByTimeAsync(100);
+
+ await expect(outcome).resolves.toEqual({
+ revision: INITIAL_CACHE_REVISION,
+ });
+ expect(adapters).toHaveLength(2);
+ expect(adapters[0].dispose).toHaveBeenCalledExactlyOnceWith({
+ graceful: false,
+ });
+ expect(adapters[0].requests).toEqual([]);
+ expect(adapters[1].requests.map(({ kind }) => kind)).toEqual([
+ 'init',
+ 'current-revision',
+ ]);
+ expect(adapters.every((adapter) => adapter.options.scope === scope)).toBe(
+ true
+ );
+ expect(localStorage.getItem('graphql-cache:scope')).toBe(scope);
+ expect(onInitializationError).not.toHaveBeenCalled();
+ expect(observations).toContainEqual(
+ expect.objectContaining({
+ name: 'graphql_cache.registration_retry',
+ outcome: 'success',
+ count: 1,
+ durationMs: 100,
+ })
+ );
+ });
+
+ it('spaces both retries and exhausts once without admitting or replaying a mutation', async () => {
+ const mutation = host.enqueueOptimisticMutation(
+ {
+ uuid: '00000000-0000-4000-8000-000000000100',
+ query: 'mutation Update { update }',
+ data: { update: true },
+ },
+ { owner: 'runner', nowMs: 1, leaseExpiresAtMs: 101 }
+ );
+ const outcomes = Promise.all([
+ host.currentRevision().catch((error: unknown) => error),
+ mutation.catch((error: unknown) => error),
+ ]);
+ await vi.advanceTimersByTimeAsync(0);
+ expect(adapters).toHaveLength(1);
+ await vi.advanceTimersByTimeAsync(99);
+ expect(adapters).toHaveLength(1);
+ await vi.advanceTimersByTimeAsync(1);
+ expect(adapters).toHaveLength(2);
+ await vi.advanceTimersByTimeAsync(199);
+ expect(adapters).toHaveLength(2);
+ await vi.advanceTimersByTimeAsync(1);
+ expect(adapters).toHaveLength(3);
+ await expect(outcomes).resolves.toEqual([loadFailure, loadFailure]);
+ expect(onInitializationError).toHaveBeenCalledExactlyOnceWith(loadFailure);
+ expect(onInitializationError.mock.calls[0][0]).not.toBeInstanceOf(
+ CacheBootstrapExhaustedError
+ );
+ expect(adapters.flatMap((adapter) => adapter.requests)).toEqual([]);
+ await expect(host.currentRevision()).rejects.toBe(loadFailure);
+ await vi.advanceTimersByTimeAsync(1_000);
+ expect(adapters).toHaveLength(3);
+ expect(localStorage.getItem('graphql-cache:scope')).toBe(scope);
+ expect(vi.getTimerCount()).toBe(0);
+ expect(
+ observations.filter(
+ ({ name }) => name === 'graphql_cache.registration_retry'
+ )
+ ).toMatchObject([
+ { outcome: 'success', count: 1, durationMs: 100 },
+ { outcome: 'success', count: 2, durationMs: 200 },
+ ]);
+ });
+
+ it.each([0, 100])(
+ 'cancels disposal during the wait starting at %s ms',
+ async (elapsed) => {
+ const outcome = host.currentRevision().catch((error: unknown) => error);
+ await vi.advanceTimersByTimeAsync(elapsed);
+ expect(adapters).toHaveLength(elapsed === 0 ? 1 : 2);
+ const attempts = adapters.length;
+ host.dispose();
+ await expect(outcome).resolves.toEqual(
+ expect.objectContaining({
+ message: 'cache worker host was disposed during startup',
+ })
+ );
+ await vi.advanceTimersByTimeAsync(1_000);
+ expect(adapters).toHaveLength(attempts);
+ expect(onInitializationError).not.toHaveBeenCalled();
+ expect(vi.getTimerCount()).toBe(0);
+ expect(observations).toContainEqual(
+ expect.objectContaining({
+ name: 'graphql_cache.registration_retry',
+ outcome: 'abrupt',
+ durationMs: 0,
+ })
+ );
+ }
+ );
+
+ it.each([0, 100])(
+ 'cancels pagehide during the wait at %s ms and restores without an orphan retry',
+ async (elapsed) => {
+ const outcome = host.currentRevision().catch((error: unknown) => error);
+ await vi.advanceTimersByTimeAsync(elapsed);
+ expect(adapters).toHaveLength(elapsed === 0 ? 1 : 2);
+ const attempts = adapters.length;
+ dispatchEvent(new PageTransitionEvent('pagehide', { persisted: true }));
+ available = true;
+ // Restore before the old startup's catch/finally has unwound.
+ dispatchEvent(new PageTransitionEvent('pageshow', { persisted: true }));
+ await expect(outcome).resolves.toBeInstanceOf(CacheNavigationError);
+ await expect(host.currentRevision()).resolves.toBe(
+ INITIAL_CACHE_REVISION
+ );
+ await vi.advanceTimersByTimeAsync(1_000);
+ expect(adapters).toHaveLength(attempts + 1);
+ expect(onInitializationError).not.toHaveBeenCalled();
+ expect(localStorage.getItem('graphql-cache:scope')).toBe(scope);
+ }
+ );
+
+ it('does not construct another adapter if disposed after the delay fires but before its continuation', async () => {
+ const outcome = host.currentRevision().catch((error: unknown) => error);
+ await vi.advanceTimersByTimeAsync(0);
+ expect(adapters).toHaveLength(1);
+ vi.advanceTimersByTime(100);
+ host.dispose();
+ await expect(outcome).resolves.toEqual(
+ expect.objectContaining({
+ message: 'cache worker host was disposed during startup',
+ })
+ );
+ await vi.advanceTimersByTimeAsync(1_000);
+ expect(adapters).toHaveLength(1);
+ expect(onInitializationError).not.toHaveBeenCalled();
+ expect(vi.getTimerCount()).toBe(0);
+ });
+
+ it('gives the next attempt its own timeout and ignores late success from the retired attempt', async () => {
+ host.dispose();
+ host = createWorkerCacheHost({
+ scope,
+ initializationTimeoutMs: 10,
+ onInitializationError,
+ });
+ const first = Promise.withResolvers<void>();
+ const second = Promise.withResolvers<void>();
+ adapterFactory.mockImplementation(
+ (options: CacheCoordinatorPageAdapterOptions) => {
+ const adapter = new RegistrationAdapter(options);
+ adapter.start.mockImplementation(() =>
+ adapters.length === 1 ? first.promise : second.promise
+ );
+ adapters.push(adapter);
+ return adapter;
+ }
+ );
+ const result = host.currentRevision().then(
+ (revision) => ({ revision }),
+ (error: unknown) => ({ error })
+ );
+ await vi.advanceTimersByTimeAsync(10);
+ first.resolve();
+ await vi.advanceTimersByTimeAsync(99);
+ expect(adapters).toHaveLength(1);
+ expect(adapters[0].requests).toEqual([]);
+ await vi.advanceTimersByTimeAsync(10);
+ expect(adapters).toHaveLength(2);
+ second.resolve();
+ await expect(result).resolves.toEqual({ revision: INITIAL_CACHE_REVISION });
+ expect(adapters[0].requests).toEqual([]);
+ expect(onInitializationError).not.toHaveBeenCalled();
+ });
+});
diff --git a/apps/web/src/lib/graphql-cache/host/worker-host.test.ts b/apps/web/src/lib/graphql-cache/host/worker-host.test.ts
index d320b1c71f..6dff0c1e77 100644
--- a/apps/web/src/lib/graphql-cache/host/worker-host.test.ts
+++ b/apps/web/src/lib/graphql-cache/host/worker-host.test.ts
@@ -677,7 +677,9 @@ describe('createWorkerCacheHost', () => {
const rejected = expect(host.currentRevision()).rejects.toThrow(
'cache worker timeout: registration'
);
- await vi.advanceTimersByTimeAsync(COORDINATOR_CONNECT_TIMEOUT_MS * 3 + 1);
+ await vi.advanceTimersByTimeAsync(
+ COORDINATOR_CONNECT_TIMEOUT_MS * 3 + 100 + 200 + 1
+ );
await rejected;
expect(adapterFactory).toHaveBeenCalledTimes(3);
expect(onInitializationError).toHaveBeenCalledOnce();
diff --git a/apps/web/src/lib/graphql-cache/host/worker-host.ts b/apps/web/src/lib/graphql-cache/host/worker-host.ts
index f8832f0793..25aa5ff26a 100644
--- a/apps/web/src/lib/graphql-cache/host/worker-host.ts
+++ b/apps/web/src/lib/graphql-cache/host/worker-host.ts
@@ -67,6 +67,7 @@ import { isUnambiguousCacheScope } from '../worker/stale-databases';
import {
CacheBootstrapExhaustedError,
COORDINATOR_CONNECT_ATTEMPTS,
+ COORDINATOR_CONNECT_RETRY_DELAYS_MS,
COORDINATOR_CONNECT_TIMEOUT_MS,
STARTUP_RESPONSE_GRACE_MS,
} from '../worker/startup';
@@ -594,6 +595,11 @@ export function createWorkerCacheHost(options: WorkerHostOptions): CacheHost {
attempt < COORDINATOR_CONNECT_ATTEMPTS;
attempt += 1
) {
+ // Cancellation may arrive after a retry timer fires, before this continuation.
+ if (initializationId !== initializationAttempt)
+ throw new CacheNavigationError();
+ if (state !== 'initializing')
+ throw new Error('cache worker host was disposed during startup');
const current = getAdapter();
let timer: ReturnType<typeof setTimeout> | undefined;
try {
@@ -624,6 +630,28 @@ export function createWorkerCacheHost(options: WorkerHostOptions): CacheHost {
attempt + 1 === COORDINATOR_CONNECT_ATTEMPTS
)
throw error;
+ if (timer !== undefined) clearTimeout(timer);
+ const retryStartedAt = now();
+ let completed = false;
+ try {
+ await new Promise<void>((resolve, reject) => {
+ cancelRegistration = reject;
+ timer = setTimeout(
+ resolve,
+ COORDINATOR_CONNECT_RETRY_DELAYS_MS[attempt]
+ );
+ });
+ completed = true;
+ } finally {
+ telemetry.record({
+ name: 'graphql_cache.registration_retry',
+ operationCategory: 'initialization',
+ outcome: completed ? 'success' : 'abrupt',
+ errorCode: classifyCacheError(error),
+ count: attempt + 1,
+ durationMs: now() - retryStartedAt,
+ });
+ }
} finally {
if (timer !== undefined) clearTimeout(timer);
cancelRegistration = undefined;
diff --git a/apps/web/src/lib/graphql-cache/telemetry.ts b/apps/web/src/lib/graphql-cache/telemetry.ts
index fd5132bc58..7d17b5c6ca 100644
--- a/apps/web/src/lib/graphql-cache/telemetry.ts
+++ b/apps/web/src/lib/graphql-cache/telemetry.ts
@@ -9,6 +9,7 @@ export const CACHE_TELEMETRY_EVENT_NAMES = [
'graphql_cache.schema_init',
'graphql_cache.db_ready',
'graphql_cache.host_ready',
+ 'graphql_cache.registration_retry',
'graphql_cache.host_request',
'graphql_cache.coordinator_request',
'graphql_cache.engine_request',
diff --git a/apps/web/src/lib/graphql-cache/worker/startup.ts b/apps/web/src/lib/graphql-cache/worker/startup.ts
index bba181a6af..fde293086e 100644
--- a/apps/web/src/lib/graphql-cache/worker/startup.ts
+++ b/apps/web/src/lib/graphql-cache/worker/startup.ts
@@ -21,7 +21,13 @@ export const OWNER_LOCK_WAIT_TIMEOUT_MS =
OWNER_LOCK_RETRY_DELAYS_MS.reduce((total, delay) => total + delay, 0) + 5_000;
/** Lets the coordinator's phase timeout/retry reach a page before its guard fires. */
export const STARTUP_RESPONSE_GRACE_MS = 5_000;
-export const COORDINATOR_CONNECT_ATTEMPTS = 3;
+/** Safari can briefly reuse a failed SharedWorker startup after its creator
+ * leaves during download. Let that entry retire before consuming another try. */
+export const COORDINATOR_CONNECT_RETRY_DELAYS_MS: readonly number[] = [
+ 100, 200,
+];
+export const COORDINATOR_CONNECT_ATTEMPTS =
+ COORDINATOR_CONNECT_RETRY_DELAYS_MS.length + 1;
/** Only the live coordinator can prove that every failed attempt stayed before
* the open grant. A missing progress message is not proof of untouched storage.
diff --git a/docs/agent-guide/navigation.md b/docs/agent-guide/navigation.md
index 0ff0c72c0b..ef5b52d47e 100644
--- a/docs/agent-guide/navigation.md
+++ b/docs/agent-guide/navigation.md
@@ -76,6 +76,18 @@ optimistic enqueues must not be sent twice. A `graphql_cache.owner` event with
`ownerEvent: coordinator-restarted` distinguishes this from engine replacement
or a BFCache restore.
+Separately, leaving the creating page while its SharedWorker is still downloading
+can fail the survivor before its first ready frame. In an isolated Safari fixture,
+hold that download, connect a second page, and navigate the creator through a
+normal link. Release the asset and verify the survivor initializes on its bounded
+registration retries without a reload. Retries wait 100 ms and 200 ms; navigating
+or disposing the survivor during either wait must stop further constructors.
+Verify BFCache restoration starts fresh without resurrecting the canceled retry.
+Use `graphql_cache.registration_retry` to inspect wait timing and cancellation;
+preserve the scope, existing queue, and conservative storage-failure handling.
+Do not inject this failure into a live account or mistake it for a repeated-ready
+loop.
+
The same write-safety rule applies when an owner navigates away without restarting
the coordinator: a surviving tab's enqueue may be committed even if its reply was
lost. It must report an uncertain outcome, not retry or fall through to a second