Skip to content

armgit / macro

Public Imported from GitHub
0

Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.

Seanaye/fix/cache startup timeout (#6544)

Sean Aye8ad43ccdf401
diff --git a/apps/web/docs/graphql-normalized-cache-plan.md b/apps/web/docs/graphql-normalized-cache-plan.md
index c0435e7121..97fe764407 100644
--- a/apps/web/docs/graphql-normalized-cache-plan.md
+++ b/apps/web/docs/graphql-normalized-cache-plan.md
@@ -189,8 +189,23 @@ The cache's validated envelopes, request ids, Web Locks, owner epochs,
heartbeats, and replay policy remain application-owned. A normal `pagehide`
uses a fenced navigation-departure message: the page terminates its dedicated
worker, but the next owner opens the existing OPFS database. Only an
-unannounced liveness, heartbeat, transport, or engine failure selects
-`wipe-before-open`. Browsers missing the required worker, lock, or OPFS
+unannounced liveness, heartbeat, transport, or engine failure **after the database
+open grant** selects `wipe-before-open`.
+
+Protocol v3 separates startup into asset loading and database opening. The engine
+loads/compiles WASM without touching OPFS, reports `engine-assets-ready`, and waits
+for the coordinator's `open-engine` grant. The coordinator records that storage may
+be touched before sending this grant. Pre-grant failures preserve the current
+open/reset requirement; an earlier required wipe is never forgotten. Startup phase
+budgets are relayed to every page, including late joiners: asset loading gets five
+minutes and database opening gets twenty seconds, plus a five-second page response
+grace. Ordinary read deadlines remain ten seconds. Coordinator registration gets
+sixty seconds per attempt, with three attempts; engine recovery retains its bounded
+five retries. Exhausted bootstrap retries preserve the cache scope only when the
+coordinator explicitly proves storage was untouched, not from a page's last-seen
+phase. Missing/broken transports remain conservatively uncertain.
+
+Browsers missing the required worker, lock, or OPFS
capabilities use a storage-free no-op cache host. Tauri
detection selects the native transport before browser capability checks. All
paths remain behind the same `CacheHost` interface.
@@ -294,6 +309,13 @@ apps/web/src/lib/graphql-cache/ # JS glue
- Turso backend for both Tauri native filesystem storage and browser OPFS:
WAL mode, batch transactions, and physical reset on incompatible storage;
tested natively and in headless Chromium.
+- Recovery browser fixtures require a matched production/test-hook WASM pair.
+ Run `just build-cache-wasm-browser-production` from `apps/web` to rebuild both;
+ a standalone Vite build reuses the generated artifacts. The fixture compares
+ embedded `cacheBuildInfo()` metadata (package version, GraphQL schema hash,
+ compatibility epoch, record format, and storage schema) before creating the
+ fixture database and again before fault injection. This matched-build check
+ is intentionally stricter than normal cache compatibility.
- Deferred: stale-namespace DB cleanup (browser), `scan_prefix`/
`approx_size` for GC (hardening phase).
diff --git a/apps/web/src/features/entity/bulk-edit/BulkDeleteView.test.tsx b/apps/web/src/features/entity/bulk-edit/BulkDeleteView.test.tsx
new file mode 100644
index 0000000000..3bf384fbbe
--- /dev/null
+++ b/apps/web/src/features/entity/bulk-edit/BulkDeleteView.test.tsx
@@ -0,0 +1,153 @@
+import { Dialog } from '@kobalte/core/dialog';
+import { cleanup, fireEvent, render, screen } from '@solidjs/testing-library';
+import type { JSX } from 'solid-js';
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+import { BulkDeleteFailure } from '../queries/bulk-delete-result';
+import type { EntityData } from '../types/entity';
+import { BulkDeleteView } from './BulkDeleteView';
+
+const mutateAsync = vi.hoisted(() => vi.fn());
+vi.mock('@entity', () => ({
+ createBulkDeleteDssItemsMutation: () => ({ mutateAsync }),
+ InlineEntity: (props: { entity: EntityData }) => (
+ <span data-testid={props.entity.id}>{props.entity.name}</span>
+ ),
+}));
+vi.mock('@ui', () => ({
+ Button: (props: JSX.ButtonHTMLAttributes<HTMLButtonElement>) => (
+ <button {...props} />
+ ),
+ cn: (...values: unknown[]) =>
+ values.filter((value) => typeof value === 'string').join(' '),
+}));
+
+const failed = {
+ id: 'failed',
+ type: 'document',
+ name: 'Failed task',
+} as EntityData;
+const deleted = {
+ id: 'deleted',
+ type: 'document',
+ name: 'Deleted task',
+} as EntityData;
+
+function mount() {
+ const onFinish = vi.fn();
+ const onError = vi.fn();
+ const onCancel = vi.fn();
+ const onPartialDelete = vi.fn();
+ render(() => (
+ <Dialog open modal={false}>
+ <Dialog.Content>
+ <BulkDeleteView
+ entities={[failed, deleted]}
+ onFinish={onFinish}
+ onError={onError}
+ onCancel={onCancel}
+ onPartialDelete={onPartialDelete}
+ />
+ </Dialog.Content>
+ </Dialog>
+ ));
+ return { onFinish, onError, onCancel, onPartialDelete };
+}
+
+beforeEach(() => {
+ mutateAsync.mockReset();
+ vi.stubGlobal(
+ 'requestAnimationFrame',
+ vi.fn(() => 0)
+ );
+});
+afterEach(() => {
+ cleanup();
+ vi.unstubAllGlobals();
+ vi.restoreAllMocks();
+});
+
+describe('BulkDeleteView partial failure', () => {
+ it('reports partial completion and retries only the failed items', async () => {
+ mutateAsync
+ .mockRejectedValueOnce(
+ new BulkDeleteFailure(
+ [failed, deleted],
+ [false, true],
+ new Error('failed delete')
+ )
+ )
+ .mockResolvedValueOnce([true]);
+ const { onFinish, onError, onPartialDelete } = mount();
+ fireEvent.click(screen.getByRole('button', { name: 'Delete' }));
+ await vi.waitFor(() =>
+ expect(screen.getByRole('status').textContent).toContain(
+ 'Deleted 1 of 2 items; 1 failed'
+ )
+ );
+ expect(screen.getByTestId('failed')).toBeTruthy();
+ expect(screen.queryByTestId('deleted')).toBeNull();
+ expect(onFinish).not.toHaveBeenCalled();
+ expect(onError).not.toHaveBeenCalled(); // no second, misleading full-failure callback
+ fireEvent.click(screen.getByRole('button', { name: 'Delete' }));
+ await vi.waitFor(() => expect(onFinish).toHaveBeenCalledOnce());
+ expect(mutateAsync).toHaveBeenNthCalledWith(1, [failed, deleted]);
+ expect(mutateAsync).toHaveBeenNthCalledWith(2, [failed]);
+ expect(onPartialDelete).toHaveBeenCalledExactlyOnceWith(
+ [deleted],
+ [failed]
+ );
+ });
+
+ it('reports confirmed successes before cancellation without completing the whole batch', async () => {
+ mutateAsync.mockRejectedValueOnce(
+ new BulkDeleteFailure([failed, deleted], [false, true])
+ );
+ const { onFinish, onError, onCancel, onPartialDelete } = mount();
+ fireEvent.click(screen.getByRole('button', { name: 'Delete' }));
+ await vi.waitFor(() =>
+ expect(onPartialDelete).toHaveBeenCalledExactlyOnceWith(
+ [deleted],
+ [failed]
+ )
+ );
+ expect(onCancel).not.toHaveBeenCalled();
+ fireEvent.click(screen.getByRole('button', { name: 'Cancel' }));
+ expect(onCancel).toHaveBeenCalledOnce();
+ expect(onPartialDelete).toHaveBeenCalledOnce();
+ expect(onFinish).not.toHaveBeenCalled();
+ expect(onError).not.toHaveBeenCalled();
+ });
+
+ it('does not retry successful ids when caller cleanup throws', async () => {
+ vi.spyOn(console, 'error').mockImplementation(() => {});
+ mutateAsync
+ .mockRejectedValueOnce(
+ new BulkDeleteFailure([failed, deleted], [false, true])
+ )
+ .mockResolvedValueOnce([true]);
+ const { onFinish, onError, onPartialDelete } = mount();
+ onPartialDelete.mockImplementation(() => {
+ throw new Error('cleanup failed');
+ });
+ fireEvent.click(screen.getByRole('button', { name: 'Delete' }));
+ await vi.waitFor(() => expect(onPartialDelete).toHaveBeenCalledOnce());
+ expect(screen.queryByTestId('deleted')).toBeNull();
+ fireEvent.click(screen.getByRole('button', { name: 'Delete' }));
+ await vi.waitFor(() => expect(onFinish).toHaveBeenCalledOnce());
+ expect(mutateAsync).toHaveBeenNthCalledWith(2, [failed]);
+ expect(onError).not.toHaveBeenCalled();
+ });
+
+ it('preserves the existing full-failure callback and original retry targets', async () => {
+ vi.spyOn(console, 'error').mockImplementation(() => {});
+ const error = new Error('all failed');
+ mutateAsync.mockRejectedValueOnce(error);
+ const { onFinish, onError } = mount();
+ fireEvent.click(screen.getByRole('button', { name: 'Delete' }));
+ await vi.waitFor(() => expect(onError).toHaveBeenCalledWith(error));
+ expect(onFinish).not.toHaveBeenCalled();
+ expect(screen.getByTestId('failed')).toBeTruthy();
+ expect(screen.getByTestId('deleted')).toBeTruthy();
+ expect(screen.queryByRole('status')).toBeNull();
+ });
+});
diff --git a/apps/web/src/features/entity/bulk-edit/BulkDeleteView.tsx b/apps/web/src/features/entity/bulk-edit/BulkDeleteView.tsx
index 3cb0f3b70a..58455ac9f3 100644
--- a/apps/web/src/features/entity/bulk-edit/BulkDeleteView.tsx
+++ b/apps/web/src/features/entity/bulk-edit/BulkDeleteView.tsx
@@ -6,15 +6,27 @@ import {
import { Dialog } from '@kobalte/core/dialog';
import CloseIcon from '@phosphor-icons/core/regular/x.svg?component-solid';
import { Button, cn } from '@ui';
-import { For, onMount, Show } from 'solid-js';
+import { createSignal, For, onMount, Show } from 'solid-js';
+import { BulkDeleteFailure } from '../queries/bulk-delete-result';
+
+/** Confirmed progress without closing a partially failed deletion dialog. */
+export type PartialDeleteHandler = (
+ deleted: EntityData[],
+ remaining: EntityData[]
+) => void;
export const BulkDeleteView = (props: {
entities: EntityData[];
onFinish: () => void;
onCancel: () => void;
onError?: (error: unknown) => void;
+ onPartialDelete?: PartialDeleteHandler;
}) => {
const bulkDelete = createBulkDeleteDssItemsMutation();
+ const [remainingEntities, setRemainingEntities] =
+ createSignal<EntityData[]>();
+ const [partialMessage, setPartialMessage] = createSignal<string>();
+ const entities = () => remainingEntities() ?? props.entities;
let deleteButton: HTMLButtonElement | undefined;
const focusDeleteButton = () => {
@@ -26,10 +38,32 @@ export const BulkDeleteView = (props: {
onMount(focusDeleteButton);
const handleDelete = async () => {
+ // Keep the completion attached to the caller that submitted this attempt,
+ // even if another modal replaces this one while the request is in flight.
+ const onPartialDelete = props.onPartialDelete;
+ setPartialMessage(undefined);
try {
- await bulkDelete.mutateAsync(props.entities);
+ await bulkDelete.mutateAsync(entities());
props.onFinish();
} catch (error) {
+ if (
+ error instanceof BulkDeleteFailure &&
+ error.deletedEntities.length > 0
+ ) {
+ setRemainingEntities(error.failedEntities);
+ setPartialMessage(error.message);
+ try {
+ onPartialDelete?.(error.deletedEntities, error.failedEntities);
+ } catch (cleanupError) {
+ // A caller cleanup failure must not turn confirmed deletes into
+ // retry targets or suppress the partial-success feedback.
+ console.error(
+ 'Failed to clean up partially deleted entities:',
+ cleanupError
+ );
+ }
+ return;
+ }
console.error('Failed to delete entities:', error);
props.onError?.(error);
}
@@ -47,21 +81,19 @@ export const BulkDeleteView = (props: {
</Dialog.CloseButton>
<Dialog.Title as="span" class="text-sm font-medium p-0 m-0">
Delete{' '}
- {props.entities.length === 1
- ? 'Item'
- : `${props.entities.length} Items`}
+ {entities().length === 1 ? 'Item' : `${entities().length} Items`}
</Dialog.Title>
</div>
<div class="p-2 border-b border-edge-muted">
<div class="flex items-center gap-2">
- <For each={props.entities.slice(0, 2)}>
+ <For each={entities().slice(0, 2)}>
{(entity) => (
<div
class={cn(
'bg-hover border border-edge-muted px-2 py-1 truncate text-xs rounded-xs',
{
- 'max-w-[50%]': props.entities.length === 2,
+ 'max-w-[50%]': entities().length === 2,
}
)}
>
@@ -69,19 +101,26 @@ export const BulkDeleteView = (props: {
</div>
)}
</For>
- <Show when={props.entities.length > 2}>
+ <Show when={entities().length > 2}>
<div class="text-ink-muted text-xs px-2 py-1">
- +{props.entities.length - 2} more
+ +{entities().length - 2} more
</div>
</Show>
</div>
</div>
<div class="p-3 flex flex-col gap-3">
+ <Show when={partialMessage()}>
+ {(message) => (
+ <p role="status" class="text-sm text-ink-muted">
+ {message()}. Only failed items remain for retry.
+ </p>
+ )}
+ </Show>
<p class="text-sm text-ink-muted">
- {props.entities.length === 1
+ {entities().length === 1
? 'You are about to delete this item. This action cannot be undone.'
- : `You are about to delete ${props.entities.length} items. This action cannot be undone.`}
+ : `You are about to delete ${entities().length} items. This action cannot be undone.`}
</p>
<div class="flex justify-end gap-2">
diff --git a/apps/web/src/features/entity/bulk-edit/BulkEditEntityModal.test.tsx b/apps/web/src/features/entity/bulk-edit/BulkEditEntityModal.test.tsx
new file mode 100644
index 0000000000..07e2610058
--- /dev/null
+++ b/apps/web/src/features/entity/bulk-edit/BulkEditEntityModal.test.tsx
@@ -0,0 +1,96 @@
+import { cleanup, fireEvent, render, screen } from '@solidjs/testing-library';
+import { type ParentProps, Show } from 'solid-js';
+import { afterEach, describe, expect, it, vi } from 'vitest';
+import type { EntityData } from '../types/entity';
+import type { PartialDeleteHandler } from './BulkDeleteView';
+import {
+ GlobalBulkEditEntityModal,
+ openBulkEditModal,
+} from './BulkEditEntityModal';
+
+vi.mock('@core/util/createControlledOpenSignal', async () => {
+ const { createSignal } = await import('solid-js');
+ return { createControlledOpenSignal: (open: boolean) => createSignal(open) };
+});
+vi.mock('@ui', () => ({
+ Dialog: (
+ props: ParentProps<{ open: boolean; onOpenChange: (open: boolean) => void }>
+ ) => (
+ <Show when={props.open}>
+ <div role="dialog">
+ <button type="button" onClick={() => props.onOpenChange(false)}>
+ Dismiss
+ </button>
+ {props.children}
+ </div>
+ </Show>
+ ),
+ Surface: (props: ParentProps) => <div>{props.children}</div>,
+}));
+vi.mock('./BulkMoveToProjectView', () => ({
+ BulkMoveToProjectView: () => null,
+}));
+vi.mock('./BulkRenameEntitiesView', () => ({
+ BulkRenameEntitiesView: () => null,
+}));
+vi.mock('./BulkDeleteView', () => ({
+ BulkDeleteView: (props: {
+ entities: EntityData[];
+ onPartialDelete?: PartialDeleteHandler;
+ onFinish: () => void;
+ onCancel: () => void;
+ }) => (
+ <div data-testid="delete-view">
+ <button
+ type="button"
+ onClick={() =>
+ props.onPartialDelete?.(
+ props.entities.slice(0, 1),
+ props.entities.slice(1)
+ )
+ }
+ >
+ Partial success
+ </button>
+ <button type="button" onClick={props.onCancel}>
+ Cancel
+ </button>
+ <button type="button" onClick={props.onFinish}>
+ Finish
+ </button>
+ </div>
+ ),
+}));
+
+afterEach(cleanup);
+
+describe('bulk-delete modal progress wiring', () => {
+ it.each(['Cancel', 'Dismiss'])(
+ 'forwards progress without completing the batch before %s',
+ (close) => {
+ const deleted = { id: 'deleted', type: 'document' } as EntityData;
+ const failed = { id: 'failed', type: 'document' } as EntityData;
+ const onPartialDelete = vi.fn();
+ const onFinish = vi.fn();
+ const onCancel = vi.fn();
+ openBulkEditModal({
+ view: 'delete',
+ entities: [deleted, failed],
+ onPartialDelete,
+ onFinish,
+ onCancel,
+ });
+ render(() => <GlobalBulkEditEntityModal />);
+ fireEvent.click(screen.getByRole('button', { name: 'Partial success' }));
+ expect(onPartialDelete).toHaveBeenCalledWith([deleted], [failed]);
+ expect(onFinish).not.toHaveBeenCalled();
+ expect(onCancel).not.toHaveBeenCalled();
+ expect(screen.getByTestId('delete-view')).toBeTruthy();
+ fireEvent.click(screen.getByRole('button', { name: close }));
+ expect(onCancel).toHaveBeenCalledOnce();
+ expect(onFinish).not.toHaveBeenCalled();
+ expect(onPartialDelete).toHaveBeenCalledOnce();
+ expect(screen.queryByTestId('delete-view')).toBeNull();
+ }
+ );
+});
diff --git a/apps/web/src/features/entity/bulk-edit/BulkEditEntityModal.tsx b/apps/web/src/features/entity/bulk-edit/BulkEditEntityModal.tsx
index 089540eab3..f6087ef9e1 100644
--- a/apps/web/src/features/entity/bulk-edit/BulkEditEntityModal.tsx
+++ b/apps/web/src/features/entity/bulk-edit/BulkEditEntityModal.tsx
@@ -8,7 +8,7 @@ import {
type Setter,
Show,
} from 'solid-js';
-import { BulkDeleteView } from './BulkDeleteView';
+import { BulkDeleteView, type PartialDeleteHandler } from './BulkDeleteView';
import { BulkMoveToProjectView } from './BulkMoveToProjectView';
import { BulkRenameEntitiesView } from './BulkRenameEntitiesView';
@@ -20,6 +20,7 @@ const BulkEditEntityModalContent = (props: {
onFinish?: () => void;
onCancel?: () => void;
onError?: (error: unknown) => void;
+ onPartialDelete?: PartialDeleteHandler;
}) => {
const handleFinish = () => {
props.setIsOpen(false);
@@ -68,6 +69,7 @@ const BulkEditEntityModalContent = (props: {
onFinish={handleFinish}
onCancel={handleCancel}
onError={handleError}
+ onPartialDelete={props.onPartialDelete}
/>
</Show>
</div>
@@ -105,6 +107,7 @@ const [globalModalProps, setGlobalModalProps] = createSignal<{
onFinish?: () => void;
onCancel?: () => void;
onError?: (error: unknown) => void;
+ onPartialDelete?: PartialDeleteHandler;
} | null>(null);
const [modalOpen, setModalOpen] = createControlledOpenSignal(false, {
id: 'entity-edit',
@@ -116,6 +119,7 @@ export const openBulkEditModal = (props: {
onFinish?: () => void;
onCancel?: () => void;
onError?: (error: unknown) => void;
+ onPartialDelete?: PartialDeleteHandler;
}) => {
setModalOpen(true);
setGlobalModalProps(props);
@@ -155,6 +159,7 @@ export const GlobalBulkEditEntityModal = () => {
onFinish={handleFinish}
onCancel={handleCancel}
onError={handleError}
+ onPartialDelete={props().onPartialDelete}
/>
)}
</Show>
diff --git a/apps/web/src/features/entity/queries/bulk-delete-result.ts b/apps/web/src/features/entity/queries/bulk-delete-result.ts
new file mode 100644
index 0000000000..05ad9cc32e
--- /dev/null
+++ b/apps/web/src/features/entity/queries/bulk-delete-result.ts
@@ -0,0 +1,31 @@
+import type { EntityData } from '../types/entity';
+
+/** Per-item outcomes of a GraphQL-enabled bulk delete, including retry targets. */
+export class BulkDeleteFailure extends Error {
+ readonly deletedEntities: EntityData[];
+ readonly failedEntities: EntityData[];
+
+ constructor(
+ entities: EntityData[],
+ readonly results: boolean[],
+ cause?: unknown
+ ) {
+ const deletedEntities = entities.filter(
+ (_, index) => results[index] === true
+ );
+ const failedEntities = entities.filter(
+ (_, index) => results[index] !== true
+ );
+ super(
+ deletedEntities.length > 0
+ ? `Deleted ${deletedEntities.length} of ${entities.length} items; ${failedEntities.length} failed`
+ : cause instanceof Error
+ ? cause.message
+ : 'Failed to delete items',
+ { cause }
+ );
+ this.name = 'BulkDeleteFailure';
+ this.deletedEntities = deletedEntities;
+ this.failedEntities = failedEntities;
+ }
+}
diff --git a/apps/web/src/features/entity/queries/dss.test.tsx b/apps/web/src/features/entity/queries/dss.test.tsx
new file mode 100644
index 0000000000..f176316125
--- /dev/null
+++ b/apps/web/src/features/entity/queries/dss.test.tsx
@@ -0,0 +1,491 @@
+import type { EntityData } from '@entity';
+import {
+ onlineManager,
+ QueryClient,
+ QueryClientProvider,
+} from '@tanstack/solid-query';
+import { err, ok } from 'neverthrow';
+import { render } from 'solid-js/web';
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+
+const mocks = vi.hoisted(() => ({
+ graphqlEnabled: vi.fn(() => true),
+ deleteItem: vi.fn(),
+ deleteCall: vi.fn(),
+ deleteReminder: vi.fn(),
+ deleteSchedule: vi.fn(),
+ removeSoup: vi.fn(),
+ removeSearch: vi.fn(),
+ rollbackSoup: vi.fn(),
+ rollbackSearch: vi.fn(),
+ refresh: vi.fn(),
+ failure: vi.fn(),
+}));
+vi.mock('@core/component/FileList/itemOperations', () => ({
+ deleteItem: mocks.deleteItem,
+ copyItem: vi.fn(),
+ moveToFolder: vi.fn(),
+}));
+vi.mock('@core/component/Toast/Toast', () => ({
+ toast: { failure: mocks.failure },
+}));
+vi.mock('@core/constant/featureFlags', () => ({
+ enableGraphqlSoup: {},
+ isFeatureEnabled: mocks.graphqlEnabled,
+}));
+vi.mock('@queries/client', () => ({
+ get queryClient() {
+ return client;
+ },
+}));
+vi.mock('@queries/agent-session/entity-mutations', () => ({
+ deleteAgentSession: vi.fn(),
+}));
+vi.mock('@queries/soup/cache', () => ({
+ removeSoupEntities: mocks.removeSoup,
+ removeSearchEntities: mocks.removeSearch,
+}));
+vi.mock('@queries/soup/graphql/active-queries', () => ({
+ refreshActiveGraphqlSoupQueries: mocks.refresh,
+}));
+vi.mock('@service-call/client', () => ({
+ callServiceClient: { deleteCallRecord: mocks.deleteCall },
+}));
+vi.mock('@service-scheduled-action/client', () => ({
+ scheduledActionClient: { deleteSchedule: mocks.deleteSchedule },
+}));
+vi.mock('@service-storage/client', () => ({
+ storageServiceClient: { reminders: { deleteReminder: mocks.deleteReminder } },
+}));
+
+import { scheduledActionKeys } from '@queries/agent-schedule/keys';
+import { callKeys } from '@queries/call/keys';
+import { notificationKeys } from '@queries/notification/keys';
+import { reminderKeys } from '@queries/reminders/keys';
+import {
+ GRAPHQL_SOUP_DELETE_MUTATION_KEY,
+ GRAPHQL_SOUP_DELETE_RETENTION_MS,
+ type GraphqlSoupDeleteContext,
+ usePendingGraphqlSoupDeleteIds,
+} from '@queries/soup/graphql/optimistic-deletions';
+import { BulkDeleteFailure } from './bulk-delete-result';
+import { createBulkDeleteDssItemsMutation } from './dss';
+
+let client: QueryClient;
+let dispose: (() => void) | undefined;
+const entity = (id: string, type: EntityData['type'] = 'document') =>
+ ({ id, type }) as EntityData;
+
+function deferred<T>() {
+ let resolve!: (value: T) => void;
+ let reject!: (error: Error) => void;
+ const promise = new Promise<T>((res, rej) => {
+ resolve = res;
+ reject = rej;
+ });
+ return { promise, resolve, reject };
+}
+
+function mount() {
+ let result!: {
+ mutation: ReturnType<typeof createBulkDeleteDssItemsMutation>;
+ pendingIds: ReturnType<typeof usePendingGraphqlSoupDeleteIds>;
+ };
+ dispose = render(
+ () => (
+ <QueryClientProvider client={client}>
+ {(() => {
+ result = {
+ mutation: createBulkDeleteDssItemsMutation(),
+ pendingIds: usePendingGraphqlSoupDeleteIds(),
+ };
+ return null;
+ })()}
+ </QueryClientProvider>
+ ),
+ document.body
+ );
+ return result;
+}
+
+beforeEach(() => {
+ vi.resetAllMocks();
+ mocks.graphqlEnabled.mockReturnValue(true);
+ mocks.removeSoup.mockReturnValue({ rollback: mocks.rollbackSoup });
+ mocks.removeSearch.mockReturnValue({ rollback: mocks.rollbackSearch });
+ mocks.refresh.mockResolvedValue(undefined);
+ mocks.deleteItem.mockResolvedValue(true);
+ onlineManager.setOnline(true);
+ client = new QueryClient({ defaultOptions: { mutations: { retry: false } } });
+});
+afterEach(() => {
+ for (const mutation of client.getMutationCache().getAll()) {
+ (
+ mutation.state.context as GraphqlSoupDeleteContext | undefined
+ )?.graphqlDeletion?.release();
+ }
+ dispose?.();
+ client.clear();
+ vi.useRealTimers();
+ document.body.replaceChildren();
+ vi.restoreAllMocks();
+});
+
+function cacheRows(
+ remove: typeof mocks.removeSoup,
+ rollback: typeof mocks.rollbackSoup,
+ initial: string[]
+) {
+ let rows = new Set(initial);
+ remove.mockImplementation((ids: Set<string>) => {
+ const previous = new Set(rows);
+ rows = new Set([...rows].filter((id) => !ids.has(id)));
+ return {
+ rollback: () => {
+ rollback();
+ rows = new Set(previous);
+ },
+ };
+ });
+ return () => [...rows].sort();
+}
+
+describe('bulk delete GraphQL optimism', () => {
+ it.each(['throw', 'false'] as const)(
+ 'restores only failed rows in Soup and search after a partial %s failure',
+ async (failure) => {
+ vi.spyOn(console, 'error').mockImplementation(() => {});
+ const soup = cacheRows(mocks.removeSoup, mocks.rollbackSoup, [
+ 'failed',
+ 'deleted',
+ 'other',
+ ]);
+ const search = cacheRows(mocks.removeSearch, mocks.rollbackSearch, [
+ 'failed',
+ 'deleted',
+ 'other',
+ ]);
+ const failed = entity('failed');
+ const deleted = entity('deleted');
+ if (failure === 'throw')
+ mocks.deleteItem.mockRejectedValueOnce(new Error('failed delete'));
+ else mocks.deleteItem.mockResolvedValueOnce(false);
+ mocks.deleteItem.mockResolvedValueOnce(true);
+ const refreshed = deferred<void>();
+ mocks.refresh.mockReturnValue(refreshed.promise);
+ const { mutation, pendingIds } = mount();
+ try {
+ await expect(
+ mutation.mutateAsync([failed, deleted])
+ ).rejects.toMatchObject({
+ results: [false, true],
+ deletedEntities: [deleted],
+ failedEntities: [failed],
+ });
+ expect(soup()).toEqual(['failed', 'other']);
+ expect(search()).toEqual(['failed', 'other']);
+ expect([...pendingIds()]).toEqual(['deleted']);
+ expect(mocks.failure).toHaveBeenCalledWith(
+ 'Deleted 1 of 2 items; 1 failed'
+ );
+ expect(mocks.removeSoup).toHaveBeenLastCalledWith(new Set(['deleted']));
+ expect(mocks.removeSearch).toHaveBeenLastCalledWith(
+ new Set(['deleted'])
+ );
+ } finally {
+ refreshed.resolve();
+ }
+ }
+ );
+
+ it('reconciles successful call/reminder/automation deletes despite sibling failures', async () => {
+ vi.spyOn(console, 'error').mockImplementation(() => {});
+ const invalidate = vi
+ .spyOn(client, 'invalidateQueries')
+ .mockResolvedValue(undefined);
+ const schedules = [
+ { id: 'deleted-auto' },
+ { id: 'failed-auto' },
+ { id: 'other-auto' },
+ ];
+ client.setQueryData(scheduledActionKeys.list.queryKey, schedules);
+ mocks.deleteItem.mockRejectedValue(new Error('document delete failed'));
+ mocks.deleteCall.mockResolvedValue(ok(undefined));
+ mocks.deleteReminder.mockResolvedValue(ok(undefined));
+ mocks.deleteSchedule.mockImplementation(
+ async ({ scheduleId }: { scheduleId: string }) =>
+ scheduleId === 'deleted-auto'
+ ? ok(undefined)
+ : err([{ code: 'FORBIDDEN', message: 'schedule delete failed' }])
+ );
+ const { mutation } = mount();
+ await expect(
+ mutation.mutateAsync([
+ entity('failed-document'),
+ entity('deleted-call', 'call'),
+ entity('deleted-reminder', 'reminder'),
+ entity('deleted-auto', 'automation'),
+ entity('failed-auto', 'automation'),
+ ])
+ ).rejects.toBeInstanceOf(BulkDeleteFailure);
+ expect(invalidate).toHaveBeenCalledWith({ queryKey: callKeys._def });
+ expect(invalidate).toHaveBeenCalledWith({ queryKey: reminderKeys._def });
+ expect(invalidate).toHaveBeenCalledWith({
+ queryKey: notificationKeys._def,
+ });
+ expect(invalidate).toHaveBeenCalledWith({
+ queryKey: scheduledActionKeys.list.queryKey,
+ });
+ expect(client.getQueryData(scheduledActionKeys.list.queryKey)).toEqual([
+ schedules[1],
+ schedules[2],
+ ]);
+ expect(mocks.failure).toHaveBeenCalledWith(
+ 'Deleted 3 of 5 items; 2 failed'
+ );
+ });
+
+ it('does not remove or invalidate unsuccessful side-effect entities', async () => {
+ vi.spyOn(console, 'error').mockImplementation(() => {});
+ const invalidate = vi
+ .spyOn(client, 'invalidateQueries')
+ .mockResolvedValue(undefined);
+ const schedules = [{ id: 'failed-auto' }];
+ client.setQueryData(scheduledActionKeys.list.queryKey, schedules);
+ mocks.deleteCall.mockRejectedValue(new Error('failed call'));
+ mocks.deleteReminder.mockRejectedValue(new Error('failed reminder'));
+ mocks.deleteSchedule.mockRejectedValue(new Error('failed schedule'));
+ const { mutation } = mount();
+ await expect(
+ mutation.mutateAsync([
+ entity('failed-call', 'call'),
+ entity('failed-reminder', 'reminder'),
+ entity('failed-auto', 'automation'),
+ ])
+ ).rejects.toThrow('failed call');
+ expect(invalidate).not.toHaveBeenCalled();
+ expect(client.getQueryData(scheduledActionKeys.list.queryKey)).toEqual(
+ schedules
+ );
+ });
+
+ it.each([true, false])(
+ 'handles boolean-only total failure without changing the disabled path (%s)',
+ async (graphql) => {
+ vi.spyOn(console, 'error').mockImplementation(() => {});
+ mocks.graphqlEnabled.mockReturnValue(graphql);
+ mocks.deleteItem.mockResolvedValue(false);
+ const { mutation } = mount();
+ const result = mutation.mutateAsync([entity('failed')]);
+ if (graphql) {
+ await expect(result).rejects.toThrow('Failed to delete items');
+ expect(mocks.rollbackSoup).toHaveBeenCalledOnce();
+ expect(mocks.rollbackSearch).toHaveBeenCalledOnce();
+ expect(mocks.failure).toHaveBeenCalledWith('Failed to delete items');
+ } else {
+ await expect(result).resolves.toEqual([false]);
+ expect(mocks.rollbackSoup).not.toHaveBeenCalled();
+ expect(mocks.rollbackSearch).not.toHaveBeenCalled();
+ expect(mocks.failure).not.toHaveBeenCalled();
+ }
+ }
+ );
+
+ it('hides only deletable ids until deletion and GraphQL refresh both finish', async () => {
+ const network = deferred<boolean>();
+ const refreshed = deferred<void>();
+ mocks.deleteItem.mockReturnValue(network.promise);
+ mocks.refresh.mockReturnValue(refreshed.promise);
+ const { mutation, pendingIds } = mount();
+ const result = mutation.mutateAsync([
+ entity('task-1'),
+ entity('task-2'),
+ entity('channel', 'channel'),
+ ]);
+ await vi.waitFor(() =>
+ expect([...pendingIds()]).toEqual(['task-1', 'task-2'])
+ );
+ expect(mocks.deleteItem).toHaveBeenCalledTimes(2);
+ expect(mocks.refresh).not.toHaveBeenCalled();
+ network.resolve(true);
+ await expect(result).resolves.toEqual([true, true]);
+ await vi.waitFor(() => expect(mutation.isPending).toBe(false));
+ expect(mocks.refresh).toHaveBeenCalledWith({ throwOnError: true });
+ expect([...pendingIds()]).toEqual(['task-1', 'task-2']);
+ refreshed.resolve();
+ await vi.waitFor(() => expect(pendingIds().size).toBe(0));
+ });
+
+ it('restores failed deletions even while revalidation is still pending', async () => {
+ vi.spyOn(console, 'error').mockImplementation(() => {});
+ const network = deferred<boolean>();
+ const refreshed = deferred<void>();
+ mocks.deleteItem.mockReturnValue(network.promise);
+ mocks.refresh.mockReturnValue(refreshed.promise);
+ const { mutation, pendingIds } = mount();
+ const error = new Error('delete failed');
+ const result = expect(
+ mutation.mutateAsync([entity('task')])
+ ).rejects.toThrow(error);
+ await vi.waitFor(() => expect(pendingIds().has('task')).toBe(true));
+ network.reject(error);
+ await vi.waitFor(() => expect(mocks.refresh).toHaveBeenCalledOnce());
+ expect(mocks.rollbackSoup).toHaveBeenCalledOnce();
+ expect(mocks.rollbackSearch).toHaveBeenCalledOnce();
+ expect(mocks.failure).toHaveBeenCalledWith('Failed to delete items');
+ expect(pendingIds().has('task')).toBe(false);
+ await result;
+ refreshed.resolve();
+ await vi.waitFor(() => expect(pendingIds().size).toBe(0));
+ });
+
+ it('revalidates partial boolean failures without treating every item as deleted', async () => {
+ mocks.deleteItem.mockResolvedValueOnce(true).mockResolvedValueOnce(false);
+ const refreshed = deferred<void>();
+ mocks.refresh.mockReturnValue(refreshed.promise);
+ const { mutation, pendingIds } = mount();
+ await expect(
+ mutation.mutateAsync([entity('deleted'), entity('retained')])
+ ).rejects.toMatchObject({
+ results: [true, false],
+ message: 'Deleted 1 of 2 items; 1 failed',
+ });
+ expect(mocks.refresh).toHaveBeenCalledOnce();
+ expect([...pendingIds()]).toEqual(['deleted']);
+ refreshed.resolve();
+ await vi.waitFor(() => expect(pendingIds().size).toBe(0));
+ });
+
+ it.each([true, false])(
+ 'latches the GraphQL flag at creation (initially %s)',
+ async (enabled) => {
+ mocks.graphqlEnabled.mockReturnValue(enabled);
+ const network = deferred<boolean>();
+ const refreshed = deferred<void>();
+ mocks.deleteItem.mockReturnValue(network.promise);
+ mocks.refresh.mockReturnValue(refreshed.promise);
+ const { mutation, pendingIds } = mount();
+ mocks.graphqlEnabled.mockReturnValue(!enabled);
+ const result = mutation.mutateAsync([entity('task')]);
+ await vi.waitFor(() => expect(mocks.deleteItem).toHaveBeenCalledOnce());
+ expect(pendingIds().has('task')).toBe(enabled);
+ const cached = client.getMutationCache().getAll()[0];
+ expect(cached.options.mutationKey).toEqual(
+ enabled ? GRAPHQL_SOUP_DELETE_MUTATION_KEY : undefined
+ );
+ expect(
+ Boolean(
+ (cached.state.context as GraphqlSoupDeleteContext).graphqlDeletion
+ )
+ ).toBe(enabled);
+ mocks.graphqlEnabled.mockReturnValue(enabled);
+ network.resolve(true);
+ await result;
+ expect(mocks.refresh).toHaveBeenCalledTimes(enabled ? 1 : 0);
+ expect(mocks.graphqlEnabled).toHaveBeenCalledTimes(1);
+ refreshed.resolve();
+ await vi.waitFor(() => expect(pendingIds().size).toBe(0));
+ }
+ );
+
+ it('does not recreate deletion state after the client is cleared during an API request', async () => {
+ const network = deferred<boolean>();
+ mocks.deleteItem.mockReturnValue(network.promise);
+ const { mutation } = mount();
+ const result = mutation.mutateAsync([entity('task')]);
+ await vi.waitFor(() => expect(mocks.deleteItem).toHaveBeenCalledOnce());
+ client.clear();
+ network.resolve(true);
+ await result;
+ expect(mocks.refresh).not.toHaveBeenCalled();
+ expect(client.getQueryCache().getAll()).toEqual([]);
+ });
+
+ it('does not restore old snapshots when a partial failure arrives after client clearing', async () => {
+ const network = deferred<boolean>();
+ mocks.deleteItem
+ .mockReturnValueOnce(network.promise)
+ .mockResolvedValueOnce(true);
+ const { mutation } = mount();
+ const result = expect(
+ mutation.mutateAsync([entity('failed'), entity('deleted')])
+ ).rejects.toBeInstanceOf(BulkDeleteFailure);
+ await vi.waitFor(() => expect(mocks.deleteItem).toHaveBeenCalledTimes(2));
+ client.clear();
+ network.resolve(false);
+ await result;
+ expect(mocks.rollbackSoup).not.toHaveBeenCalled();
+ expect(mocks.rollbackSearch).not.toHaveBeenCalled();
+ expect(mocks.refresh).not.toHaveBeenCalled();
+ expect(client.getQueryCache().getAll()).toEqual([]);
+ });
+
+ it('retains successful deletes after a refresh error and clears them on retry success', async () => {
+ vi.useFakeTimers();
+ mocks.refresh
+ .mockRejectedValueOnce(new Error('offline'))
+ .mockResolvedValue(undefined);
+ const { mutation, pendingIds } = mount();
+ await mutation.mutateAsync([entity('task')]);
+ await vi.advanceTimersByTimeAsync(0);
+ expect(mutation.isPending).toBe(false);
+ expect([...pendingIds()]).toEqual(['task']);
+ await vi.advanceTimersByTimeAsync(1000);
+ expect(mocks.refresh).toHaveBeenCalledTimes(2);
+ expect(pendingIds().size).toBe(0);
+ await vi.advanceTimersByTimeAsync(GRAPHQL_SOUP_DELETE_RETENTION_MS);
+ expect(mocks.refresh).toHaveBeenCalledTimes(2);
+ });
+
+ it('retains the successful siblings when another delete throws, waiting for every outcome', async () => {
+ vi.spyOn(console, 'error').mockImplementation(() => {});
+ const slowSuccess = deferred<boolean>();
+ const refreshed = deferred<void>();
+ mocks.deleteItem
+ .mockRejectedValueOnce(new Error('delete failed'))
+ .mockReturnValueOnce(slowSuccess.promise);
+ mocks.refresh.mockReturnValue(refreshed.promise);
+ const { mutation, pendingIds } = mount();
+ const result = expect(
+ mutation.mutateAsync([entity('failed'), entity('deleted')])
+ ).rejects.toThrow('Deleted 1 of 2 items; 1 failed');
+ await vi.waitFor(() => expect(mocks.deleteItem).toHaveBeenCalledTimes(2));
+ expect(mocks.refresh).not.toHaveBeenCalled();
+ slowSuccess.resolve(true);
+ await result;
+ expect([...pendingIds()]).toEqual(['deleted']);
+ refreshed.resolve();
+ await vi.waitFor(() => expect(pendingIds().size).toBe(0));
+ });
+
+ it.each([false, true])(
+ 'preserves disabled-path behavior (failure: %s)',
+ async (fails) => {
+ mocks.graphqlEnabled.mockReturnValue(false);
+ vi.spyOn(console, 'error').mockImplementation(() => {});
+ const network = deferred<boolean>();
+ mocks.deleteItem.mockReturnValue(network.promise);
+ const { mutation, pendingIds } = mount();
+ const result = mutation.mutateAsync([entity('task')]);
+ const settled = fails
+ ? expect(result).rejects.toThrow('delete failed')
+ : expect(result).resolves.toEqual([true]);
+ await vi.waitFor(() => expect(mocks.deleteItem).toHaveBeenCalledOnce());
+ expect(mocks.removeSoup).toHaveBeenCalledWith(new Set(['task']));
+ expect(mocks.removeSearch).toHaveBeenCalledWith(new Set(['task']));
+ expect(pendingIds().size).toBe(0);
+ expect(
+ client.getMutationCache().getAll()[0].options.mutationKey
+ ).toBeUndefined();
+ expect(
+ client.getMutationCache().getAll()[0].options.onSettled
+ ).toBeUndefined();
+ if (fails) network.reject(new Error('delete failed'));
+ else network.resolve(true);
+ await settled;
+ expect(mocks.refresh).not.toHaveBeenCalled();
+ expect(mocks.rollbackSoup).toHaveBeenCalledTimes(fails ? 1 : 0);
+ expect(mocks.rollbackSearch).toHaveBeenCalledTimes(fails ? 1 : 0);
+ }
+ );
+});
diff --git a/apps/web/src/features/entity/queries/dss.ts b/apps/web/src/features/entity/queries/dss.ts
index 1731986ca9..482b964ccb 100644
--- a/apps/web/src/features/entity/queries/dss.ts
+++ b/apps/web/src/features/entity/queries/dss.ts
@@ -5,6 +5,10 @@ import {
moveToFolder,
} from '@core/component/FileList/itemOperations';
import { toast } from '@core/component/Toast/Toast';
+import {
+ enableGraphqlSoup,
+ isFeatureEnabled,
+} from '@core/constant/featureFlags';
import { throwOnErr } from '@core/util/result';
import { scheduledActionKeys } from '@queries/agent-schedule/keys';
import { deleteAgentSession } from '@queries/agent-session/entity-mutations';
@@ -21,15 +25,74 @@ import {
removeSoupEntities,
removeSoupEntitiesFromQueriesReferencing,
} from '@queries/soup/cache';
+import {
+ createGraphqlSoupDeletion,
+ GRAPHQL_SOUP_DELETE_MUTATION_KEY,
+ GRAPHQL_SOUP_DELETE_RETENTION_MS,
+} from '@queries/soup/graphql/optimistic-deletions';
import { soupKeys } from '@queries/soup/keys';
import { ownTouchStamp } from '@queries/soup/normalized-cache/own-touch';
import { callServiceClient } from '@service-call/client';
import { scheduledActionClient } from '@service-scheduled-action/client';
import { storageServiceClient } from '@service-storage/client';
import { useMutation } from '@tanstack/solid-query';
+import { batch } from 'solid-js';
import { type EntityData, getEntityProjectId } from '../types/entity';
+import { BulkDeleteFailure } from './bulk-delete-result';
+
+function invalidateDeletedDssItems(entities: EntityData[]): void {
+ if (entities.some((e) => e.type === 'call')) {
+ void queryClient.invalidateQueries({ queryKey: callKeys._def });
+ }
+ if (entities.some((e) => e.type === 'reminder')) {
+ // Reminder deletion retracts its notification in the same server transaction.
+ void queryClient.invalidateQueries({ queryKey: reminderKeys._def });
+ void queryClient.invalidateQueries({ queryKey: notificationKeys._def });
+ }
+ if (entities.some((e) => e.type === 'automation')) {
+ const deletedIds = new Set(
+ entities.filter((e) => e.type === 'automation').map((e) => e.id)
+ );
+ queryClient.setQueryData(
+ scheduledActionKeys.list.queryKey,
+ (current: unknown) => {
+ if (!Array.isArray(current)) return current;
+ return current.filter(
+ (item: { id?: string }) => !item.id || !deletedIds.has(item.id)
+ );
+ }
+ );
+ void queryClient.invalidateQueries({
+ queryKey: scheduledActionKeys.list.queryKey,
+ });
+ }
+}
+
+async function settleGraphqlDeletes(
+ entities: EntityData[],
+ deletions: Promise<boolean>[]
+): Promise<boolean[]> {
+ const outcomes = await Promise.allSettled(deletions);
+ const results = outcomes.map(
+ (outcome) => outcome.status === 'fulfilled' && outcome.value
+ );
+ // Process confirmed successes before surfacing sibling failures. Never remove
+ // failed schedules or skip call/reminder invalidation because another API threw.
+ try {
+ invalidateDeletedDssItems(entities.filter((_, index) => results[index]));
+ } catch (error) {
+ console.error('Failed to reconcile deleted item caches', error);
+ }
+ const failure = outcomes.find((outcome) => outcome.status === 'rejected');
+ if (results.some((success) => !success)) {
+ if (failure && !results.some(Boolean)) throw failure.reason;
+ throw new BulkDeleteFailure(entities, results, failure?.reason);
+ }
+ return results;
+}
export function createBulkDeleteDssItemsMutation() {
+ const graphqlSoupEnabled = isFeatureEnabled(enableGraphqlSoup);
const isDeletable = (entity: EntityData) => {
const type = entity.type;
return (
@@ -42,63 +105,49 @@ export function createBulkDeleteDssItemsMutation() {
type === 'reminder'
);
};
+ const isCurrentContext = (context: unknown) =>
+ queryClient
+ .getMutationCache()
+ .getAll()
+ .some((mutation) => mutation.state.context === context);
return useMutation(() => ({
+ ...(graphqlSoupEnabled
+ ? {
+ mutationKey: GRAPHQL_SOUP_DELETE_MUTATION_KEY,
+ gcTime: GRAPHQL_SOUP_DELETE_RETENTION_MS,
+ }
+ : {}),
mutationFn: async (entities: EntityData[]) => {
const deletable = entities.filter(isDeletable);
- const results = await Promise.all(
- deletable.map(async (e) => {
- if (e.type === 'agent_session') {
- await deleteAgentSession(e.id);
- return true;
- }
- if (e.type === 'call') {
- return throwOnErr(() =>
- callServiceClient.deleteCallRecord(e.id)
- ).then(() => true);
- }
- if (e.type === 'automation') {
- return throwOnErr(() =>
- scheduledActionClient.deleteSchedule({ scheduleId: e.id })
- ).then(() => true);
- }
- if (e.type === 'reminder') {
- // Deleting a reminder also retracts the notification it produced —
- // the API does both in one transaction, since a reminder *is* its
- // notification's event_item.
- return throwOnErr(() =>
- storageServiceClient.reminders.deleteReminder(e.id)
- ).then(() => true);
- }
- return deleteItem({ id: e.id, itemType: e.type });
- })
- );
- if (deletable.some((e) => e.type === 'call')) {
- queryClient.invalidateQueries({ queryKey: callKeys._def });
- }
- if (deletable.some((e) => e.type === 'reminder')) {
- // The reminder lists are their own queries; the soup cache is already
- // handled by the shared optimistic removal in onMutate. Notifications
- // too, since the delete retracted them server-side.
- queryClient.invalidateQueries({ queryKey: reminderKeys._def });
- queryClient.invalidateQueries({ queryKey: notificationKeys._def });
- }
- if (deletable.some((e) => e.type === 'automation')) {
- const deletedIds = new Set(
- deletable.filter((e) => e.type === 'automation').map((e) => e.id)
- );
- queryClient.setQueryData(
- scheduledActionKeys.list.queryKey,
- (current: unknown) => {
- if (!Array.isArray(current)) return current;
- return current.filter(
- (item: { id?: string }) => !item.id || !deletedIds.has(item.id)
- );
- }
- );
- queryClient.invalidateQueries({
- queryKey: scheduledActionKeys.list.queryKey,
- });
- }
+ const deletions = deletable.map(async (e) => {
+ if (e.type === 'agent_session') {
+ await deleteAgentSession(e.id);
+ return true;
+ }
+ if (e.type === 'call') {
+ return throwOnErr(() =>
+ callServiceClient.deleteCallRecord(e.id)
+ ).then(() => true);
+ }
+ if (e.type === 'automation') {
+ return throwOnErr(() =>
+ scheduledActionClient.deleteSchedule({ scheduleId: e.id })
+ ).then(() => true);
+ }
+ if (e.type === 'reminder') {
+ // Deleting a reminder also retracts the notification it produced —
+ // the API does both in one transaction, since a reminder *is* its
+ // notification's event_item.
+ return throwOnErr(() =>
+ storageServiceClient.reminders.deleteReminder(e.id)
+ ).then(() => true);
+ }
+ return deleteItem({ id: e.id, itemType: e.type });
+ });
+ const results = graphqlSoupEnabled
+ ? await settleGraphqlDeletes(deletable, deletions)
+ : await Promise.all(deletions);
+ if (!graphqlSoupEnabled) invalidateDeletedDssItems(deletable);
return results;
},
onMutate: async (entities: EntityData[]) => {
@@ -106,13 +155,63 @@ export function createBulkDeleteDssItemsMutation() {
const ids = new Set(deletable.map((e) => e.id));
const soupSnapshot = removeSoupEntities(ids);
const searchSnapshot = removeSearchEntities(ids);
- return { soupSnapshot, searchSnapshot };
+ return {
+ soupSnapshot,
+ searchSnapshot,
+ ...(graphqlSoupEnabled
+ ? { graphqlDeletion: createGraphqlSoupDeletion([...ids]) }
+ : {}),
+ };
},
+ onSettled: graphqlSoupEnabled
+ ? (results, error, entities, context) => {
+ if (!context?.graphqlDeletion) return;
+ // Logout/cache clearing can outlive an in-flight API call. Do not
+ // repopulate session-local tombstones after their mutation was cleared.
+ if (!isCurrentContext(context)) {
+ context.graphqlDeletion.release();
+ return;
+ }
+ const outcomes =
+ error instanceof BulkDeleteFailure ? error.results : results;
+ const deletedIds = entities
+ .filter(isDeletable)
+ .flatMap((entity, index) =>
+ outcomes?.[index] === true ? [entity.id] : []
+ );
+ // API completion closes the mutation; display state survives it and
+ // independently retries revalidation. Failed ids are restored now.
+ context?.graphqlDeletion?.settle(deletedIds);
+ }
+ : undefined,
onError: (error, entities, context) => {
- context?.soupSnapshot.rollback();
- context?.searchSnapshot.rollback();
+ if (graphqlSoupEnabled && context && !isCurrentContext(context)) {
+ context.graphqlDeletion?.release();
+ return;
+ }
+ const partialFailure =
+ graphqlSoupEnabled &&
+ error instanceof BulkDeleteFailure &&
+ error.deletedEntities.length > 0;
+ if (partialFailure) {
+ const deletedIds = new Set(
+ error.deletedEntities.map((entity) => entity.id)
+ );
+ // The legacy transactions hold whole snapshots. Restore failed rows,
+ // then reapply confirmed removals in one reactive batch so successes
+ // never reappear in REST fallback lists or search results.
+ batch(() => {
+ context?.soupSnapshot.rollback();
+ context?.searchSnapshot.rollback();
+ removeSoupEntities(deletedIds);
+ removeSearchEntities(deletedIds);
+ });
+ } else {
+ context?.soupSnapshot.rollback();
+ context?.searchSnapshot.rollback();
+ }
console.error(`Failed to delete dss items`, entities, error);
- toast.failure('Failed to delete items');
+ toast.failure(partialFailure ? error.message : 'Failed to delete items');
},
}));
}
diff --git a/apps/web/src/features/next-soup/actions/make-delete-action.test.ts b/apps/web/src/features/next-soup/actions/make-delete-action.test.ts
index 0da1cd5bfb..02ac3a926f 100644
--- a/apps/web/src/features/next-soup/actions/make-delete-action.test.ts
+++ b/apps/web/src/features/next-soup/actions/make-delete-action.test.ts
@@ -1,9 +1,16 @@
+import { BulkDeleteFailure } from '@app/features/entity/queries/bulk-delete-result';
import type { EntityData } from '@entity';
-import { beforeEach, describe, expect, it, vi } from 'vitest';
+import { createRoot, createSignal } from 'solid-js';
+import { beforeEach, describe, expect, it, onTestFinished, vi } from 'vitest';
const mocks = vi.hoisted(() => ({
openBulkEditModal: vi.fn(),
bulkDeleteMutateAsync: vi.fn(async () => []),
+ splitManager: undefined as object | undefined,
+ removeHistory: vi.fn(),
+ success: vi.fn(),
+ restoreFocus: vi.fn(),
+ trashEmails: vi.fn(),
}));
// The action pulls in the bulk-edit modal, split manager and toast at module
@@ -12,25 +19,44 @@ vi.mock('@app/features/entity/bulk-edit/BulkEditEntityModal', () => ({
openBulkEditModal: mocks.openBulkEditModal,
}));
vi.mock('@app/signal/splitLayout', () => ({
- globalSplitManager: () => undefined,
+ globalSplitManager: () => mocks.splitManager,
}));
vi.mock('@components/app/split-layout/layoutUtils', () => ({
- globalRemoveFromSplitHistory: vi.fn(),
+ globalRemoveFromSplitHistory: mocks.removeHistory,
}));
vi.mock('@core/component/Toast/Toast', () => ({
- toast: { success: vi.fn(), failure: vi.fn(), dismiss: vi.fn() },
+ toast: { success: mocks.success, failure: vi.fn(), dismiss: vi.fn() },
}));
// The real barrel reaches the query clients, which open websockets under jsdom.
vi.mock('@entity', () => ({
+ isEmailEntity: (entity: EntityData) => entity.type === 'email',
createBulkDeleteDssItemsMutation: () => ({
mutateAsync: mocks.bulkDeleteMutateAsync,
}),
}));
vi.mock('../utils', () => ({
- restoreSoupFocus: vi.fn(),
- trashEmails: vi.fn(),
+ restoreSoupFocus: mocks.restoreFocus,
+ trashEmails: mocks.trashEmails,
+}));
+
+vi.mock('@app/features/next-soup/filters/configs/', () => ({
+ SOUP_FILTERS: [],
+}));
+vi.mock('@app/features/next-soup/soup-view/sort-options', () => ({
+ SORT_CONFIGS: { updated_at: { id: 'updated_at', fn: () => 0 } },
+}));
+vi.mock('@core/mobile/inputModality', () => ({ isModality: () => false }));
+vi.mock('@core/mobile/isTouchDevice', () => ({ isTouchDevice: () => false }));
+vi.mock('./make-mark-done-action', () => ({
+ canExecuteMarkDoneOnView: () => true,
}));
+import { createListController } from '@app/components/list/create-list-controller';
+import { createSoupState, type SoupRow } from '../create-soup-state';
+import {
+ type EntityActionListState,
+ toEntityActionListState,
+} from './entity-action-context';
import { makeDeleteAction } from './make-delete-action';
const ME = 'macro|[email protected]';
@@ -43,7 +69,10 @@ const entity = (
const { canExecute, execute } = makeDeleteAction({ userId: () => ME });
-beforeEach(() => vi.clearAllMocks());
+beforeEach(() => {
+ vi.clearAllMocks();
+ mocks.splitManager = undefined;
+});
describe('makeDeleteAction.execute', () => {
const reminder = entity('reminder', { ownerId: '' });
@@ -55,6 +84,25 @@ describe('makeDeleteAction.execute', () => {
expect(mocks.openBulkEditModal).not.toHaveBeenCalled();
});
+ it('reports and removes history only for reminders that succeeded in a partial batch', async () => {
+ const deleted = entity('reminder', { id: 'deleted-reminder' });
+ const failed = entity('reminder', { id: 'failed-reminder' });
+ mocks.bulkDeleteMutateAsync.mockRejectedValueOnce(
+ new BulkDeleteFailure([deleted, failed], [true, false])
+ );
+ mocks.splitManager = {};
+ const onDeleted = vi.fn();
+ const action = makeDeleteAction({ userId: () => ME, onDeleted });
+ await action.execute([deleted, failed]);
+ await vi.waitFor(() => expect(onDeleted).toHaveBeenCalledWith([deleted]));
+ expect(mocks.success).not.toHaveBeenCalled();
+ const predicate = mocks.removeHistory.mock.calls[0][1] as (entry: {
+ id: string;
+ }) => boolean;
+ expect(predicate({ id: deleted.id })).toBe(true);
+ expect(predicate({ id: failed.id })).toBe(false);
+ });
+
it('still confirms for everything else', async () => {
await execute([entity('document')]);
@@ -90,6 +138,327 @@ describe('makeDeleteAction.execute', () => {
});
});
+type DeleteModal = {
+ onPartialDelete: (deleted: EntityData[], remaining: EntityData[]) => void;
+ onFinish: () => void;
+ onCancel?: () => void;
+};
+const currentDeleteModal = () =>
+ mocks.openBulkEditModal.mock.calls.at(-1)![0] as DeleteModal;
+
+function listState(entities: EntityData[]) {
+ let focused: string | undefined = entities[0]?.id;
+ const selected = new Set(entities.map((entity) => entity.id));
+ const rows = entities.map(
+ (original, index) =>
+ ({
+ id: original.id,
+ identityKey: original.id,
+ original,
+ index,
+ group: undefined,
+ getIsGrouped: () => false,
+ getIsLoadMore: () => false,
+ isFocused: () => focused === original.id,
+ isSelected: () => selected.has(original.id),
+ }) as SoupRow
+ );
+ const focusSet = vi.fn((id: string | undefined) => {
+ focused = id;
+ });
+ const clear = vi.fn(() => selected.clear());
+ const peekOffset = vi.fn<EntityActionListState['navigate']['peekOffset']>(
+ (direction, options) => {
+ const candidates = direction >= 0 ? rows : rows.toReversed();
+ const row = candidates.find((row) => !options?.skip?.(row));
+ return row ? { row, index: row.index } : undefined;
+ }
+ );
+ const soup: EntityActionListState = {
+ focus: {
+ id: () => focused,
+ index: () => rows.findIndex((row) => row.id === focused),
+ set: focusSet,
+ },
+ items: {
+ count: () => rows.length,
+ at: (index) => rows[index],
+ get: (id) => rows.find((row) => row.id === id),
+ },
+ navigate: { peekOffset },
+ selection: { clear },
+ collapseEntity: { shouldCollapse: () => false, callback: () => undefined },
+ };
+ return { soup, selected, focusSet, clear, rows };
+}
+
+function removedByCleanup(call: number, id: string) {
+ const predicate = mocks.removeHistory.mock.calls[call][1] as (entry: {
+ id: string;
+ }) => boolean;
+ return predicate({ id });
+}
+
+describe('confirmed partial deletion cleanup', () => {
+ const deleted = entity('document', { id: 'deleted' });
+ const failed = entity('document', { id: 'failed' });
+ const survivor = entity('document', { id: 'survivor' });
+
+ it('notifies direct callers immediately and does not report the same success again after retry', async () => {
+ mocks.splitManager = {};
+ const onDeleted = vi.fn();
+ const action = makeDeleteAction({ userId: () => ME, onDeleted });
+ await action.execute([deleted, failed]);
+ const modal = currentDeleteModal();
+ modal.onPartialDelete([deleted], [failed]);
+ expect(onDeleted).toHaveBeenCalledWith([deleted]);
+ expect(mocks.success).not.toHaveBeenCalled();
+ expect(removedByCleanup(0, deleted.id)).toBe(true);
+ expect(removedByCleanup(0, failed.id)).toBe(false);
+ modal.onFinish();
+ expect(onDeleted).toHaveBeenNthCalledWith(2, [failed]);
+ expect(onDeleted).toHaveBeenCalledTimes(2);
+ expect(removedByCleanup(1, deleted.id)).toBe(false);
+ expect(removedByCleanup(1, failed.id)).toBe(true);
+ });
+
+ it('cleans successful history/selection before Cancel and restores focus to a failed item', async () => {
+ mocks.splitManager = {};
+ const { soup, selected, focusSet } = listState([deleted, failed, survivor]);
+ const action = makeDeleteAction({ userId: () => ME });
+ await action.executeWithSoup([deleted, failed], soup);
+ const modal = currentDeleteModal();
+ modal.onPartialDelete([deleted], [failed]);
+ expect(selected.size).toBe(0);
+ expect(removedByCleanup(0, deleted.id)).toBe(true);
+ expect(removedByCleanup(0, failed.id)).toBe(false);
+ expect(mocks.success).not.toHaveBeenCalled();
+ modal.onCancel?.();
+ expect(focusSet).toHaveBeenLastCalledWith(failed.id);
+ expect(mocks.restoreFocus).toHaveBeenLastCalledWith(failed.id);
+ expect(mocks.removeHistory).toHaveBeenCalledOnce();
+ });
+
+ it('avoids already deleted focus targets after multiple partial attempts and a final success', async () => {
+ mocks.splitManager = {};
+ const second = entity('document', { id: 'second-deleted' });
+ const { soup, focusSet } = listState([deleted, second, failed, survivor]);
+ const action = makeDeleteAction({ userId: () => ME });
+ await action.executeWithSoup([deleted, second, failed], soup);
+ const modal = currentDeleteModal();
+ modal.onPartialDelete([deleted], [second, failed]);
+ modal.onPartialDelete([second], [failed]);
+ modal.onFinish();
+ expect(mocks.removeHistory).toHaveBeenCalledTimes(3);
+ expect(removedByCleanup(2, deleted.id)).toBe(false);
+ expect(removedByCleanup(2, second.id)).toBe(false);
+ expect(removedByCleanup(2, failed.id)).toBe(true);
+ expect(focusSet).toHaveBeenLastCalledWith(survivor.id);
+ });
+
+ it('uses a live surviving neighbour if the failed row disappeared before Cancel', async () => {
+ const { soup, rows, focusSet } = listState([deleted, failed, survivor]);
+ const action = makeDeleteAction({ userId: () => ME });
+ await action.executeWithSoup([deleted, failed], soup);
+ const modal = currentDeleteModal();
+ modal.onPartialDelete([deleted], [failed]);
+ rows.splice(
+ rows.findIndex((row) => row.id === failed.id),
+ 1
+ );
+ modal.onCancel?.();
+ expect(focusSet).toHaveBeenLastCalledWith(survivor.id);
+ expect(mocks.restoreFocus).toHaveBeenLastCalledWith(survivor.id);
+ });
+
+ it('does not run the deferred email deletion lane after a partial failure and Cancel', async () => {
+ const email = entity('email', { id: 'email' });
+ const { soup } = listState([deleted, failed, email]);
+ const action = makeDeleteAction({ userId: () => ME });
+ await action.executeWithSoup([deleted, failed, email], soup);
+ const modal = currentDeleteModal();
+ modal.onPartialDelete([deleted], [failed]);
+ modal.onCancel?.();
+ expect(mocks.trashEmails).not.toHaveBeenCalled();
+ });
+});
+
+function liveListState(
+ adapter: 'legacy' | 'controller',
+ initial: EntityData[],
+ focusId: string
+) {
+ return createRoot((dispose) => {
+ onTestFinished(dispose);
+ if (adapter === 'legacy') {
+ const soup = createSoupState({ initialData: initial });
+ soup.focus.set(focusId);
+ return {
+ soup,
+ setItems: (items: EntityData[]) =>
+ soup.setRows(
+ items.map((original, index) =>
+ soup.buildRow({ id: original.id, original, index })
+ )
+ ),
+ };
+ }
+ const [items, setItems] = createSignal(initial);
+ const controller = createListController({
+ items,
+ getKey: (item: EntityData) => `row:${item.id}`,
+ });
+ const soup = toEntityActionListState({
+ controller,
+ getEntity: (item) => item,
+ });
+ soup.focus.set(focusId);
+ return {
+ soup,
+ setItems: (next: EntityData[]) => {
+ setItems(next);
+ },
+ };
+ });
+}
+
+const focusCases = [
+ {
+ name: 'preserves the originally captured next neighbour',
+ initial: ['top', 'before', 'deleted', 'neighbour', 'retry', 'tail'],
+ final: ['top', 'before', 'neighbour', 'tail'],
+ expected: 'neighbour',
+ },
+ {
+ name: 'skips the rest of a contiguous deleted block',
+ initial: ['top', 'before', 'deleted', 'retry', 'neighbour', 'tail'],
+ final: ['top', 'before', 'neighbour', 'tail'],
+ expected: 'neighbour',
+ },
+ {
+ name: 'falls back to the preceding neighbour at the end of the list',
+ initial: ['top', 'before', 'deleted', 'retry'],
+ final: ['top', 'before'],
+ expected: 'before',
+ },
+ {
+ name: 'keeps the neighbour by identity when rows are inserted or reordered',
+ initial: ['top', 'before', 'deleted', 'neighbour', 'retry', 'tail'],
+ final: ['inserted', 'top', 'before', 'tail', 'neighbour'],
+ expected: 'neighbour',
+ },
+ {
+ name: 'uses the preceding anchor when the next neighbour also disappears',
+ initial: ['top', 'before', 'deleted', 'retry', 'neighbour', 'tail'],
+ final: ['top', 'before', 'tail'],
+ expected: 'before',
+ },
+ {
+ name: 'uses a bounded original position when both anchors disappear',
+ initial: ['top', 'before', 'deleted', 'retry', 'neighbour', 'tail'],
+ final: ['new-top', 'new-nearby'],
+ expected: 'new-nearby',
+ },
+ {
+ name: 'clears focus when no rows remain',
+ initial: ['deleted', 'retry'],
+ final: [],
+ expected: undefined,
+ },
+];
+
+describe.each(['legacy', 'controller'] as const)(
+ 'partial delete focus on the real %s list',
+ (adapter) => {
+ it.each(focusCases)('$name', async ({ initial, final, expected }) => {
+ const items = new Map(
+ [...initial, ...final].map((id) => [id, entity('document', { id })])
+ );
+ const { soup, setItems } = liveListState(
+ adapter,
+ initial.map((id) => items.get(id)!),
+ 'deleted'
+ );
+ const deleted = items.get('deleted')!;
+ const retry = items.get('retry')!;
+ const action = makeDeleteAction({ userId: () => ME });
+ await action.executeWithSoup([deleted, retry], soup);
+ const modal = currentDeleteModal();
+ // Real list updates invalidate the focused row's index. A mock that only
+ // skips deleted ids, without losing focus, misses the jump-to-top bug.
+ setItems(
+ initial.filter((id) => id !== deleted.id).map((id) => items.get(id)!)
+ );
+ modal.onPartialDelete([deleted], [retry]);
+ setItems(final.map((id) => items.get(id)!));
+ expect(soup.focus.index()).toBe(-1);
+ modal.onFinish();
+ const focusId = soup.focus.id();
+ expect(
+ focusId === undefined ? undefined : soup.items.get(focusId)?.original.id
+ ).toBe(expected);
+ expect(mocks.restoreFocus).toHaveBeenLastCalledWith(focusId);
+ });
+ }
+);
+
+it('skips grouped headers and load-more rows when anchoring partial-delete focus', async () => {
+ const soup = createRoot((dispose) => {
+ onTestFinished(dispose);
+ return createSoupState();
+ });
+ const top = entity('document', { id: 'top' });
+ const deleted = entity('document', { id: 'deleted' });
+ const retry = entity('document', { id: 'retry' });
+ const neighbour = entity('document', { id: 'neighbour' });
+ const group = {
+ key: 'group',
+ label: 'Group',
+ value: 'group',
+ count: 1,
+ isExpanded: () => true,
+ toggle: () => {},
+ };
+ const structural = [
+ soup.buildRow({
+ id: 'header',
+ index: 3,
+ original: neighbour,
+ group,
+ isGrouped: true,
+ }),
+ soup.buildRow({
+ id: 'more',
+ index: 4,
+ original: neighbour,
+ group,
+ isLoadMore: true,
+ }),
+ ];
+ soup.setRows([
+ ...[top, deleted, retry].map((original, index) =>
+ soup.buildRow({ id: original.id, original, index })
+ ),
+ ...structural,
+ soup.buildRow({ id: neighbour.id, original: neighbour, index: 5, group }),
+ ]);
+ soup.focus.set(deleted.id);
+ await makeDeleteAction({ userId: () => ME }).executeWithSoup(
+ [deleted, retry],
+ soup
+ );
+ const modal = currentDeleteModal();
+ modal.onPartialDelete([deleted], [retry]);
+ soup.setRows([
+ soup.buildRow({ id: top.id, original: top, index: 0 }),
+ ...structural.map((row, index) => ({ ...row, index: index + 1 })),
+ soup.buildRow({ id: neighbour.id, original: neighbour, index: 3, group }),
+ ]);
+ expect(soup.focus.index()).toBe(-1);
+ modal.onFinish();
+ expect(soup.focus.id()).toBe(neighbour.id);
+});
+
describe('makeDeleteAction.canExecute', () => {
it('only offers session deletion to its owner', () => {
expect(canExecute(entity('agent_session'))).toBe(true);
diff --git a/apps/web/src/features/next-soup/actions/make-delete-action.ts b/apps/web/src/features/next-soup/actions/make-delete-action.ts
index 49fcb9f491..87b742329d 100644
--- a/apps/web/src/features/next-soup/actions/make-delete-action.ts
+++ b/apps/web/src/features/next-soup/actions/make-delete-action.ts
@@ -1,4 +1,5 @@
import { openBulkEditModal } from '@app/features/entity/bulk-edit/BulkEditEntityModal';
+import { BulkDeleteFailure } from '@app/features/entity/queries/bulk-delete-result';
import { globalSplitManager } from '@app/signal/splitLayout';
import { globalRemoveFromSplitHistory } from '@components/app/split-layout/layoutUtils';
import { toast } from '@core/component/Toast/Toast';
@@ -8,6 +9,7 @@ import {
isEmailEntity,
} from '@entity';
import ArrowCounterClockwise from '@phosphor-icons/core/regular/arrow-counter-clockwise.svg?component-solid';
+import type { SoupRow } from '../create-soup-state';
import { restoreSoupFocus, trashEmails } from '../utils';
import type { EntityActionListState } from './entity-action-context';
@@ -16,6 +18,30 @@ type MakeDeleteOptions = {
onDeleted?: (entities: EntityData[]) => void;
};
+/** Cleanup follows confirmed outcomes, not whether the dialog eventually closes. */
+function createDeletionCleanup() {
+ const deletedIds = new Set<string>();
+ return {
+ deletedIds,
+ confirm: (entities: EntityData[]) => {
+ const newlyDeleted = entities.filter((entity) => {
+ if (deletedIds.has(entity.id)) return false;
+ deletedIds.add(entity.id);
+ return true;
+ });
+ if (newlyDeleted.length === 0) return newlyDeleted;
+ const splitManager = globalSplitManager();
+ if (splitManager) {
+ const ids = new Set(newlyDeleted.map((entity) => entity.id));
+ globalRemoveFromSplitHistory(splitManager, (entry) =>
+ ids.has(entry.id)
+ );
+ }
+ return newlyDeleted;
+ },
+ };
+}
+
export const makeDeleteAction = (options: MakeDeleteOptions) => {
const { userId } = options;
const bulkDelete = createBulkDeleteDssItemsMutation();
@@ -34,25 +60,37 @@ export const makeDeleteAction = (options: MakeDeleteOptions) => {
const deleteRemindersNow = async (reminders: EntityData[]) => {
if (reminders.length === 0) return;
try {
- await bulkDelete.mutateAsync(reminders);
+ let deleted = reminders;
+ try {
+ await bulkDelete.mutateAsync(reminders);
+ } catch (error) {
+ if (
+ !(error instanceof BulkDeleteFailure) ||
+ error.deletedEntities.length === 0
+ )
+ throw error;
+ deleted = error.deletedEntities;
+ }
// Only after the delete lands: the mutation restores the rows on
// failure, and a dropped split-history entry cannot be restored with
// them.
const splitManager = globalSplitManager();
if (splitManager) {
- const ids = new Set(reminders.map(({ id }) => id));
+ const ids = new Set(deleted.map(({ id }) => id));
globalRemoveFromSplitHistory(splitManager, (entry) =>
ids.has(entry.id)
);
}
- toast.success(
- reminders.length > 1
- ? `Deleted ${reminders.length} reminders`
- : 'Reminder deleted'
- );
- options.onDeleted?.(reminders);
+ if (deleted.length === reminders.length) {
+ toast.success(
+ reminders.length > 1
+ ? `Deleted ${reminders.length} reminders`
+ : 'Reminder deleted'
+ );
+ }
+ options.onDeleted?.(deleted);
} catch {
- // createBulkDeleteDssItemsMutation already toasts and restores the rows.
+ // The mutation already reports failure and restores the failed rows.
}
};
@@ -81,21 +119,20 @@ export const makeDeleteAction = (options: MakeDeleteOptions) => {
void deleteRemindersNow(reminders);
if (rest.length === 0) return;
+ const cleanup = createDeletionCleanup();
openBulkEditModal({
view: 'delete',
entities: rest,
+ onPartialDelete: (deleted) => {
+ const confirmed = cleanup.confirm(deleted);
+ if (confirmed.length > 0) options.onDeleted?.(confirmed);
+ },
onFinish: () => {
- const splitManager = globalSplitManager();
- if (splitManager) {
- const entityIdSet = new Set(rest.map(({ id }) => id));
- globalRemoveFromSplitHistory(splitManager, (entry) =>
- entityIdSet.has(entry.id)
- );
- }
+ const confirmed = cleanup.confirm(rest);
toast.success(
rest.length > 1 ? `Deleted ${rest.length} items` : 'Deleted'
);
- options.onDeleted?.(rest);
+ if (confirmed.length > 0) options.onDeleted?.(confirmed);
},
});
};
@@ -107,6 +144,30 @@ export const makeDeleteAction = (options: MakeDeleteOptions) => {
const currentIndex = soup.focus.index();
const nextRow =
soup.items.at(currentIndex + 1) ?? soup.items.at(currentIndex - 1);
+ const isFocusableRow = (row: SoupRow | undefined): row is SoupRow =>
+ row !== undefined &&
+ !row.getIsGrouped() &&
+ !row.getIsLoadMore() &&
+ (!row.group || row.group.isExpanded());
+ const captureAnchor = (row: SoupRow | undefined) =>
+ isFocusableRow(row)
+ ? { rowId: row.id, entityId: row.original.id }
+ : undefined;
+ const requestedIds = new Set(entities.map((entity) => entity.id));
+ const anchorNavigation = {
+ wrapNavigation: false,
+ skipGroupHeaders: true,
+ skipLoadMore: true,
+ skip: (row: SoupRow) => requestedIds.has(row.original.id),
+ };
+ // Capture identities while the focused row still exists. The immediate
+ // neighbour can belong to the selected block, so also remember the first
+ // non-target row on either side. Do not retain mutable row-store proxies.
+ const focusAnchors = [
+ captureAnchor(nextRow),
+ captureAnchor(soup.navigate.peekOffset(1, anchorNavigation)?.row),
+ captureAnchor(soup.navigate.peekOffset(-1, anchorNavigation)?.row),
+ ];
// Three lanes: emails trash immediately (with Undo), reminders delete
// immediately (no Undo to give), everything else confirms first.
@@ -116,10 +177,44 @@ export const makeDeleteAction = (options: MakeDeleteOptions) => {
(e) => e.type !== 'email' && e.type !== 'reminder'
);
+ const cleanup = createDeletionCleanup();
+ let remainingEntities: EntityData[] = nonEmailEntities;
+ let hadPartialDeletion = false;
+ const nextSurvivingRow = (alsoRemoved: string[] = []) => {
+ if (!hadPartialDeletion) return nextRow;
+ const removed = new Set([...cleanup.deletedIds, ...alsoRemoved]);
+ const survives = (row: SoupRow | undefined): row is SoupRow =>
+ isFocusableRow(row) && !removed.has(row.original.id);
+ for (const anchor of focusAnchors) {
+ if (!anchor || removed.has(anchor.entityId)) continue;
+ const keyed = soup.items.get(anchor.rowId);
+ const live =
+ keyed?.original.id === anchor.entityId
+ ? keyed
+ : soup.items.get(anchor.entityId);
+ if (survives(live) && live.original.id === anchor.entityId) return live;
+ }
+ // A concurrent refresh can remove both neighbours. Search around the
+ // captured position, not the now-missing focus (peekOffset would start
+ // from the first/last row). The index is bounded by the current list.
+ const count = soup.items.count();
+ if (count === 0) return undefined;
+ const start = Math.min(Math.max(currentIndex, 0), count - 1);
+ for (let index = start; index < count; index += 1) {
+ const row = soup.items.at(index);
+ if (survives(row)) return row;
+ }
+ for (let index = start - 1; index >= 0; index -= 1) {
+ const row = soup.items.at(index);
+ if (survives(row)) return row;
+ }
+ return undefined;
+ };
const advancePastDeleted = () => {
soup.selection.clear();
- if (nextRow) soup.focus.set(nextRow.id);
- restoreSoupFocus(nextRow?.id);
+ const next = nextSurvivingRow();
+ if (next || hadPartialDeletion) soup.focus.set(next?.id);
+ restoreSoupFocus(next?.id);
};
const trashEmailEntities = () => {
@@ -136,8 +231,9 @@ export const makeDeleteAction = (options: MakeDeleteOptions) => {
}
soup.selection.clear();
- if (nextRow) {
- soup.focus.set(nextRow.id);
+ const next = nextSurvivingRow(emailEntities.map((entity) => entity.id));
+ if (next || hadPartialDeletion) {
+ soup.focus.set(next?.id);
}
const toastId = toast.success(
@@ -167,7 +263,7 @@ export const makeDeleteAction = (options: MakeDeleteOptions) => {
toast.failure('Failed to move to Trash');
});
- restoreSoupFocus(nextRow?.id);
+ restoreSoupFocus(next?.id);
};
if (nonEmailEntities.length > 0) {
@@ -176,15 +272,14 @@ export const makeDeleteAction = (options: MakeDeleteOptions) => {
openBulkEditModal({
view: 'delete',
entities: nonEmailEntities,
+ onPartialDelete: (deleted, remaining) => {
+ hadPartialDeletion = true;
+ remainingEntities = remaining;
+ soup.selection.clear();
+ cleanup.confirm(deleted);
+ },
onFinish: () => {
- const splitManager = globalSplitManager();
- if (splitManager) {
- const entityIdSet = new Set(nonEmailEntities.map(({ id }) => id));
- globalRemoveFromSplitHistory(splitManager, (entry) =>
- entityIdSet.has(entry.id)
- );
- }
-
+ cleanup.confirm(nonEmailEntities);
toast.success(
nonEmailEntities.length > 1
? `Deleted ${nonEmailEntities.length} items`
@@ -198,11 +293,18 @@ export const makeDeleteAction = (options: MakeDeleteOptions) => {
}
},
onCancel: () => {
- const firstEntity = nonEmailEntities[0];
- if (firstEntity) {
- soup.focus.set(firstEntity.id);
- }
- restoreSoupFocus(firstEntity?.id);
+ const firstRemaining = hadPartialDeletion
+ ? remainingEntities.find(
+ (entity) =>
+ !cleanup.deletedIds.has(entity.id) &&
+ soup.items.get(entity.id)
+ )
+ : nonEmailEntities[0];
+ const target =
+ firstRemaining?.id ??
+ (hadPartialDeletion ? nextSurvivingRow()?.id : undefined);
+ if (target || hadPartialDeletion) soup.focus.set(target);
+ restoreSoupFocus(target);
},
});
} else if (emailEntities.length > 0) {
diff --git a/apps/web/src/lib/graphql-cache/host/worker-host.test.ts b/apps/web/src/lib/graphql-cache/host/worker-host.test.ts
index 36fca67ebb..e84ecc165a 100644
--- a/apps/web/src/lib/graphql-cache/host/worker-host.test.ts
+++ b/apps/web/src/lib/graphql-cache/host/worker-host.test.ts
@@ -18,6 +18,11 @@ vi.mock('../worker/coordinator-page-adapter', () => ({
createCacheCoordinatorPageAdapter: adapterFactory,
}));
+import { CACHE_COORDINATOR_PROTOCOL_VERSION } from '../worker/coordinator-protocol';
+import {
+ CacheBootstrapExhaustedError,
+ COORDINATOR_CONNECT_TIMEOUT_MS,
+} from '../worker/startup';
import { createWorkerCacheHost } from './worker-host';
const CLIENT_ID = '00000000-0000-4000-8000-000000000007';
@@ -75,6 +80,7 @@ function responseFor(request: CacheRequest): unknown {
class FakePageAdapter {
onmessage: ((event: MessageEvent<WorkerMessage>) => void) | null = null;
readonly requests: CacheRequest[] = [];
+ readonly start = vi.fn(async (): Promise<void> => {});
readonly ignoredKinds = new Set<CacheRequest['kind']>();
readonly errors = new Map<CacheRequest['kind'], string>();
readonly dispose = vi.fn(
@@ -336,7 +342,7 @@ describe('createWorkerCacheHost', () => {
const read = host.readQuery({ query: '{ x }' });
expect(adapterFactory).toHaveBeenCalledOnce();
- expect(indexedDbDelete).toHaveBeenCalledOnce();
+ await vi.waitFor(() => expect(indexedDbDelete).toHaveBeenCalledOnce());
expect(indexedDbDelete).toHaveBeenCalledWith('graphql-cache:lazy-scope');
expect(adapterFactory).toHaveBeenCalledWith(
expect.objectContaining({ scope: 'lazy-scope' })
@@ -515,6 +521,159 @@ describe('createWorkerCacheHost', () => {
host.dispose();
});
+ it('follows coordinator phase deadlines instead of applying the read timeout to startup', async () => {
+ vi.useFakeTimers();
+ configureAdapter = (fake) => {
+ fake.ignoredKinds.add('init');
+ fake.ignoredKinds.add('read');
+ };
+ const onInitializationError = vi.fn();
+ const host = createWorkerCacheHost({
+ scope: 'scope-1',
+ requestTimeoutMs: 10,
+ onInitializationError,
+ });
+ const read = host.readQuery({ query: '{ user }' });
+ const rejected = expect(read).rejects.toThrow('cache worker timeout: read');
+ await vi.advanceTimersByTimeAsync(0);
+ const adapter = requireAdapter();
+ const progress = (
+ ownerEpoch: number,
+ phase: 'loading-assets' | 'opening-database',
+ timeoutMs: number
+ ) =>
+ adapter.options.onStartupProgress?.({
+ coordinatorVersion: CACHE_COORDINATOR_PROTOCOL_VERSION,
+ kind: 'engine-startup',
+ ownerEpoch,
+ databaseAction: 'open-existing',
+ phase,
+ timeoutMs,
+ });
+ progress(1, 'loading-assets', 60_000);
+ await vi.advanceTimersByTimeAsync(30_000);
+ expect(onInitializationError).not.toHaveBeenCalled();
+ progress(1, 'opening-database', 20_000);
+ await vi.advanceTimersByTimeAsync(20_001);
+ // The coordinator retries before the page's grace expires.
+ progress(2, 'loading-assets', 60_000);
+ await vi.advanceTimersByTimeAsync(30_000);
+ expect(onInitializationError).not.toHaveBeenCalled();
+ adapter.respond(1, null);
+ await vi.advanceTimersByTimeAsync(11);
+ await rejected;
+ expect(onInitializationError).not.toHaveBeenCalled();
+ host.dispose();
+ });
+
+ it('retries a transient coordinator registration failure without disabling or quarantining the cache', async () => {
+ let attempt = 0;
+ let first: FakePageAdapter | undefined;
+ configureAdapter = (fake) => {
+ if (attempt++ !== 0) return;
+ first = fake;
+ fake.start.mockImplementation(async () => {
+ const error = new Error('SharedWorker load failed');
+ fake.terminalError(error);
+ throw error;
+ });
+ };
+ const onInitializationError = vi.fn();
+ const host = createWorkerCacheHost({
+ scope: 'scope-1',
+ onInitializationError,
+ });
+ await expect(host.readQuery({ query: '{ user }' })).resolves.toEqual({
+ kind: 'miss',
+ });
+ expect(adapterFactory).toHaveBeenCalledTimes(2);
+ expect(first?.dispose).toHaveBeenCalledWith({ graceful: false });
+ expect(first?.requests).toEqual([]);
+ expect(onInitializationError).not.toHaveBeenCalled();
+ host.dispose();
+ });
+
+ it('bounds hung coordinator registration retries and reports terminal failure only once', async () => {
+ vi.useFakeTimers();
+ configureAdapter = (fake) =>
+ fake.start.mockImplementation(() => new Promise(() => {}));
+ const onInitializationError = vi.fn();
+ const host = createWorkerCacheHost({
+ scope: 'scope-1',
+ onInitializationError,
+ });
+ const rejected = expect(host.currentRevision()).rejects.toThrow(
+ 'cache worker timeout: registration'
+ );
+ await vi.advanceTimersByTimeAsync(COORDINATOR_CONNECT_TIMEOUT_MS * 3 + 1);
+ await rejected;
+ expect(adapterFactory).toHaveBeenCalledTimes(3);
+ expect(onInitializationError).toHaveBeenCalledOnce();
+ expect(requireAdapter().requests).toEqual([]);
+ expect(vi.getTimerCount()).toBe(0);
+ });
+
+ it('cancels coordinator registration on disposal without retrying or reporting failure', async () => {
+ vi.useFakeTimers();
+ configureAdapter = (fake) =>
+ fake.start.mockImplementation(() => new Promise(() => {}));
+ const onInitializationError = vi.fn();
+ const host = createWorkerCacheHost({
+ scope: 'scope-1',
+ onInitializationError,
+ });
+ const rejected = expect(host.currentRevision()).rejects.toThrow(
+ 'disposed during startup'
+ );
+ host.dispose();
+ await rejected;
+ expect(adapterFactory).toHaveBeenCalledOnce();
+ expect(onInitializationError).not.toHaveBeenCalled();
+ expect(vi.getTimerCount()).toBe(0);
+ });
+
+ it.each([
+ ['open-existing', true],
+ ['open-existing', false],
+ ['wipe-before-open', false],
+ ] as const)(
+ 'requires coordinator proof to preserve a failed startup scope (%s, proof: %s)',
+ async (databaseAction, provenUntouched) => {
+ configureAdapter = (fake) => fake.ignoredKinds.add('init');
+ localStorage.setItem('graphql-cache:scope', 'scope-1');
+ const onInitializationError = vi.fn();
+ const host = createWorkerCacheHost({
+ scope: 'scope-1',
+ onInitializationError,
+ });
+ const rejected = expect(host.currentRevision()).rejects.toThrow(
+ 'bootstrap attempts exhausted'
+ );
+ const adapter = requireAdapter();
+ await vi.waitFor(() => expect(adapter.requests).toHaveLength(1));
+ adapter.options.onStartupProgress?.({
+ coordinatorVersion: CACHE_COORDINATOR_PROTOCOL_VERSION,
+ kind: 'engine-startup',
+ ownerEpoch: 1,
+ phase: 'loading-assets',
+ databaseAction,
+ timeoutMs: 60_000,
+ });
+ adapter.terminalError(
+ provenUntouched
+ ? new CacheBootstrapExhaustedError('bootstrap attempts exhausted')
+ : new Error('bootstrap attempts exhausted')
+ );
+ await rejected;
+ await vi.waitFor(() =>
+ expect(onInitializationError).toHaveBeenCalledOnce()
+ );
+ expect(localStorage.getItem('graphql-cache:scope')).toBe(
+ provenUntouched ? 'scope-1' : 'quarantine:scope-1'
+ );
+ }
+ );
+
it('bounds hung registration/init before admitting reads or mutations', async () => {
vi.useFakeTimers();
configureAdapter = (fake) => fake.ignoredKinds.add('init');
@@ -605,6 +764,7 @@ describe('createWorkerCacheHost', () => {
errorCode: 'owner-epoch-lost',
});
const adapter = requireAdapter();
+ await vi.waitFor(() => expect(adapter.requests).toHaveLength(1));
adapter.reject(1, 'owner epoch 1 was lost', 'owner-epoch-lost');
await Promise.all([readRejected, mutationRejected]);
@@ -651,6 +811,7 @@ describe('createWorkerCacheHost', () => {
const read = host.readQuery({ opKey: 5, query: 'query Read { user }' });
const adapter = requireAdapter();
+ await vi.waitFor(() => expect(adapter.requests).toHaveLength(1));
adapter.replace(2);
adapter.respond(1, null);
await expect(read).resolves.toEqual({ kind: 'miss' });
diff --git a/apps/web/src/lib/graphql-cache/host/worker-host.ts b/apps/web/src/lib/graphql-cache/host/worker-host.ts
index 0053a1a098..aa6183b64f 100644
--- a/apps/web/src/lib/graphql-cache/host/worker-host.ts
+++ b/apps/web/src/lib/graphql-cache/host/worker-host.ts
@@ -50,6 +50,12 @@ import {
type CacheCoordinatorPageAdapter,
createCacheCoordinatorPageAdapter,
} from '../worker/coordinator-page-adapter';
+import {
+ CacheBootstrapExhaustedError,
+ COORDINATOR_CONNECT_ATTEMPTS,
+ COORDINATOR_CONNECT_TIMEOUT_MS,
+ STARTUP_RESPONSE_GRACE_MS,
+} from '../worker/startup';
import { createNoopCacheHost } from './noop-host';
import type {
CacheHost,
@@ -95,8 +101,9 @@ export interface WorkerHostOptions {
*/
requestTimeoutMs?: number;
/**
- * Registration/initialization timeout in ms. Defaults to requestTimeoutMs,
- * so callers cannot hang before a read-only request timer can start.
+ * Overrides each startup-phase timeout (primarily for tests). By default,
+ * registration has its own budget and engine startup follows the coordinator's
+ * asset/open deadlines, with a response grace period.
*/
initializationTimeoutMs?: number;
/** Reports terminal initialization or coordinator-transport failure. */
@@ -201,8 +208,10 @@ export function createWorkerCacheHost(options: WorkerHostOptions): CacheHost {
>();
const requestTimeoutMs =
options.requestTimeoutMs ?? DEFAULT_REQUEST_TIMEOUT_MS;
- const initializationTimeoutMs =
- options.initializationTimeoutMs ?? requestTimeoutMs;
+ let initializationTimeoutMs =
+ options.initializationTimeoutMs ?? COORDINATOR_CONNECT_TIMEOUT_MS;
+ let registrationInProgress = false;
+ let cancelRegistration: (() => void) | undefined;
let nextRequestId = 1;
let state: HostState = 'idle';
let initialization: Promise<void> | undefined;
@@ -213,6 +222,7 @@ export function createWorkerCacheHost(options: WorkerHostOptions): CacheHost {
let failureReported = false;
let terminalFailureHandled = false;
let adapter: CacheCoordinatorPageAdapter | undefined;
+ let registeredAdapter: CacheCoordinatorPageAdapter | undefined;
let adapterDisposePromise: Promise<void> | undefined;
let adapterDisposeWasGraceful = false;
let adapterDisposalStarted = false;
@@ -386,7 +396,22 @@ export function createWorkerCacheHost(options: WorkerHostOptions): CacheHost {
scope: options.scope,
hotCapacity: options.hotCapacity,
onEngineReplaced,
- onTerminalError: failTransport,
+ onStartupProgress: (progress) => {
+ if (adapter !== created) return;
+ initializationTimeoutMs =
+ options.initializationTimeoutMs ??
+ progress.timeoutMs + STARTUP_RESPONSE_GRACE_MS;
+ for (const [id, entry] of pending) {
+ if (entry.kind === 'init')
+ armRequestTimeout(id, entry, initializationTimeoutMs);
+ }
+ },
+ onTerminalError: (error) => {
+ // start() also rejects; let the bounded registration loop retry before
+ // reporting a terminal failure or quarantining any cache scope.
+ if (adapter === created && !registrationInProgress)
+ failTransport(error);
+ },
telemetry,
});
created.onmessage = onMessage;
@@ -395,6 +420,54 @@ export function createWorkerCacheHost(options: WorkerHostOptions): CacheHost {
return created;
}
+ async function registerAdapter(): Promise<void> {
+ registrationInProgress = true;
+ try {
+ for (
+ let attempt = 0;
+ attempt < COORDINATOR_CONNECT_ATTEMPTS;
+ attempt += 1
+ ) {
+ const current = getAdapter();
+ let timer: ReturnType<typeof setTimeout> | undefined;
+ try {
+ await Promise.race([
+ current.start(),
+ new Promise<never>((_resolve, reject) => {
+ cancelRegistration = () =>
+ reject(
+ new Error('cache worker host was disposed during startup')
+ );
+ timer = setTimeout(
+ () => reject(new Error('cache worker timeout: registration')),
+ options.initializationTimeoutMs ??
+ COORDINATOR_CONNECT_TIMEOUT_MS
+ );
+ }),
+ ]);
+ if (state !== 'initializing')
+ throw new Error('cache worker host was disposed during startup');
+ registeredAdapter = current;
+ return;
+ } catch (error) {
+ current.onmessage = null;
+ await current.dispose({ graceful: false });
+ if (adapter === current) adapter = undefined;
+ if (
+ state !== 'initializing' ||
+ attempt + 1 === COORDINATOR_CONNECT_ATTEMPTS
+ )
+ throw error;
+ } finally {
+ if (timer !== undefined) clearTimeout(timer);
+ cancelRegistration = undefined;
+ }
+ }
+ } finally {
+ registrationInProgress = false;
+ }
+ }
+
function startLegacyIdbDeletion(): void {
if (legacyIdbDeletionStarted) return;
legacyIdbDeletionStarted = true;
@@ -475,7 +548,13 @@ export function createWorkerCacheHost(options: WorkerHostOptions): CacheHost {
}
function failInitialization(error: Error): void {
- if (state === 'failed' || state === 'disposed' || state === 'ready') return;
+ if (
+ state === 'failed' ||
+ state === 'disposing' ||
+ state === 'disposed' ||
+ state === 'ready'
+ )
+ return;
telemetry?.record({
name: 'graphql_cache.host_ready',
operationCategory: 'initialization',
@@ -502,6 +581,7 @@ export function createWorkerCacheHost(options: WorkerHostOptions): CacheHost {
function failTransport(error: Error): void {
if (terminalFailureHandled) return;
+ const storageWasUntouched = error instanceof CacheBootstrapExhaustedError;
stopStorageHealthSampling();
const admittedWorkIsUncertain = [...pending.values()].some(
(entry) => entry.admitted
@@ -517,7 +597,10 @@ export function createWorkerCacheHost(options: WorkerHostOptions): CacheHost {
void disposeAdapter(false).then(finishTelemetry);
// Product failure handling may immediately construct another host. Make
// the matching old scope unreachable before invoking that callback.
- void quarantineCacheScope(options.scope).then(() => {
+ const quarantine = storageWasUntouched
+ ? Promise.resolve()
+ : quarantineCacheScope(options.scope);
+ void quarantine.then(() => {
try {
reportFailure(error);
} catch {
@@ -569,6 +652,7 @@ export function createWorkerCacheHost(options: WorkerHostOptions): CacheHost {
}
function disposeHost(graceful: boolean, preserveDatabase = false): void {
+ cancelRegistration?.();
stopStorageHealthSampling();
if (state === 'disposed') return;
if (state === 'disposing') {
@@ -633,6 +717,22 @@ export function createWorkerCacheHost(options: WorkerHostOptions): CacheHost {
disposeHost(false, true);
}
+ function armRequestTimeout(
+ id: number,
+ entry: Pending,
+ timeoutMs: number
+ ): void {
+ if (entry.timer !== undefined) clearTimeout(entry.timer);
+ entry.timer = setTimeout(() => {
+ if (pending.delete(id)) {
+ const error = new Error(`cache worker timeout: ${entry.kind}`);
+ recordRequestOutcome(entry, 'error', error);
+ entry.reject(error);
+ finishGracefulDisposeIfDrained();
+ }
+ }, timeoutMs);
+ }
+
function request(
msg: DistributiveOmit<CacheRequest, 'id'>,
opKey?: number
@@ -669,14 +769,7 @@ export function createWorkerCacheHost(options: WorkerHostOptions): CacheHost {
? requestTimeoutMs
: undefined;
if (timeoutMs !== undefined) {
- entry.timer = setTimeout(() => {
- if (pending.delete(id)) {
- const error = new Error(`cache worker timeout: ${msg.kind}`);
- recordRequestOutcome(entry, 'error', error);
- reject(error);
- finishGracefulDisposeIfDrained();
- }
- }, timeoutMs);
+ armRequestTimeout(id, entry, timeoutMs);
}
pending.set(id, entry);
try {
@@ -746,11 +839,20 @@ export function createWorkerCacheHost(options: WorkerHostOptions): CacheHost {
state = 'initializing';
replacementError = undefined;
initializationStartedAt = now();
- const handshake = request({
- kind: 'init',
- scope: options.scope,
- hotCapacity: options.hotCapacity,
- }).then(
+ const sendInit = () =>
+ request({
+ kind: 'init',
+ scope: options.scope,
+ hotCapacity: options.hotCapacity,
+ });
+ const ready =
+ adapter && registeredAdapter === adapter
+ ? sendInit()
+ : (async () => {
+ await registerAdapter();
+ return await sendInit();
+ })();
+ const handshake = ready.then(
() => {
if (state !== 'initializing') return;
state = 'ready';
diff --git a/apps/web/src/lib/graphql-cache/worker/browser-test/browser-test-wasm-module.ts b/apps/web/src/lib/graphql-cache/worker/browser-test/browser-test-wasm-module.ts
index 56c112c02f..45b4a7f049 100644
--- a/apps/web/src/lib/graphql-cache/worker/browser-test/browser-test-wasm-module.ts
+++ b/apps/web/src/lib/graphql-cache/worker/browser-test/browser-test-wasm-module.ts
@@ -1,17 +1,29 @@
-type BrowserTestCacheWasmModule = {
+import { loadCacheWasm } from '../wasm-module';
+import {
+ assertMatchingCacheWasmBuilds,
+ type CacheWasmBuildInfo,
+ type CacheWasmBuildInfoSource,
+} from './wasm-build-compatibility';
+
+type BrowserTestCacheWasmModule = CacheWasmBuildInfoSource & {
default(input?: { module_or_path?: unknown }): Promise<unknown>;
browserTestMakeNamespaceIncompatible(scope: string): Promise<void>;
browserTestCorruptQueuePayload(scope: string): Promise<void>;
};
let modulePromise:
- | Promise<{ module: BrowserTestCacheWasmModule; wasmUrl: string }>
+ | Promise<{
+ module: BrowserTestCacheWasmModule;
+ wasmUrl: string;
+ buildInfo: CacheWasmBuildInfo;
+ }>
| undefined;
-/** Loads the separate feature-gated destructive-hook artifact for tests only. */
+/** Loads and verifies the test/production artifact pair without touching OPFS. */
export function loadBrowserTestCacheWasm(): Promise<{
module: BrowserTestCacheWasmModule;
wasmUrl: string;
+ buildInfo: CacheWasmBuildInfo;
}> {
if (modulePromise) return modulePromise;
const initialization = (async () => {
@@ -32,7 +44,9 @@ export function loadBrowserTestCacheWasm(): Promise<{
}
const compiled = await WebAssembly.compile(await response.arrayBuffer());
await module.default({ module_or_path: compiled });
- return { module, wasmUrl };
+ const production = await loadCacheWasm();
+ const buildInfo = assertMatchingCacheWasmBuilds(production, module);
+ return { module, wasmUrl, buildInfo };
})();
modulePromise = initialization;
void initialization.catch(() => {
diff --git a/apps/web/src/lib/graphql-cache/worker/browser-test/cache-artifact-compatibility.browser.e2e.ts b/apps/web/src/lib/graphql-cache/worker/browser-test/cache-artifact-compatibility.browser.e2e.ts
new file mode 100644
index 0000000000..34fae47579
--- /dev/null
+++ b/apps/web/src/lib/graphql-cache/worker/browser-test/cache-artifact-compatibility.browser.e2e.ts
@@ -0,0 +1,47 @@
+import { expect, test } from '@playwright/test';
+
+test('recovery preflight rejects stale helper metadata before opening or modifying OPFS', async ({
+ context,
+ page,
+}) => {
+ let intercepted = false;
+ const engineRequests: string[] = [];
+ context.on('request', (request) => {
+ if (/cache\.engine-worker[^/]*\.js$/.test(request.url()))
+ engineRequests.push(request.url());
+ });
+ await context.route(
+ /cache_wasm_browser_test_hooks-[^/]+\.js$/,
+ async (route) => {
+ const response = await route.fetch();
+ intercepted = true;
+ // Emulate an old helper without checking an obsolete WASM binary into git.
+ const body = `${await response.text()}\nconst originalBuildInfo = cacheBuildInfo;\ncacheBuildInfo = () => ({ ...originalBuildInfo(), formatVersion: 0 });\n`;
+ await route.fulfill({ response, body });
+ }
+ );
+ await page.goto('/app/cache-recovery.html');
+ await expect(page.locator('#result')).toHaveAttribute('data-status', 'ready');
+ const error = await page.evaluate(async () => {
+ try {
+ await window.cacheRecoveryHarness.runRecoveryKind(
+ 'incompatible-namespace'
+ );
+ return undefined;
+ } catch (error) {
+ return error instanceof Error ? error.message : String(error);
+ }
+ });
+ expect(intercepted).toBe(true);
+ expect(error).toContain('Cache recovery fixture WASM mismatch');
+ expect(error).toContain('just build-cache-wasm-browser-production');
+ expect(engineRequests).toEqual([]);
+ expect(
+ await page.evaluate(async () => {
+ const root = await navigator.storage.getDirectory();
+ const entries: string[] = [];
+ for await (const name of root.keys()) entries.push(name);
+ return entries;
+ })
+ ).toEqual([]);
+});
diff --git a/apps/web/src/lib/graphql-cache/worker/browser-test/cache-lifecycle-harness.ts b/apps/web/src/lib/graphql-cache/worker/browser-test/cache-lifecycle-harness.ts
index 70d3118fd2..2c21603da7 100644
--- a/apps/web/src/lib/graphql-cache/worker/browser-test/cache-lifecycle-harness.ts
+++ b/apps/web/src/lib/graphql-cache/worker/browser-test/cache-lifecycle-harness.ts
@@ -78,7 +78,6 @@ async function startOwner(registerForLogout = false): Promise<void> {
owner = createWorkerCacheHost({
scope,
requestTimeoutMs: 20_000,
- initializationTimeoutMs: 20_000,
rolloutCohort: 'treatment',
});
if (registerForLogout) unregisterOwner = registerCacheHost(owner);
diff --git a/apps/web/src/lib/graphql-cache/worker/browser-test/cache-recovery-harness.ts b/apps/web/src/lib/graphql-cache/worker/browser-test/cache-recovery-harness.ts
index fd37d89163..aff787ba94 100644
--- a/apps/web/src/lib/graphql-cache/worker/browser-test/cache-recovery-harness.ts
+++ b/apps/web/src/lib/graphql-cache/worker/browser-test/cache-recovery-harness.ts
@@ -538,9 +538,9 @@ async function runFaultKind(kind: MutatingRequestKind) {
}
}
-async function applyStorageMutation(
+async function runStorageControl(
scope: string,
- kind: RecoveryKind
+ kind: RecoveryKind | 'verify-artifacts'
): Promise<string> {
const worker = new Worker(
new URL('./production-cache.storage-control-worker.ts', import.meta.url),
@@ -578,6 +578,8 @@ async function applyStorageMutation(
async function runRecoveryKind(kind: RecoveryKind) {
const scope = `cache-recovery-${kind}-${crypto.randomUUID()}`;
+ // Fail before seeding or corrupting storage if either generated artifact is stale.
+ await runStorageControl(scope, 'verify-artifacts');
const session = new ProductionSession(scope);
try {
await session.openTab('before-corruption');
@@ -590,7 +592,7 @@ async function runRecoveryKind(kind: RecoveryKind) {
request: enqueueRequest('recovery-seed-owner', 10),
});
await session.gracefulClose('before-corruption');
- const browserTestWasmUrl = await applyStorageMutation(scope, kind);
+ const browserTestWasmUrl = await runStorageControl(scope, kind);
const telemetryBeforeReopen = session.cacheTelemetry.length;
await session.openTab('after-corruption');
await waitUntil('recovery outcome and authoritative wipe telemetry', () => {
diff --git a/apps/web/src/lib/graphql-cache/worker/browser-test/cache-startup.browser.e2e.ts b/apps/web/src/lib/graphql-cache/worker/browser-test/cache-startup.browser.e2e.ts
new file mode 100644
index 0000000000..52ce360cea
--- /dev/null
+++ b/apps/web/src/lib/graphql-cache/worker/browser-test/cache-startup.browser.e2e.ts
@@ -0,0 +1,62 @@
+import { expect, test } from '@playwright/test';
+
+for (const failure of ['slow-download', 'failed-download'] as const) {
+ test(`cache startup preserves existing data through ${failure}`, async ({
+ context,
+ page,
+ }, testInfo) => {
+ const path = testInfo.project.name.includes('production')
+ ? '/app/cache-lifecycle.html'
+ : '/cache-lifecycle.html';
+ const scope = `cache-startup-${crypto.randomUUID()}`;
+ const url = `${path}?treatment=true&scope=${scope}`;
+ await page.goto(url);
+ await expect(page.locator('#result')).toHaveAttribute(
+ 'data-status',
+ 'ready'
+ );
+ await page.evaluate(() => window.cacheLifecycleHarness.startSingle());
+ await page.evaluate(() =>
+ window.cacheLifecycleHarness.write('startup-preserved')
+ );
+ await page.goto('about:blank');
+
+ let downloads = 0;
+ await context.route(
+ /\/cache_wasm_bg[^/]*\.wasm(?:\?.*)?$/,
+ async (route) => {
+ downloads += 1;
+ if (downloads === 1) {
+ if (failure === 'failed-download') {
+ await route.fulfill({
+ status: 503,
+ body: 'Transient asset failure',
+ });
+ return;
+ }
+ // Longer than both former deadlines: 10s host init and 20s activation.
+ await new Promise((resolve) => setTimeout(resolve, 25_000));
+ }
+ await route.continue();
+ }
+ );
+ await page.goto(url);
+ await expect(page.locator('#result')).toHaveAttribute(
+ 'data-status',
+ 'ready'
+ );
+ await page.evaluate(() => window.cacheLifecycleHarness.startSingle());
+ expect(downloads).toBe(failure === 'slow-download' ? 1 : 2);
+ expect(
+ await page.evaluate(() =>
+ window.cacheLifecycleHarness.engineWorkerCount()
+ )
+ ).toBe(failure === 'slow-download' ? 1 : 2);
+ expect(
+ await page.evaluate(() => window.cacheLifecycleHarness.read())
+ ).toMatchObject({
+ kind: 'hit',
+ data: { user: { soup: { items: [{ id: 'startup-preserved' }] } } },
+ });
+ });
+}
diff --git a/apps/web/src/lib/graphql-cache/worker/browser-test/fake-cache.engine-worker.ts b/apps/web/src/lib/graphql-cache/worker/browser-test/fake-cache.engine-worker.ts
index 746e4a7f7b..eedb3d5aa0 100644
--- a/apps/web/src/lib/graphql-cache/worker/browser-test/fake-cache.engine-worker.ts
+++ b/apps/web/src/lib/graphql-cache/worker/browser-test/fake-cache.engine-worker.ts
@@ -16,7 +16,7 @@ import {
declare const self: DedicatedWorkerGlobalScope;
-const withVersion = <T extends { coordinatorVersion: 2 }>(
+const withVersion = <T extends { coordinatorVersion: 3 }>(
value: T extends unknown ? Omit<T, 'coordinatorVersion'> : never
): T =>
({
@@ -157,6 +157,28 @@ async function activate(
activationValue = activation;
controlPort = port;
telemetry = new BroadcastChannel(`graphql-cache-wp08:${activation.scope}`);
+ await new Promise<void>((resolve) => {
+ port.onmessage = ({ data }) => {
+ const parsed = validateCoordinatorToEngineEnvelope(
+ Array.isArray(data) ? data[1] : data
+ );
+ if (
+ parsed.ok &&
+ parsed.value.kind === 'open-engine' &&
+ parsed.value.ownerEpoch === activation.ownerEpoch
+ )
+ resolve();
+ };
+ port.start();
+ port.postMessage([0]);
+ sendEngine(
+ withVersion<EngineToCoordinatorEnvelope>({
+ kind: 'engine-assets-ready',
+ tabId: activation.tabId,
+ ownerEpoch: activation.ownerEpoch,
+ })
+ );
+ });
await navigator.locks.request(
activation.ownerLockName,
{ mode: 'exclusive' },
@@ -277,8 +299,6 @@ async function activate(
break;
}
};
- port.start();
- port.postMessage([0]);
sendEngine(
withVersion<EngineToCoordinatorEnvelope>({
kind: 'engine-ready',
diff --git a/apps/web/src/lib/graphql-cache/worker/browser-test/playwright.production.config.ts b/apps/web/src/lib/graphql-cache/worker/browser-test/playwright.production.config.ts
index d072dc6beb..66dc8f73af 100644
--- a/apps/web/src/lib/graphql-cache/worker/browser-test/playwright.production.config.ts
+++ b/apps/web/src/lib/graphql-cache/worker/browser-test/playwright.production.config.ts
@@ -8,6 +8,8 @@ export default defineConfig({
testDir: directory,
testMatch: [
'coordinator.browser.e2e.ts',
+ 'cache-startup.browser.e2e.ts',
+ 'cache-artifact-compatibility.browser.e2e.ts',
'cache-wasm-packaging.browser.e2e.ts',
'cache-lifecycle.browser.e2e.ts',
],
diff --git a/apps/web/src/lib/graphql-cache/worker/browser-test/production-cache.storage-control-worker.test.ts b/apps/web/src/lib/graphql-cache/worker/browser-test/production-cache.storage-control-worker.test.ts
new file mode 100644
index 0000000000..172f8ad1e1
--- /dev/null
+++ b/apps/web/src/lib/graphql-cache/worker/browser-test/production-cache.storage-control-worker.test.ts
@@ -0,0 +1,92 @@
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+
+const load = vi.hoisted(() => vi.fn());
+vi.mock('./browser-test-wasm-module', () => ({
+ loadBrowserTestCacheWasm: load,
+}));
+const hooks = {
+ browserTestMakeNamespaceIncompatible: vi.fn(async () => {}),
+ browserTestCorruptQueuePayload: vi.fn(async () => {}),
+};
+const buildInfo = {
+ packageVersion: '0.6.15',
+ schemaHash: 'a'.repeat(64),
+ schemaCompatibilityEpoch: 2,
+ formatVersion: 3,
+ storageSchemaVersion: 11,
+};
+let worker: {
+ onmessage: ((event: MessageEvent) => void) | null;
+ postMessage: ReturnType<typeof vi.fn>;
+};
+
+beforeEach(async () => {
+ vi.resetModules();
+ vi.clearAllMocks();
+ worker = { onmessage: null, postMessage: vi.fn() };
+ vi.stubGlobal('self', worker);
+ load.mockResolvedValue({ module: hooks, wasmUrl: 'hooks.wasm', buildInfo });
+ await import('./production-cache.storage-control-worker');
+});
+afterEach(() => vi.unstubAllGlobals());
+
+function send(kind: string) {
+ worker.onmessage?.(
+ new MessageEvent('message', {
+ data: { id: 1, scope: 'fixture-only', kind },
+ })
+ );
+}
+
+describe('storage-control preflight', () => {
+ it('verifies the artifact pair without invoking any storage hook', async () => {
+ send('verify-artifacts');
+ await vi.waitFor(() =>
+ expect(worker.postMessage).toHaveBeenCalledWith({
+ id: 1,
+ ok: true,
+ wasmUrl: 'hooks.wasm',
+ buildInfo,
+ })
+ );
+ expect(hooks.browserTestMakeNamespaceIncompatible).not.toHaveBeenCalled();
+ expect(hooks.browserTestCorruptQueuePayload).not.toHaveBeenCalled();
+ });
+
+ it.each([
+ 'verify-artifacts',
+ 'incompatible-namespace',
+ 'corrupt-queue-payload',
+ ])('stops %s when artifact verification fails', async (kind) => {
+ load.mockRejectedValue(new Error('Cache recovery fixture WASM mismatch'));
+ send(kind);
+ await vi.waitFor(() =>
+ expect(worker.postMessage).toHaveBeenCalledWith({
+ id: 1,
+ ok: false,
+ error: 'Cache recovery fixture WASM mismatch',
+ })
+ );
+ expect(hooks.browserTestMakeNamespaceIncompatible).not.toHaveBeenCalled();
+ expect(hooks.browserTestCorruptQueuePayload).not.toHaveBeenCalled();
+ });
+
+ it.each([
+ ['incompatible-namespace', 'browserTestMakeNamespaceIncompatible'],
+ ['corrupt-queue-payload', 'browserTestCorruptQueuePayload'],
+ ] as const)('runs %s only after verification', async (kind, hook) => {
+ send(kind);
+ await vi.waitFor(() =>
+ expect(worker.postMessage).toHaveBeenCalledWith({
+ id: 1,
+ ok: true,
+ wasmUrl: 'hooks.wasm',
+ buildInfo,
+ })
+ );
+ expect(hooks[hook]).toHaveBeenCalledWith('fixture-only');
+ expect(load.mock.invocationCallOrder[0]).toBeLessThan(
+ hooks[hook].mock.invocationCallOrder[0]
+ );
+ });
+});
diff --git a/apps/web/src/lib/graphql-cache/worker/browser-test/production-cache.storage-control-worker.ts b/apps/web/src/lib/graphql-cache/worker/browser-test/production-cache.storage-control-worker.ts
index dd6fb414a1..9a172c28ed 100644
--- a/apps/web/src/lib/graphql-cache/worker/browser-test/production-cache.storage-control-worker.ts
+++ b/apps/web/src/lib/graphql-cache/worker/browser-test/production-cache.storage-control-worker.ts
@@ -2,15 +2,16 @@
import { match } from 'ts-pattern';
import { loadBrowserTestCacheWasm } from './browser-test-wasm-module';
+import type { CacheWasmBuildInfo } from './wasm-build-compatibility';
type StorageControlRequest = {
id: number;
scope: string;
- kind: 'incompatible-namespace' | 'corrupt-queue-payload';
+ kind: 'verify-artifacts' | 'incompatible-namespace' | 'corrupt-queue-payload';
};
type StorageControlResponse =
- | { id: number; ok: true; wasmUrl: string }
+ | { id: number; ok: true; wasmUrl: string; buildInfo: CacheWasmBuildInfo }
| { id: number; ok: false; error: string };
const worker = self as unknown as DedicatedWorkerGlobalScope;
@@ -18,8 +19,9 @@ const worker = self as unknown as DedicatedWorkerGlobalScope;
worker.onmessage = (event: MessageEvent<StorageControlRequest>) => {
const request = event.data;
void (async () => {
- const { module, wasmUrl } = await loadBrowserTestCacheWasm();
+ const { module, wasmUrl, buildInfo } = await loadBrowserTestCacheWasm();
await match(request)
+ .with({ kind: 'verify-artifacts' }, () => undefined)
.with({ kind: 'incompatible-namespace' }, ({ scope }) =>
module.browserTestMakeNamespaceIncompatible(scope)
)
@@ -31,6 +33,7 @@ worker.onmessage = (event: MessageEvent<StorageControlRequest>) => {
id: request.id,
ok: true,
wasmUrl,
+ buildInfo,
} satisfies StorageControlResponse);
})().catch((error: unknown) => {
worker.postMessage({
diff --git a/apps/web/src/lib/graphql-cache/worker/browser-test/wasm-build-compatibility.test.ts b/apps/web/src/lib/graphql-cache/worker/browser-test/wasm-build-compatibility.test.ts
new file mode 100644
index 0000000000..3e96574f44
--- /dev/null
+++ b/apps/web/src/lib/graphql-cache/worker/browser-test/wasm-build-compatibility.test.ts
@@ -0,0 +1,71 @@
+import { describe, expect, it } from 'vitest';
+import {
+ assertMatchingCacheWasmBuilds,
+ type CacheWasmBuildInfo,
+} from './wasm-build-compatibility';
+
+const current: CacheWasmBuildInfo = {
+ packageVersion: '0.6.15',
+ schemaHash: 'a'.repeat(64),
+ schemaCompatibilityEpoch: 2,
+ formatVersion: 3,
+ storageSchemaVersion: 11,
+};
+const source = (value: unknown) => ({ cacheBuildInfo: () => value });
+
+describe('recovery fixture artifact compatibility', () => {
+ it('accepts a matched pair of embedded build identities', () => {
+ expect(
+ assertMatchingCacheWasmBuilds(source(current), source({ ...current }))
+ ).toEqual(current);
+ });
+
+ it.each([
+ { packageVersion: '0.6.4' },
+ { schemaHash: 'b'.repeat(64) },
+ { schemaCompatibilityEpoch: 1 },
+ { formatVersion: 2 },
+ { storageSchemaVersion: 10 },
+ ])('rejects a differing build field: %j', (difference) => {
+ expect(() =>
+ assertMatchingCacheWasmBuilds(
+ source(current),
+ source({ ...current, ...difference })
+ )
+ ).toThrow('Cache recovery fixture WASM mismatch');
+ });
+
+ it('identifies the incompatible artifacts and the rebuild command', () => {
+ expect(() =>
+ assertMatchingCacheWasmBuilds(
+ source(current),
+ source({ ...current, packageVersion: '0.6.4', formatVersion: 2 })
+ )
+ ).toThrow(
+ /Production: cache-wasm 0\.6\.15, s2:v3.*Browser-test hooks: cache-wasm 0\.6\.4, s2:v2.*just build-cache-wasm-browser-production/
+ );
+ });
+
+ it.each(['production', 'browser-test hooks'])(
+ 'diagnoses a stale %s binary missing the export',
+ (label) => {
+ const sources =
+ label === 'production' ? [{}, source(current)] : [source(current), {}];
+ expect(() =>
+ assertMatchingCacheWasmBuilds(sources[0], sources[1])
+ ).toThrow(`${label} is missing cacheBuildInfo (stale WASM artifact)`);
+ }
+ );
+
+ it.each([
+ null,
+ {},
+ { ...current, formatVersion: '3' },
+ { ...current, storageSchemaVersion: NaN },
+ { ...current, schemaCompatibilityEpoch: -1 },
+ ])('rejects malformed metadata: %j', (metadata) => {
+ expect(() =>
+ assertMatchingCacheWasmBuilds(source(current), source(metadata))
+ ).toThrow('browser-test hooks returned invalid cacheBuildInfo');
+ });
+});
diff --git a/apps/web/src/lib/graphql-cache/worker/browser-test/wasm-build-compatibility.ts b/apps/web/src/lib/graphql-cache/worker/browser-test/wasm-build-compatibility.ts
new file mode 100644
index 0000000000..a3aaadfb01
--- /dev/null
+++ b/apps/web/src/lib/graphql-cache/worker/browser-test/wasm-build-compatibility.ts
@@ -0,0 +1,86 @@
+/** Embedded binary metadata, not the adjacent generated package.json. */
+export type CacheWasmBuildInfo = {
+ packageVersion: string;
+ schemaHash: string;
+ schemaCompatibilityEpoch: number;
+ formatVersion: number;
+ storageSchemaVersion: number;
+};
+
+export type CacheWasmBuildInfoSource = {
+ cacheBuildInfo?: () => unknown;
+};
+
+const REBUILD_INSTRUCTION =
+ 'Rebuild both artifacts from the same checkout: run just build-cache-wasm-browser-production in apps/web.';
+
+function readBuildInfo(
+ source: CacheWasmBuildInfoSource,
+ label: string
+): CacheWasmBuildInfo {
+ if (typeof source.cacheBuildInfo !== 'function') {
+ throw new Error(
+ `Cache recovery fixture: ${label} is missing cacheBuildInfo (stale WASM artifact). ${REBUILD_INSTRUCTION}`
+ );
+ }
+ const info = source.cacheBuildInfo();
+ if (
+ typeof info !== 'object' ||
+ info === null ||
+ !('packageVersion' in info) ||
+ typeof info.packageVersion !== 'string' ||
+ !info.packageVersion ||
+ !('schemaHash' in info) ||
+ typeof info.schemaHash !== 'string' ||
+ !info.schemaHash ||
+ !('schemaCompatibilityEpoch' in info) ||
+ !isVersion(info.schemaCompatibilityEpoch) ||
+ !('formatVersion' in info) ||
+ !isVersion(info.formatVersion) ||
+ !('storageSchemaVersion' in info) ||
+ !isVersion(info.storageSchemaVersion)
+ ) {
+ throw new Error(
+ `Cache recovery fixture: ${label} returned invalid cacheBuildInfo. ${REBUILD_INSTRUCTION}`
+ );
+ }
+ return {
+ packageVersion: info.packageVersion,
+ schemaHash: info.schemaHash,
+ schemaCompatibilityEpoch: info.schemaCompatibilityEpoch,
+ formatVersion: info.formatVersion,
+ storageSchemaVersion: info.storageSchemaVersion,
+ };
+}
+
+function isVersion(value: unknown): value is number {
+ return typeof value === 'number' && Number.isSafeInteger(value) && value >= 0;
+}
+
+function describeBuild(info: CacheWasmBuildInfo): string {
+ return `cache-wasm ${info.packageVersion}, s${info.schemaCompatibilityEpoch}:v${info.formatVersion}, storage schema ${info.storageSchemaVersion}, GraphQL schema ${info.schemaHash}`;
+}
+
+/** Fault injection requires a matched build pair, intentionally stricter than
+ * normal cache compatibility (where additive GraphQL changes are permitted).
+ * This check never opens, validates, or resets a database.
+ */
+export function assertMatchingCacheWasmBuilds(
+ production: CacheWasmBuildInfoSource,
+ hooks: CacheWasmBuildInfoSource
+): CacheWasmBuildInfo {
+ const expected = readBuildInfo(production, 'production');
+ const actual = readBuildInfo(hooks, 'browser-test hooks');
+ if (
+ expected.packageVersion !== actual.packageVersion ||
+ expected.schemaHash !== actual.schemaHash ||
+ expected.schemaCompatibilityEpoch !== actual.schemaCompatibilityEpoch ||
+ expected.formatVersion !== actual.formatVersion ||
+ expected.storageSchemaVersion !== actual.storageSchemaVersion
+ ) {
+ throw new Error(
+ `Cache recovery fixture WASM mismatch. Production: ${describeBuild(expected)}. Browser-test hooks: ${describeBuild(actual)}. ${REBUILD_INSTRUCTION}`
+ );
+ }
+ return expected;
+}
diff --git a/apps/web/src/lib/graphql-cache/worker/cache-coordinator-runtime.test.ts b/apps/web/src/lib/graphql-cache/worker/cache-coordinator-runtime.test.ts
index 3626dd4347..18dc8cf195 100644
--- a/apps/web/src/lib/graphql-cache/worker/cache-coordinator-runtime.test.ts
+++ b/apps/web/src/lib/graphql-cache/worker/cache-coordinator-runtime.test.ts
@@ -44,7 +44,7 @@ class FakeEndpoint extends EventTarget {
}
const message = {
- coordinatorVersion: 2,
+ coordinatorVersion: 3,
kind: 'disconnect-tab',
tabId: 'tab-a',
reason: 'test',
@@ -90,7 +90,7 @@ describe('cache coordinator runtime', () => {
expect(() =>
ports[0]?.postMessage({
- coordinatorVersion: 2,
+ coordinatorVersion: 3,
kind: 'protocol-error',
error: 'test',
} satisfies CoordinatorToTabEnvelope)
diff --git a/apps/web/src/lib/graphql-cache/worker/cache-engine-runtime.test.ts b/apps/web/src/lib/graphql-cache/worker/cache-engine-runtime.test.ts
index 28c543fa52..119f8169d6 100644
--- a/apps/web/src/lib/graphql-cache/worker/cache-engine-runtime.test.ts
+++ b/apps/web/src/lib/graphql-cache/worker/cache-engine-runtime.test.ts
@@ -22,6 +22,7 @@ import type { CacheWorkerCoreOptions } from './worker-core';
class FakePort extends EventTarget {
readonly messages: unknown[] = [];
+ autoOpen = true;
closed = false;
started = false;
onmessage: ((event: MessageEvent<unknown>) => void) | null = null;
@@ -29,6 +30,18 @@ class FakePort extends EventTarget {
postMessage(message: unknown): void {
this.messages.push(message);
+ const payload = effectPayload(message) as
+ | { kind?: string; ownerEpoch?: number }
+ | undefined;
+ if (this.autoOpen && payload?.kind === 'engine-assets-ready') {
+ queueMicrotask(() =>
+ this.receive({
+ ...version,
+ kind: 'open-engine',
+ ownerEpoch: payload.ownerEpoch,
+ })
+ );
+ }
}
close(): void {
@@ -171,6 +184,70 @@ const cacheResponses = (port: FakePort) =>
);
describe('cache engine worker runtime', () => {
+ it('does not touch storage until assets load and the coordinator grants opening', async () => {
+ const scope = new FakeWorkerScope();
+ const direct = new FakePort();
+ direct.autoOpen = false;
+ let finishLoading!: () => void;
+ const prepare = vi.fn(
+ () =>
+ new Promise<void>((resolve) => {
+ finishLoading = resolve;
+ })
+ );
+ const handleRequest = vi.fn(async (port, request: CacheRequest) => {
+ port.postMessage({ id: request.id, ok: true, result: null });
+ });
+ installCacheEngineWorker({
+ scope,
+ ownerLockIsHeld: async () => true,
+ createCore: () => ({
+ prepare,
+ handleRequest,
+ addPort: vi.fn(),
+ drain: vi.fn(),
+ }),
+ });
+ scope.activate(activation(), direct);
+ expect(prepare).toHaveBeenCalledOnce();
+ expect(handleRequest).not.toHaveBeenCalled();
+ expect(messagesOfKind(direct, 'engine-assets-ready')).toHaveLength(0);
+ finishLoading();
+ await vi.waitFor(() =>
+ expect(messagesOfKind(direct, 'engine-assets-ready')).toHaveLength(1)
+ );
+ expect(handleRequest).not.toHaveBeenCalled();
+ direct.receive({ ...version, kind: 'open-engine', ownerEpoch: 7 });
+ await vi.waitFor(() =>
+ expect(messagesOfKind(direct, 'engine-ready')).toHaveLength(1)
+ );
+ expect(handleRequest).toHaveBeenCalledOnce();
+ });
+
+ it('reports failed asset loading without initializing or opening storage', async () => {
+ const scope = new FakeWorkerScope();
+ const direct = new FakePort();
+ const handleRequest = vi.fn();
+ installCacheEngineWorker({
+ scope,
+ createCore: () => ({
+ prepare: async () => {
+ throw new Error('WASM download failed');
+ },
+ handleRequest,
+ addPort: vi.fn(),
+ drain: vi.fn(),
+ }),
+ });
+ scope.activate(activation(), direct);
+ await vi.waitFor(() =>
+ expect(messagesOfKind(direct, 'activation-failed')).toHaveLength(1)
+ );
+ expect(handleRequest).not.toHaveBeenCalled();
+ expect(messagesOfKind(direct, 'engine-assets-ready')).toHaveLength(0);
+ expect(messagesOfKind(direct, 'engine-ready')).toHaveLength(0);
+ });
+
it('selects atomic recovery-open and only proves wipe after initialization', async () => {
const scope = new FakeWorkerScope();
const direct = new FakePort();
diff --git a/apps/web/src/lib/graphql-cache/worker/cache-engine-runtime.ts b/apps/web/src/lib/graphql-cache/worker/cache-engine-runtime.ts
index 1c22c13300..bd97ce04aa 100644
--- a/apps/web/src/lib/graphql-cache/worker/cache-engine-runtime.ts
+++ b/apps/web/src/lib/graphql-cache/worker/cache-engine-runtime.ts
@@ -57,6 +57,7 @@ export interface CacheEngineRuntimeHooks {
}
interface CacheWorkerCoreLike {
+ prepare?(): Promise<void>;
addPort(port: { postMessage(message: unknown): void }): void;
handleRequest(
port: { postMessage(message: unknown): void },
@@ -83,7 +84,7 @@ export interface CacheEngineRuntimeOptions {
memoryTelemetryIntervalMs?: number;
}
-const withVersion = <T extends { coordinatorVersion: 2 }>(
+const withVersion = <T extends { coordinatorVersion: 3 }>(
value: T extends unknown ? Omit<T, 'coordinatorVersion'> : never
): T =>
({
@@ -180,6 +181,12 @@ async function activate(
};
emitEvent({ kind: 'activation-started', activation });
let failed = false;
+ let assetsReady = false;
+ let openGranted = false;
+ let resolveOpen!: () => void;
+ const openPermission = new Promise<void>((resolve) => {
+ resolveOpen = resolve;
+ });
let initializationOpenOutcome: EngineOpenOutcome =
activation.databaseAction === 'wipe-before-open'
? 'reset-storage-uncertain'
@@ -220,6 +227,7 @@ async function activate(
): void => {
if (failed) return;
failed = true;
+ resolveOpen();
emitEvent({ kind: 'fatal', activation, reason, fatalCode });
if (runnerFailed) return;
post(
@@ -313,6 +321,14 @@ async function activate(
return;
}
match(message)
+ .with({ kind: 'open-engine' }, () => {
+ if (!assetsReady || openGranted) {
+ fatal('unexpected database-open grant');
+ return;
+ }
+ openGranted = true;
+ resolveOpen();
+ })
.with({ kind: 'engine-request' }, ({ request }) => {
if (draining) {
fatal('coordinator routed a request after drain began');
@@ -379,6 +395,20 @@ async function activate(
);
try {
+ await core.prepare?.();
+ if (failed) return;
+ assetsReady = true;
+ post(
+ withVersion<EngineToCoordinatorEnvelope>({
+ kind: 'engine-assets-ready',
+ tabId: activation.tabId,
+ ownerEpoch: activation.ownerEpoch,
+ })
+ );
+ // Sending readiness is not permission: the coordinator must first record
+ // that this epoch may touch storage before the first OPFS operation.
+ await openPermission;
+ if (failed) return;
await initializeCore(core, activation);
if (failed) return;
const ownerLockIsHeld = options.ownerLockIsHeld ?? defaultOwnerLockIsHeld;
diff --git a/apps/web/src/lib/graphql-cache/worker/coordinator-core.test.ts b/apps/web/src/lib/graphql-cache/worker/coordinator-core.test.ts
index 3887c24ec5..2f85b40f9a 100644
--- a/apps/web/src/lib/graphql-cache/worker/coordinator-core.test.ts
+++ b/apps/web/src/lib/graphql-cache/worker/coordinator-core.test.ts
@@ -19,8 +19,9 @@ const ready = (
tabId: string,
ownerEpoch: number,
databaseActionProof: DatabaseActionProof
-): CoordinatorAction[] =>
- core.engineReady({
+): CoordinatorAction[] => {
+ core.beginEngineOpen(tabId, ownerEpoch);
+ return core.engineReady({
tabId,
ownerEpoch,
ownerLockName: OWNER_LOCK,
@@ -32,12 +33,48 @@ const ready = (
? 'reset-storage-uncertain'
: 'opened-existing',
});
+};
const init = (id: number) => ({ id, kind: 'init', scope: 'scope' }) as const;
const clear = (id: number) => ({ id, kind: 'clear' }) as const;
describe('CoordinatorCore', () => {
+ it('preserves storage after bootstrap loss but requires recovery once opening was granted', () => {
+ const core = new CoordinatorCore('scope');
+ core.registerTab('tab-a');
+ core.ownerLost('tab-a', 1, 'asset timeout');
+ expect(action(core.resumeAfterLoss(), 'elect-owner').databaseAction).toBe(
+ 'open-existing'
+ );
+ expect(core.beginEngineOpen('tab-a', 1)).toBe(false);
+ expect(core.beginEngineOpen('tab-a', 2)).toBe(true);
+ expect(core.beginEngineOpen('tab-a', 2)).toBe(false);
+ core.ownerLost('tab-a', 2, 'database open timeout');
+ expect(action(core.resumeAfterLoss(), 'elect-owner').databaseAction).toBe(
+ 'wipe-before-open'
+ );
+ // Failing to load assets during recovery must not erase a prior reset requirement.
+ core.ownerLost('tab-a', 3, 'asset download failed');
+ expect(action(core.resumeAfterLoss(), 'elect-owner').databaseAction).toBe(
+ 'wipe-before-open'
+ );
+ });
+
+ it('preserves storage when the only tab closes before the open grant', () => {
+ const core = new CoordinatorCore('scope');
+ core.registerTab('tab-a');
+ core.tabLost('tab-a');
+ core.resumeAfterLoss();
+ expect(core.state).toEqual({
+ kind: 'waiting-for-tab',
+ nextDatabaseAction: 'open-existing',
+ });
+ expect(
+ action(core.registerTab('tab-b'), 'elect-owner').databaseAction
+ ).toBe('open-existing');
+ });
+
it('covers waiting, activating, active, draining, and graceful reactivation', () => {
const core = new CoordinatorCore('scope');
expect(core.state).toEqual({
@@ -221,6 +258,7 @@ describe('CoordinatorCore', () => {
previousTabId: 'tab-a',
previousEpoch: 1,
nextEpoch: 2,
+ databaseAction: 'wipe-before-open',
reason: 'worker failed',
});
expect(action(core.resumeAfterLoss(), 'elect-owner')).toMatchObject({
@@ -377,6 +415,7 @@ describe('CoordinatorCore', () => {
core.registerTab('tab-a');
core.registerTab('tab-b');
+ core.beginEngineOpen('tab-a', 1);
const wrongLock = core.engineReady({
tabId: 'tab-a',
ownerEpoch: 1,
@@ -390,6 +429,7 @@ describe('CoordinatorCore', () => {
'wrong physical owner lock'
);
core.resumeAfterLoss();
+ core.beginEngineOpen('tab-b', 2);
const wrongProof = core.engineReady({
tabId: 'tab-b',
ownerEpoch: 2,
diff --git a/apps/web/src/lib/graphql-cache/worker/coordinator-core.ts b/apps/web/src/lib/graphql-cache/worker/coordinator-core.ts
index 5b807389f7..314392a360 100644
--- a/apps/web/src/lib/graphql-cache/worker/coordinator-core.ts
+++ b/apps/web/src/lib/graphql-cache/worker/coordinator-core.ts
@@ -9,6 +9,7 @@ import type {
DatabaseAction,
DatabaseActionProof,
EngineOpenOutcome,
+ EngineStartupPhase,
OwnerEpoch,
RouteId,
} from './coordinator-protocol';
@@ -20,6 +21,7 @@ export type CoordinatorState =
tabId: string;
ownerEpoch: OwnerEpoch;
databaseAction: DatabaseAction;
+ phase: EngineStartupPhase;
}
| { kind: 'active'; tabId: string; ownerEpoch: OwnerEpoch }
| { kind: 'draining'; tabId: string; ownerEpoch: OwnerEpoch }
@@ -28,9 +30,10 @@ export type CoordinatorState =
previousTabId: string;
previousEpoch: OwnerEpoch;
nextEpoch: OwnerEpoch;
+ databaseAction: DatabaseAction;
reason: string;
}
- | { kind: 'failed'; reason: string };
+ | { kind: 'failed'; reason: string; storageUntouched?: true };
export type CoordinatorSnapshot = {
scope: string;
@@ -91,7 +94,7 @@ export type CoordinatorAction =
reason: 'stale-epoch' | 'unknown-route' | 'inactive-engine';
}
| { kind: 'protocol-violation'; error: string }
- | { kind: 'terminal-failure'; error: string };
+ | { kind: 'terminal-failure'; error: string; storageUntouched?: true };
export type EngineReady = {
tabId: string;
@@ -147,7 +150,15 @@ export class CoordinatorCore {
if (this.tabs.includes(tabId)) return [];
this.tabs.push(tabId);
if (this.stateValue.kind === 'failed') {
- return [{ kind: 'terminal-failure', error: this.stateValue.reason }];
+ return [
+ {
+ kind: 'terminal-failure',
+ error: this.stateValue.reason,
+ ...(this.stateValue.storageUntouched
+ ? { storageUntouched: true as const }
+ : {}),
+ },
+ ];
}
if (this.stateValue.kind !== 'waiting-for-tab') return [];
return this.activateNext(this.stateValue.nextDatabaseAction);
@@ -174,12 +185,29 @@ export class CoordinatorCore {
return [this.route(queued, this.stateValue)];
}
+ /** Grant storage access only after assets have loaded. Before this grant an
+ * abandoned worker cannot have changed OPFS, so its successor preserves it.
+ */
+ beginEngineOpen(tabId: string, ownerEpoch: OwnerEpoch): boolean {
+ const state = this.stateValue;
+ if (
+ state.kind !== 'activating' ||
+ state.tabId !== tabId ||
+ state.ownerEpoch !== ownerEpoch ||
+ state.phase !== 'loading-assets'
+ )
+ return false;
+ this.stateValue = { ...state, phase: 'opening-database' };
+ return true;
+ }
+
engineReady(ready: EngineReady): CoordinatorAction[] {
const state = this.stateValue;
if (
state.kind !== 'activating' ||
state.tabId !== ready.tabId ||
- state.ownerEpoch !== ready.ownerEpoch
+ state.ownerEpoch !== ready.ownerEpoch ||
+ state.phase !== 'opening-database'
) {
return this.recordProtocolViolation(
`unexpected engine-ready from ${ready.tabId} at epoch ${ready.ownerEpoch}`
@@ -314,11 +342,24 @@ export class CoordinatorCore {
/** Stops owner recovery and fails every current or future page connection. */
terminalFailure(reason: string): CoordinatorAction[] {
if (this.stateValue.kind === 'failed') return [];
+ const storageUntouched =
+ this.stateValue.kind === 'resetting-after-loss' &&
+ this.stateValue.databaseAction === 'open-existing';
this.queuedRequests.length = 0;
this.inFlight.clear();
- this.stateValue = { kind: 'failed', reason };
+ this.stateValue = {
+ kind: 'failed',
+ reason,
+ ...(storageUntouched ? { storageUntouched: true as const } : {}),
+ };
this.assertInvariants();
- return [{ kind: 'terminal-failure', error: reason }];
+ return [
+ {
+ kind: 'terminal-failure',
+ error: reason,
+ ...(storageUntouched ? { storageUntouched: true as const } : {}),
+ },
+ ];
}
departForNavigation(
@@ -392,7 +433,7 @@ export class CoordinatorCore {
if (!candidate) {
this.stateValue = {
kind: 'waiting-for-tab',
- nextDatabaseAction: 'wipe-before-open',
+ nextDatabaseAction: state.databaseAction,
};
return [];
}
@@ -401,14 +442,15 @@ export class CoordinatorCore {
kind: 'activating',
tabId: candidate,
ownerEpoch: state.nextEpoch,
- databaseAction: 'wipe-before-open',
+ databaseAction: state.databaseAction,
+ phase: 'loading-assets',
};
return [
{
kind: 'elect-owner',
tabId: candidate,
ownerEpoch: state.nextEpoch,
- databaseAction: 'wipe-before-open',
+ databaseAction: state.databaseAction,
},
];
}
@@ -451,6 +493,10 @@ export class CoordinatorCore {
previousTabId: tabId,
previousEpoch: ownerEpoch,
nextEpoch: this.currentEpoch + 1,
+ databaseAction:
+ state.kind === 'activating' && state.phase === 'loading-assets'
+ ? state.databaseAction
+ : 'wipe-before-open',
reason,
};
this.assertInvariants();
@@ -488,6 +534,7 @@ export class CoordinatorCore {
tabId: candidate,
ownerEpoch: this.currentEpoch,
databaseAction,
+ phase: 'loading-assets',
};
return [
{
diff --git a/apps/web/src/lib/graphql-cache/worker/coordinator-page-adapter.test.ts b/apps/web/src/lib/graphql-cache/worker/coordinator-page-adapter.test.ts
index 547f155a29..b943954d13 100644
--- a/apps/web/src/lib/graphql-cache/worker/coordinator-page-adapter.test.ts
+++ b/apps/web/src/lib/graphql-cache/worker/coordinator-page-adapter.test.ts
@@ -172,6 +172,42 @@ describe('CacheCoordinatorPageAdapter', () => {
expect(attach?.transfer).toHaveLength(1);
});
+ it('forwards startup phases once, ignores stale epochs, and rejects a backwards phase', async () => {
+ const port = new FakeCoordinatorPort();
+ const progress = vi.fn();
+ const error = vi.fn();
+ const adapter = createCacheCoordinatorPageAdapter({
+ scope: 'scope',
+ tabId: 'tab-a',
+ lockManager: heldLockManager(),
+ createSharedWorker: () => ({ port: port as unknown as MessagePort }),
+ createDedicatedWorker: () => new FakeWorker(),
+ onStartupProgress: progress,
+ onTerminalError: error,
+ });
+ const started = adapter.start();
+ await vi.waitFor(() => expect(port.messages).toHaveLength(1));
+ port.receive({ ...version, kind: 'registered', tabId: 'tab-a' });
+ await started;
+ const loading = {
+ ...version,
+ kind: 'engine-startup',
+ ownerEpoch: 2,
+ phase: 'loading-assets',
+ databaseAction: 'open-existing',
+ timeoutMs: 300_000,
+ };
+ port.receive(loading);
+ port.receive(loading);
+ port.receive({ ...loading, ownerEpoch: 1 });
+ expect(progress).toHaveBeenCalledTimes(1);
+ port.receive({ ...loading, phase: 'opening-database', timeoutMs: 20_000 });
+ expect(progress).toHaveBeenCalledTimes(2);
+ port.receive(loading);
+ expect(error).toHaveBeenCalledOnce();
+ expect(port.closed).toBe(true);
+ });
+
it('terminates and clears a failed worker before reporting owner loss', async () => {
const order: string[] = [];
const coordinatorPort = new FakeCoordinatorPort();
diff --git a/apps/web/src/lib/graphql-cache/worker/coordinator-page-adapter.ts b/apps/web/src/lib/graphql-cache/worker/coordinator-page-adapter.ts
index 0b755a1011..b2cef2215b 100644
--- a/apps/web/src/lib/graphql-cache/worker/coordinator-page-adapter.ts
+++ b/apps/web/src/lib/graphql-cache/worker/coordinator-page-adapter.ts
@@ -18,6 +18,7 @@ import {
createEffectWorkerTransport,
type EffectWorkerTransport,
} from './effect-worker-transport';
+import { CacheBootstrapExhaustedError } from './startup';
export interface SharedWorkerLike {
readonly port: MessagePort;
@@ -42,6 +43,9 @@ export interface CacheCoordinatorPageAdapterOptions {
) => DedicatedWorkerLike;
lockManager?: Pick<LockManager, 'request'>;
onEngineReplaced?: (ownerEpoch: number) => void;
+ onStartupProgress?: (
+ progress: Extract<CoordinatorToTabEnvelope, { kind: 'engine-startup' }>
+ ) => void;
onOwnerChanged?: (ownerEpoch: number | undefined) => void;
onWorkerCreated?: (worker: DedicatedWorkerLike, ownerEpoch: number) => void;
onWorkerTerminated?: (ownerEpoch: number, reason: string) => void;
@@ -67,7 +71,7 @@ interface CoordinatorConnection {
const DEFAULT_GRACEFUL_TIMEOUT_MS = 10_000;
-const withVersion = <T extends { coordinatorVersion: 2 }>(
+const withVersion = <T extends { coordinatorVersion: 3 }>(
value: T extends unknown ? Omit<T, 'coordinatorVersion'> : never
): T =>
({
@@ -124,6 +128,9 @@ export class CacheCoordinatorPageAdapter {
private readonly terminatedOwnerEpochs = new Set<number>();
private highestOwnerEpochSeen = 0;
private latestEngineReplacedEpoch = 0;
+ private startupProgress:
+ | Extract<CoordinatorToTabEnvelope, { kind: 'engine-startup' }>
+ | undefined;
private closed = false;
private readonly telemetry: CacheTelemetryRecorderLike;
private readonly now = (): number =>
@@ -535,6 +542,30 @@ export class CacheCoordinatorPageAdapter {
}
this.finishDispose();
break;
+ case 'engine-startup': {
+ if (!this.registered) {
+ this.failTerminal(new Error('engine startup preceded registration'));
+ return;
+ }
+ const previous = this.startupProgress;
+ if (
+ previous &&
+ (message.ownerEpoch < previous.ownerEpoch ||
+ (message.ownerEpoch === previous.ownerEpoch &&
+ message.phase === previous.phase))
+ )
+ return;
+ if (
+ previous?.ownerEpoch === message.ownerEpoch &&
+ previous.phase === 'opening-database'
+ ) {
+ this.failTerminal(new Error('engine startup phase moved backwards'));
+ return;
+ }
+ this.startupProgress = message;
+ this.options.onStartupProgress?.(message);
+ break;
+ }
case 'engine-replaced':
if (message.ownerEpoch <= this.latestEngineReplacedEpoch) return;
this.latestEngineReplacedEpoch = message.ownerEpoch;
@@ -551,7 +582,11 @@ export class CacheCoordinatorPageAdapter {
break;
}
case 'terminal-error':
- this.failTerminal(new Error(message.error));
+ this.failTerminal(
+ message.storageUntouched
+ ? new CacheBootstrapExhaustedError(message.error)
+ : new Error(message.error)
+ );
break;
}
}
diff --git a/apps/web/src/lib/graphql-cache/worker/coordinator-protocol.test.ts b/apps/web/src/lib/graphql-cache/worker/coordinator-protocol.test.ts
index 8dc150cad1..a427dbaf91 100644
--- a/apps/web/src/lib/graphql-cache/worker/coordinator-protocol.test.ts
+++ b/apps/web/src/lib/graphql-cache/worker/coordinator-protocol.test.ts
@@ -24,6 +24,51 @@ const enginePort = {
} as unknown as MessagePort;
describe('coordinator runtime protocol', () => {
+ it('validates the staged startup handshake and rejects the old ungated protocol', () => {
+ const progress = {
+ ...version,
+ kind: 'engine-startup',
+ ownerEpoch: 1,
+ phase: 'loading-assets',
+ databaseAction: 'open-existing',
+ timeoutMs: 300_000,
+ };
+ expect(validateCoordinatorToTabEnvelope(progress).ok).toBe(true);
+ expect(
+ validateCoordinatorToTabEnvelope({
+ ...progress,
+ phase: 'opening-database',
+ }).ok
+ ).toBe(true);
+ for (const invalid of [
+ { phase: 'unknown' },
+ { timeoutMs: 0 },
+ { timeoutMs: Infinity },
+ { databaseAction: 'forget' },
+ { coordinatorVersion: 2 },
+ ]) {
+ expect(
+ validateCoordinatorToTabEnvelope({ ...progress, ...invalid }).ok
+ ).toBe(false);
+ }
+ const prepared = {
+ ...version,
+ kind: 'engine-assets-ready',
+ tabId: 'tab',
+ ownerEpoch: 1,
+ };
+ expect(validateEngineToCoordinatorEnvelope(prepared).ok).toBe(true);
+ expect(
+ validateEngineToCoordinatorEnvelope({ ...prepared, extra: true }).ok
+ ).toBe(false);
+ const grant = { ...version, kind: 'open-engine', ownerEpoch: 1 };
+ expect(validateCoordinatorToEngineEnvelope(grant).ok).toBe(true);
+ expect(
+ validateCoordinatorToEngineEnvelope({ ...grant, coordinatorVersion: 2 })
+ .ok
+ ).toBe(false);
+ });
+
it('validates bounded reconciliation evidence without changing exact requests', () => {
const request = {
filters: {},
diff --git a/apps/web/src/lib/graphql-cache/worker/coordinator-protocol.ts b/apps/web/src/lib/graphql-cache/worker/coordinator-protocol.ts
index e52fd00501..9041b3332d 100644
--- a/apps/web/src/lib/graphql-cache/worker/coordinator-protocol.ts
+++ b/apps/web/src/lib/graphql-cache/worker/coordinator-protocol.ts
@@ -20,7 +20,8 @@ import {
export { isCachePush, isCacheResponse, isWorkerMessage };
/** Version of the topology envelope and routed cache RPC surface. */
-export const CACHE_COORDINATOR_PROTOCOL_VERSION = 2 as const;
+export const CACHE_COORDINATOR_PROTOCOL_VERSION = 3 as const;
+export type EngineStartupPhase = 'loading-assets' | 'opening-database';
export type OwnerEpoch = number;
export type RouteId = number;
@@ -39,7 +40,7 @@ export type ActivationFailureCode =
export type TabToCoordinatorEnvelope =
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'register-tab';
scope: string;
tabId: string;
@@ -47,40 +48,40 @@ export type TabToCoordinatorEnvelope =
hotCapacity?: number;
}
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'cache-request';
tabId: string;
request: CacheRequest;
}
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'attach-engine-port';
tabId: string;
ownerEpoch: OwnerEpoch;
enginePort: MessagePort;
}
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'graceful-departure';
tabId: string;
ownerEpoch: OwnerEpoch;
}
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'navigation-departure';
tabId: string;
ownerEpoch: OwnerEpoch;
reason: string;
}
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'engine-lost';
tabId: string;
ownerEpoch: OwnerEpoch;
reason: string;
}
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'disconnect-tab';
tabId: string;
reason: string;
@@ -88,12 +89,12 @@ export type TabToCoordinatorEnvelope =
export type CoordinatorToTabEnvelope =
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'registered';
tabId: string;
}
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'become-owner';
scope: string;
tabId: string;
@@ -103,41 +104,50 @@ export type CoordinatorToTabEnvelope =
hotCapacity?: number;
}
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'cache-message';
message: WorkerMessage;
}
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'terminate-engine';
tabId: string;
ownerEpoch: OwnerEpoch;
reason: string;
}
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'retire-complete';
tabId: string;
ownerEpoch: OwnerEpoch;
}
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
+ kind: 'engine-startup';
+ ownerEpoch: OwnerEpoch;
+ phase: EngineStartupPhase;
+ databaseAction: DatabaseAction;
+ timeoutMs: number;
+ }
+ | {
+ coordinatorVersion: 3;
kind: 'engine-replaced';
ownerEpoch: OwnerEpoch;
}
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'protocol-error';
error: string;
}
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'terminal-error';
error: string;
+ storageUntouched?: true;
};
export type PageToEngineEnvelope = {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'activate-engine';
scope: string;
tabId: string;
@@ -149,19 +159,24 @@ export type PageToEngineEnvelope = {
export type CoordinatorToEngineEnvelope =
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
+ kind: 'open-engine';
+ ownerEpoch: OwnerEpoch;
+ }
+ | {
+ coordinatorVersion: 3;
kind: 'engine-request';
ownerEpoch: OwnerEpoch;
routeId: RouteId;
request: CacheRequest;
}
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'drain-engine';
ownerEpoch: OwnerEpoch;
}
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'heartbeat';
ownerEpoch: OwnerEpoch;
heartbeatId: number;
@@ -169,7 +184,13 @@ export type CoordinatorToEngineEnvelope =
export type EngineToCoordinatorEnvelope =
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
+ kind: 'engine-assets-ready';
+ tabId: string;
+ ownerEpoch: OwnerEpoch;
+ }
+ | {
+ coordinatorVersion: 3;
kind: 'engine-ready';
tabId: string;
ownerEpoch: OwnerEpoch;
@@ -179,26 +200,26 @@ export type EngineToCoordinatorEnvelope =
openOutcome: EngineOpenOutcome;
}
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'engine-response';
ownerEpoch: OwnerEpoch;
routeId: RouteId;
response: CacheResponse;
}
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'engine-push';
ownerEpoch: OwnerEpoch;
push: CachePush;
}
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'engine-drained';
tabId: string;
ownerEpoch: OwnerEpoch;
}
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'engine-fatal';
tabId: string;
ownerEpoch: OwnerEpoch;
@@ -206,7 +227,7 @@ export type EngineToCoordinatorEnvelope =
fatalCode: EngineFatalCode;
}
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'activation-failed';
tabId: string;
ownerEpoch: OwnerEpoch;
@@ -214,7 +235,7 @@ export type EngineToCoordinatorEnvelope =
failureCode: ActivationFailureCode;
}
| {
- coordinatorVersion: 2;
+ coordinatorVersion: 3;
kind: 'heartbeat-ack';
ownerEpoch: OwnerEpoch;
heartbeatId: number;
@@ -818,6 +839,25 @@ export function validateCoordinatorToTabEnvelope(
return pass(value as CoordinatorToTabEnvelope);
}
break;
+ case 'engine-startup':
+ if (
+ hasOnlyKeys(value, [
+ 'coordinatorVersion',
+ 'kind',
+ 'ownerEpoch',
+ 'phase',
+ 'databaseAction',
+ 'timeoutMs',
+ ]) &&
+ isPositiveInteger(value.ownerEpoch) &&
+ (value.phase === 'loading-assets' ||
+ value.phase === 'opening-database') &&
+ isDatabaseAction(value.databaseAction) &&
+ isPositiveInteger(value.timeoutMs)
+ ) {
+ return pass(value as CoordinatorToTabEnvelope);
+ }
+ break;
case 'engine-replaced':
if (
hasOnlyKeys(value, ['coordinatorVersion', 'kind', 'ownerEpoch']) &&
@@ -826,8 +866,21 @@ export function validateCoordinatorToTabEnvelope(
return pass(value as CoordinatorToTabEnvelope);
}
break;
- case 'protocol-error':
case 'terminal-error':
+ if (
+ hasOnlyKeys(value, [
+ 'coordinatorVersion',
+ 'kind',
+ 'error',
+ 'storageUntouched',
+ ]) &&
+ isNonEmptyString(value.error) &&
+ (value.storageUntouched === undefined ||
+ value.storageUntouched === true)
+ )
+ return pass(value as CoordinatorToTabEnvelope);
+ break;
+ case 'protocol-error':
if (
hasOnlyKeys(value, ['coordinatorVersion', 'kind', 'error']) &&
isNonEmptyString(value.error)
@@ -894,6 +947,7 @@ export function validateCoordinatorToEngineEnvelope(
return pass(value as CoordinatorToEngineEnvelope);
}
break;
+ case 'open-engine':
case 'drain-engine':
if (
hasOnlyKeys(value, ['coordinatorVersion', 'kind', 'ownerEpoch']) &&
@@ -982,6 +1036,7 @@ export function validateEngineToCoordinatorEnvelope(
return pass(value as EngineToCoordinatorEnvelope);
}
break;
+ case 'engine-assets-ready':
case 'engine-drained':
if (
hasOnlyKeys(value, [
diff --git a/apps/web/src/lib/graphql-cache/worker/coordinator-router.test.ts b/apps/web/src/lib/graphql-cache/worker/coordinator-router.test.ts
index d8d83efb8e..3beb7d4155 100644
--- a/apps/web/src/lib/graphql-cache/worker/coordinator-router.test.ts
+++ b/apps/web/src/lib/graphql-cache/worker/coordinator-router.test.ts
@@ -94,6 +94,14 @@ const ready = (
ownerEpoch: number,
proof: 'opened-existing' | 'wiped-before-open'
): void => {
+ if (messagesOfKind(enginePort, 'open-engine').length === 0) {
+ enginePort.receive({
+ ...version,
+ kind: 'engine-assets-ready',
+ tabId,
+ ownerEpoch,
+ });
+ }
enginePort.receive({
...version,
kind: 'engine-ready',
@@ -220,6 +228,12 @@ describe('CoordinatorRouter', () => {
const initialEngine = new FakePort();
await attach(router, tabA, 'tab-a', 1, initialEngine);
+ initialEngine.receive({
+ ...version,
+ kind: 'engine-assets-ready',
+ tabId: 'tab-a',
+ ownerEpoch: 1,
+ });
initialEngine.receive({
...version,
kind: 'activation-failed',
@@ -245,6 +259,12 @@ describe('CoordinatorRouter', () => {
const tab = tabId === 'tab-a' ? tabA : tabB;
const engine = new FakePort();
await attach(router, tab, tabId, state.ownerEpoch, engine);
+ engine.receive({
+ ...version,
+ kind: 'engine-assets-ready',
+ tabId,
+ ownerEpoch: state.ownerEpoch,
+ });
engine.receive({
...version,
kind: 'activation-failed',
@@ -671,6 +691,114 @@ describe('CoordinatorRouter', () => {
);
});
+ it('allows slow asset loading and reports phase budgets to owners and late joiners', async () => {
+ vi.useFakeTimers();
+ const router = new CoordinatorRouter({
+ assetLoadTimeoutMs: 100,
+ activationTimeoutMs: 10,
+ verifyTabLockHeld: async () => true,
+ watchTabLock: () => () => {},
+ });
+ const tabA = new FakePort();
+ const tabB = new FakePort();
+ const engine = new FakePort();
+ await register(router, tabA, 'tab-a');
+ await attach(router, tabA, 'tab-a', 1, engine);
+ await vi.advanceTimersByTimeAsync(50);
+ expect(messagesOfKind(tabA, 'terminate-engine')).toHaveLength(0);
+ expect(messagesOfKind(engine, 'open-engine')).toHaveLength(0);
+ await register(router, tabB, 'tab-b');
+ for (const tab of [tabA, tabB]) {
+ expect(messagesOfKind(tab, 'engine-startup').at(-1)).toMatchObject({
+ ownerEpoch: 1,
+ phase: 'loading-assets',
+ timeoutMs: 100,
+ });
+ }
+ engine.receive({
+ ...version,
+ kind: 'engine-assets-ready',
+ tabId: 'tab-a',
+ ownerEpoch: 1,
+ });
+ expect(messagesOfKind(engine, 'open-engine')).toHaveLength(1);
+ expect(messagesOfKind(tabB, 'engine-startup').at(-1)).toMatchObject({
+ phase: 'opening-database',
+ timeoutMs: 10,
+ });
+ await vi.advanceTimersByTimeAsync(9);
+ ready(engine, 'tab-a', 1, 'opened-existing');
+ await vi.advanceTimersByTimeAsync(50);
+ expect(router.snapshot()?.state.kind).toBe('active');
+ expect(messagesOfKind(tabA, 'terminate-engine')).toHaveLength(0);
+ });
+
+ it('retries failed bootstrap without storage resets and keeps the queued init', async () => {
+ vi.useFakeTimers();
+ const observations: Array<{ name: string }> = [];
+ const router = new CoordinatorRouter({
+ assetLoadTimeoutMs: 10,
+ verifyTabLockHeld: async () => true,
+ watchTabLock: () => () => {},
+ telemetry: {
+ record: (event) => observations.push(event),
+ flush: vi.fn(),
+ },
+ });
+ const tab = new FakePort();
+ await register(router, tab, 'tab-a');
+ await router.handleTabMessage(tab as CoordinatorMessagePort, {
+ ...version,
+ kind: 'cache-request',
+ tabId: 'tab-a',
+ request: { kind: 'init', id: 9, scope: 'scope' },
+ });
+ await vi.advanceTimersByTimeAsync(11);
+ expect(router.snapshot()?.state).toMatchObject({
+ kind: 'activating',
+ ownerEpoch: 2,
+ databaseAction: 'open-existing',
+ });
+ expect(router.snapshot()?.queuedRequestCount).toBe(1);
+ expect(messagesOfKind(tab, 'cache-message')).toHaveLength(0);
+ expect(
+ observations.some(
+ (event) => event.name === 'graphql_cache.storage_reset_required'
+ )
+ ).toBe(false);
+ const engine = new FakePort();
+ await attach(router, tab, 'tab-a', 2, engine);
+ ready(engine, 'tab-a', 2, 'opened-existing');
+ expect(messagesOfKind(engine, 'engine-request')).toHaveLength(1);
+ });
+
+ it('bounds repeated asset failures without ever requesting a wipe', async () => {
+ vi.useFakeTimers();
+ const router = new CoordinatorRouter({
+ assetLoadTimeoutMs: 10,
+ verifyTabLockHeld: async () => true,
+ watchTabLock: () => () => {},
+ });
+ const tab = new FakePort();
+ await register(router, tab, 'tab-a');
+ await vi.advanceTimersByTimeAsync(2000);
+ expect(router.snapshot()?.state.kind).toBe('failed');
+ expect(messagesOfKind(tab, 'become-owner')).toHaveLength(6);
+ expect(
+ messagesOfKind(tab, 'become-owner').every(
+ (message) => message.databaseAction === 'open-existing'
+ )
+ ).toBe(true);
+ expect(messagesOfKind(tab, 'terminal-error')).toEqual([
+ expect.objectContaining({ storageUntouched: true }),
+ ]);
+ const late = new FakePort();
+ await register(router, late, 'tab-b');
+ expect(messagesOfKind(late, 'terminal-error')).toEqual([
+ expect.objectContaining({ storageUntouched: true }),
+ ]);
+ });
+
it('uses activation and heartbeat watchdogs to terminate and wipe', async () => {
vi.useFakeTimers();
const router = new CoordinatorRouter({
@@ -685,11 +813,19 @@ describe('CoordinatorRouter', () => {
await register(router, tabA, 'tab-a');
await register(router, tabB, 'tab-b');
+ const openingEngine = new FakePort();
+ await attach(router, tabA, 'tab-a', 1, openingEngine);
+ openingEngine.receive({
+ ...version,
+ kind: 'engine-assets-ready',
+ tabId: 'tab-a',
+ ownerEpoch: 1,
+ });
await vi.advanceTimersByTimeAsync(11);
expect(messagesOfKind(tabA, 'terminate-engine')).toContainEqual(
expect.objectContaining({
ownerEpoch: 1,
- reason: 'engine activation watchdog timed out',
+ reason: 'engine database opening watchdog timed out',
})
);
expect(messagesOfKind(tabB, 'become-owner')).toContainEqual(
@@ -1046,6 +1182,12 @@ describe('CoordinatorRouter', () => {
const engine = new FakePort();
await attach(router, tabA, 'tab-a', 1, engine);
+ engine.receive({
+ ...version,
+ kind: 'engine-assets-ready',
+ tabId: 'tab-a',
+ ownerEpoch: 1,
+ });
engine.receive({ kind: 'engine-ready', ownerEpoch: 1 });
await Promise.resolve();
diff --git a/apps/web/src/lib/graphql-cache/worker/coordinator-router.ts b/apps/web/src/lib/graphql-cache/worker/coordinator-router.ts
index 98d1b905d8..251d4095a4 100644
--- a/apps/web/src/lib/graphql-cache/worker/coordinator-router.ts
+++ b/apps/web/src/lib/graphql-cache/worker/coordinator-router.ts
@@ -35,6 +35,10 @@ import {
createEffectWorkerTransport,
type EffectWorkerTransport,
} from './effect-worker-transport';
+import {
+ ENGINE_ASSET_LOAD_TIMEOUT_MS,
+ ENGINE_DATABASE_OPEN_TIMEOUT_MS,
+} from './startup';
export interface CoordinatorMessagePort {
onmessage: ((event: MessageEvent<unknown>) => void) | null;
@@ -47,6 +51,7 @@ export interface CoordinatorMessagePort {
export type CancelLivenessWatch = () => void;
export interface CoordinatorRouterOptions {
+ assetLoadTimeoutMs?: number;
activationTimeoutMs?: number;
heartbeatIntervalMs?: number;
heartbeatTimeoutMs?: number;
@@ -74,7 +79,6 @@ type EngineRoute = {
transport: EffectWorkerTransport<CoordinatorToEngineEnvelope>;
};
-const DEFAULT_ACTIVATION_TIMEOUT_MS = 20_000;
const DEFAULT_HEARTBEAT_INTERVAL_MS = 2_000;
const DEFAULT_HEARTBEAT_TIMEOUT_MS = 5_000;
const RECOVERY_RETRY_LIMIT = 5;
@@ -117,7 +121,7 @@ type WithoutVersion<T> = T extends unknown
? Omit<T, 'coordinatorVersion'>
: never;
-const envelope = <T extends { coordinatorVersion: 2 }>(
+const envelope = <T extends { coordinatorVersion: 3 }>(
value: WithoutVersion<T>
): T =>
({
@@ -180,6 +184,7 @@ export class CoordinatorRouter {
| { ownerEpoch: number; heartbeatId: number }
| undefined;
+ private readonly assetLoadTimeoutMs: number;
private readonly activationTimeoutMs: number;
private readonly heartbeatIntervalMs: number;
private readonly heartbeatTimeoutMs: number;
@@ -213,8 +218,10 @@ export class CoordinatorRouter {
globalThis.performance?.now() ?? Date.now();
constructor(options: CoordinatorRouterOptions = {}) {
+ this.assetLoadTimeoutMs =
+ options.assetLoadTimeoutMs ?? ENGINE_ASSET_LOAD_TIMEOUT_MS;
this.activationTimeoutMs =
- options.activationTimeoutMs ?? DEFAULT_ACTIVATION_TIMEOUT_MS;
+ options.activationTimeoutMs ?? ENGINE_DATABASE_OPEN_TIMEOUT_MS;
this.heartbeatIntervalMs =
options.heartbeatIntervalMs ?? DEFAULT_HEARTBEAT_INTERVAL_MS;
this.heartbeatTimeoutMs =
@@ -436,7 +443,13 @@ export class CoordinatorRouter {
tabId: message.tabId,
})
);
- this.applyActions(this.coreValue.registerTab(message.tabId));
+ const actions = this.coreValue.registerTab(message.tabId);
+ this.applyActions(actions);
+ // Late joiners must use the same startup budget as the elected owner.
+ if (!actions.some((action) => action.kind === 'elect-owner')) {
+ const progress = this.startupProgress();
+ if (progress) this.postToTab(message.tabId, progress);
+ }
}
private attachEnginePort(
@@ -526,6 +539,25 @@ export class CoordinatorRouter {
}
switch (message.kind) {
+ case 'engine-assets-ready': {
+ if (!core.beginEngineOpen(message.tabId, message.ownerEpoch)) {
+ this.failOwner(
+ message.tabId,
+ message.ownerEpoch,
+ 'unexpected engine asset readiness'
+ );
+ break;
+ }
+ this.armStartupWatchdog(message.tabId, message.ownerEpoch);
+ this.sendToEngine(
+ route,
+ envelope<CoordinatorToEngineEnvelope>({
+ kind: 'open-engine',
+ ownerEpoch: message.ownerEpoch,
+ })
+ );
+ break;
+ }
case 'engine-ready': {
const actions = core.engineReady({
...message,
@@ -683,6 +715,7 @@ export class CoordinatorRouter {
);
this.nextRecoveryResetReason = undefined;
}
+ this.armStartupWatchdog(action.tabId, action.ownerEpoch);
this.postToTab(
action.tabId,
envelope<CoordinatorToTabEnvelope>({
@@ -695,13 +728,6 @@ export class CoordinatorRouter {
hotCapacity: this.hotCapacity,
})
);
- this.activationTimer = this.setTimeoutFn(() => {
- this.failOwner(
- action.tabId,
- action.ownerEpoch,
- 'engine activation watchdog timed out'
- );
- }, this.activationTimeoutMs);
break;
case 'route-request': {
this.routeStarted.set(action.routeId, {
@@ -861,6 +887,7 @@ export class CoordinatorRouter {
envelope<CoordinatorToTabEnvelope>({
kind: 'terminal-error',
error: action.error,
+ ...(action.storageUntouched ? { storageUntouched: true } : {}),
})
);
break;
@@ -868,6 +895,41 @@ export class CoordinatorRouter {
}
}
+ private startupProgress():
+ | Extract<CoordinatorToTabEnvelope, { kind: 'engine-startup' }>
+ | undefined {
+ const state = this.coreValue?.state;
+ if (state?.kind !== 'activating') return;
+ return envelope<
+ Extract<CoordinatorToTabEnvelope, { kind: 'engine-startup' }>
+ >({
+ kind: 'engine-startup',
+ ownerEpoch: state.ownerEpoch,
+ phase: state.phase,
+ databaseAction: state.databaseAction,
+ timeoutMs:
+ state.phase === 'loading-assets'
+ ? this.assetLoadTimeoutMs
+ : this.activationTimeoutMs,
+ });
+ }
+
+ private armStartupWatchdog(tabId: string, ownerEpoch: number): void {
+ this.clearActivationTimer();
+ const progress = this.startupProgress();
+ if (!progress) return;
+ this.broadcast(progress);
+ this.activationTimer = this.setTimeoutFn(() => {
+ this.failOwner(
+ tabId,
+ ownerEpoch,
+ progress.phase === 'loading-assets'
+ ? 'engine asset loading watchdog timed out'
+ : 'engine database opening watchdog timed out'
+ );
+ }, progress.timeoutMs);
+ }
+
private scheduleResetActivation(): void {
const core = this.coreValue;
if (!core || core.state.kind !== 'resetting-after-loss') return;
@@ -916,19 +978,24 @@ export class CoordinatorRouter {
const actions = core.ownerLost(tabId, ownerEpoch, reason);
if (actions.length === 0) return;
this.activationStarted.delete(ownerEpoch);
- const pendingReason = this.recordPendingResetFailure(
- ownerEpoch,
- reason,
- failureCode
- );
- const resetReason =
- pendingReason ??
- (fatalCode === 'storage-reset-required'
- ? 'storage-reset-required'
- : 'abrupt-owner-loss');
- this.nextRecoveryResetReason = resetReason;
- if (pendingReason === undefined) {
- this.recordStorageResetRequired(ownerEpoch, resetReason);
+ const next = core.state;
+ if (
+ next.kind === 'resetting-after-loss' &&
+ next.databaseAction === 'wipe-before-open'
+ ) {
+ const pendingReason = this.recordPendingResetFailure(
+ ownerEpoch,
+ reason,
+ failureCode
+ );
+ const resetReason =
+ pendingReason ??
+ (fatalCode === 'storage-reset-required'
+ ? 'storage-reset-required'
+ : 'abrupt-owner-loss');
+ this.nextRecoveryResetReason = resetReason;
+ if (pendingReason === undefined)
+ this.recordStorageResetRequired(ownerEpoch, resetReason);
}
this.telemetry.record({
name: 'graphql_cache.owner',
@@ -1017,14 +1084,19 @@ export class CoordinatorRouter {
state.tabId === tabId
) {
this.activationStarted.delete(state.ownerEpoch);
- const pendingReason = this.recordPendingResetFailure(
- state.ownerEpoch,
- reason
- );
- const resetReason = pendingReason ?? 'abrupt-owner-loss';
- this.nextRecoveryResetReason = resetReason;
- if (pendingReason === undefined) {
- this.recordStorageResetRequired(state.ownerEpoch, resetReason);
+ if (
+ state.kind !== 'activating' ||
+ state.phase !== 'loading-assets' ||
+ state.databaseAction === 'wipe-before-open'
+ ) {
+ const pendingReason = this.recordPendingResetFailure(
+ state.ownerEpoch,
+ reason
+ );
+ const resetReason = pendingReason ?? 'abrupt-owner-loss';
+ this.nextRecoveryResetReason = resetReason;
+ if (pendingReason === undefined)
+ this.recordStorageResetRequired(state.ownerEpoch, resetReason);
}
this.telemetry.record({
name: 'graphql_cache.owner',
diff --git a/apps/web/src/lib/graphql-cache/worker/startup.ts b/apps/web/src/lib/graphql-cache/worker/startup.ts
new file mode 100644
index 0000000000..59f842a172
--- /dev/null
+++ b/apps/web/src/lib/graphql-cache/worker/startup.ts
@@ -0,0 +1,14 @@
+/** Startup budgets are independent of the 10s cache-read deadline. Loading the
+ * worker module graph and the WASM binary must not look like a wedged database.
+ */
+export const COORDINATOR_CONNECT_TIMEOUT_MS = 60_000;
+export const ENGINE_ASSET_LOAD_TIMEOUT_MS = 5 * 60_000;
+export const ENGINE_DATABASE_OPEN_TIMEOUT_MS = 20_000;
+/** Lets the coordinator's phase timeout/retry reach a page before its guard fires. */
+export const STARTUP_RESPONSE_GRACE_MS = 5_000;
+export const COORDINATOR_CONNECT_ATTEMPTS = 3;
+
+/** Only the live coordinator can prove that every failed attempt stayed before
+ * the open grant. A missing progress message is not proof of untouched storage.
+ */
+export class CacheBootstrapExhaustedError extends Error {}
diff --git a/apps/web/src/lib/graphql-cache/worker/wasm-module.ts b/apps/web/src/lib/graphql-cache/worker/wasm-module.ts
index 7f52b0886d..01357e7286 100644
--- a/apps/web/src/lib/graphql-cache/worker/wasm-module.ts
+++ b/apps/web/src/lib/graphql-cache/worker/wasm-module.ts
@@ -192,6 +192,8 @@ export interface CacheWasmModule {
): Promise<CacheOpenResult>;
destroyCache(scope: string): Promise<void>;
schemaHash(): string;
+ /** Read-only binary metadata; optional so fixtures can diagnose stale artifacts. */
+ cacheBuildInfo?(): unknown;
}
let modulePromise: Promise<CacheWasmModule> | undefined;
diff --git a/apps/web/src/lib/graphql-cache/worker/worker-core.ts b/apps/web/src/lib/graphql-cache/worker/worker-core.ts
index 9d4c7553da..491db518ca 100644
--- a/apps/web/src/lib/graphql-cache/worker/worker-core.ts
+++ b/apps/web/src/lib/graphql-cache/worker/worker-core.ts
@@ -825,6 +825,11 @@ export class CacheWorkerCore {
}
}
+ /** Download/compile/instantiate without acquiring the database lock or opening OPFS. */
+ async prepare(): Promise<void> {
+ await loadCacheWasm();
+ }
+
private async init(scope: string, hotCapacity?: number): Promise<void> {
if (this.initPromise) {
// Subsequent page clients routed to this elected engine re-init idempotently.
diff --git a/apps/web/src/lib/queries/soup/graphql/active-queries.test.ts b/apps/web/src/lib/queries/soup/graphql/active-queries.test.ts
index b80d921236..fd847ca492 100644
--- a/apps/web/src/lib/queries/soup/graphql/active-queries.test.ts
+++ b/apps/web/src/lib/queries/soup/graphql/active-queries.test.ts
@@ -1,10 +1,88 @@
-import { describe, expect, it, vi } from 'vitest';
+import { describe, expect, it, onTestFinished, vi } from 'vitest';
import {
refreshActiveGraphqlSoupQueries,
registerActiveGraphqlSoupQuery,
} from './active-queries';
+function register(query: Parameters<typeof registerActiveGraphqlSoupQuery>[0]) {
+ const unregister = registerActiveGraphqlSoupQuery(query);
+ onTestFinished(unregister);
+ return unregister;
+}
+
describe('active GraphQL Soup queries', () => {
+ it('strict revalidation fails until every enabled reader succeeds', async () => {
+ const consoleError = vi
+ .spyOn(console, 'error')
+ .mockImplementation(() => {});
+ onTestFinished(() => consoleError.mockRestore());
+ const failure = vi
+ .fn()
+ .mockRejectedValueOnce(new Error('offline'))
+ .mockResolvedValue(undefined);
+ const success = vi.fn(async () => undefined);
+ register({ isEnabled: () => true, refresh: failure });
+ register({ isEnabled: () => true, refresh: success });
+ await expect(
+ refreshActiveGraphqlSoupQueries({ throwOnError: true })
+ ).rejects.toThrow('every active query');
+ expect(success).toHaveBeenCalledOnce();
+ await expect(
+ refreshActiveGraphqlSoupQueries({ throwOnError: true })
+ ).resolves.toBeUndefined();
+ expect(success).toHaveBeenCalledTimes(2);
+ });
+
+ it('strict revalidation ignores disabled readers', async () => {
+ const disabled = vi.fn(async () => {
+ throw new Error('must not run');
+ });
+ register({ isEnabled: () => false, refresh: disabled });
+ await expect(
+ refreshActiveGraphqlSoupQueries({ throwOnError: true })
+ ).resolves.toBeUndefined();
+ expect(disabled).not.toHaveBeenCalled();
+ });
+
+ it('does not claim success for a reader registered during a refresh', async () => {
+ let finish!: () => void;
+ const firstPass = new Promise<void>((resolve) => {
+ finish = resolve;
+ });
+ register({ isEnabled: () => true, refresh: () => firstPass });
+ const refreshed = expect(
+ refreshActiveGraphqlSoupQueries({ throwOnError: true })
+ ).rejects.toThrow('every active query');
+ const newReader = vi.fn(async () => undefined);
+ register({ isEnabled: () => true, refresh: newReader });
+ finish();
+ await refreshed;
+ expect(newReader).not.toHaveBeenCalled();
+ await expect(
+ refreshActiveGraphqlSoupQueries({ throwOnError: true })
+ ).resolves.toBeUndefined();
+ expect(newReader).toHaveBeenCalledOnce();
+ });
+
+ it('a failing reader which unmounts no longer blocks strict completion', async () => {
+ const consoleError = vi
+ .spyOn(console, 'error')
+ .mockImplementation(() => {});
+ onTestFinished(() => consoleError.mockRestore());
+ let fail!: (error: Error) => void;
+ const pending = new Promise<void>((_resolve, reject) => {
+ fail = reject;
+ });
+ const unregister = register({
+ isEnabled: () => true,
+ refresh: () => pending,
+ });
+ const result = refreshActiveGraphqlSoupQueries({ throwOnError: true });
+ unregister();
+ fail(new Error('unmounted'));
+ await expect(result).resolves.toBeUndefined();
+ });
+
it('refreshes only enabled registered queries', async () => {
const enabledRefresh = vi.fn(async () => undefined);
const disabledRefresh = vi.fn(async () => undefined);
diff --git a/apps/web/src/lib/queries/soup/graphql/active-queries.ts b/apps/web/src/lib/queries/soup/graphql/active-queries.ts
index 5268445520..0532098f3f 100644
--- a/apps/web/src/lib/queries/soup/graphql/active-queries.ts
+++ b/apps/web/src/lib/queries/soup/graphql/active-queries.ts
@@ -13,16 +13,35 @@ export function registerActiveGraphqlSoupQuery(
return () => activeQueries.delete(query);
}
-/** Network-refreshes every mounted and enabled GraphQL Soup query. */
-export async function refreshActiveGraphqlSoupQueries(): Promise<void> {
+/** Network-refreshes every mounted and enabled GraphQL Soup query.
+ * Strict callers may only release optimistic state after every active reader
+ * succeeded. Other callers retain the existing best-effort behavior.
+ */
+export async function refreshActiveGraphqlSoupQueries(
+ options: { throwOnError?: boolean } = {}
+): Promise<void> {
+ const refreshed = new Set<ActiveGraphqlSoupQuery>();
await Promise.all(
[...activeQueries].map(async (query) => {
if (!query.isEnabled()) return;
try {
await query.refresh();
+ refreshed.add(query);
} catch (error) {
console.error('[graphql-soup] failed to refresh active query', error);
}
})
);
+ // A reader enabled/registered during this pass also needs a fresh result.
+ // Conversely, a failed reader that unmounted no longer blocks completion.
+ if (
+ options.throwOnError &&
+ [...activeQueries].some(
+ (query) => query.isEnabled() && !refreshed.has(query)
+ )
+ ) {
+ throw new Error(
+ 'GraphQL Soup revalidation did not succeed for every active query'
+ );
+ }
}
diff --git a/apps/web/src/lib/queries/soup/graphql/grouped-items.ts b/apps/web/src/lib/queries/soup/graphql/grouped-items.ts
index fcee49443e..8a73563023 100644
--- a/apps/web/src/lib/queries/soup/graphql/grouped-items.ts
+++ b/apps/web/src/lib/queries/soup/graphql/grouped-items.ts
@@ -27,6 +27,10 @@ import type { SoupAstBody, SoupAstItemsData, SoupAstParams } from '../items';
import { mapSoupPageToEntityList } from '../transform-utils';
import { makeGraphqlGroupedSoupInput } from './ast';
import type { GraphqlSoupAstItemsQuery } from './items';
+import {
+ usePendingGraphqlSoupDeleteIds,
+ withoutPendingGraphqlSoupDeletes,
+} from './optimistic-deletions';
export type GraphqlGroupedSoupAstItemsQueryArgs = {
params: SoupAstParams;
@@ -83,6 +87,7 @@ export function createGraphqlGroupedSoupAstItemsQuery(
options: Accessor<GraphqlGroupedSoupAstItemsQueryOptions>
): GraphqlSoupAstItemsQuery {
const instructionsIdQuery = useInstructionsMdIdQuery();
+ const pendingDeleteIds = usePendingGraphqlSoupDeleteIds();
const input = createMemo(() => {
const { params, body, groupBy } = args();
if (!groupBy) return;
@@ -138,7 +143,8 @@ export function createGraphqlGroupedSoupAstItemsQuery(
);
return {
- data: () => query.data,
+ data: () =>
+ withoutPendingGraphqlSoupDeletes(query.data, pendingDeleteIds()),
error,
isSupported,
isEnabled: () => query.isEnabled,
diff --git a/apps/web/src/lib/queries/soup/graphql/items.test.ts b/apps/web/src/lib/queries/soup/graphql/items.test.ts
index 497dc6b858..aa48128fcd 100644
--- a/apps/web/src/lib/queries/soup/graphql/items.test.ts
+++ b/apps/web/src/lib/queries/soup/graphql/items.test.ts
@@ -68,7 +68,17 @@ vi.mock('../transform-utils', () => ({
mapSoupPageToEntityList: mapSoupPageToEntityListMock,
}));
+vi.mock('@queries/client', async () => {
+ const { QueryClient } = await import('@tanstack/solid-query');
+ return { queryClient: new QueryClient() };
+});
+
+import { queryClient } from '@queries/client';
import { createGraphqlSoupAstItemsQuery } from './items';
+import {
+ createGraphqlSoupDeletion,
+ GRAPHQL_SOUP_DELETE_MUTATION_KEY,
+} from './optimistic-deletions';
type FakeExecution = {
variables: Record<string, unknown>;
@@ -147,6 +157,7 @@ function makeFakeClient(): {
describe('createGraphqlSoupAstItemsQuery', () => {
beforeEach(() => {
+ queryClient.clear();
vi.clearAllMocks();
useInstructionsMdIdQueryMock.mockReturnValue({ isSuccess: false });
mapSoupPageToEntityListMock.mockImplementation((page) => page.items);
@@ -213,6 +224,64 @@ describe('createGraphqlSoupAstItemsQuery', () => {
}
);
+ it('hides pending deletes across loaded pages and restores fresh data on failure', async () => {
+ const fake = makeFakeClient();
+ getGraphqlSoupClientMock.mockReturnValue(fake.client);
+ const { query, dispose } = createRoot((dispose) => ({
+ dispose,
+ query: createGraphqlSoupAstItemsQuery(
+ () => ({ params: { sort_method: 'updated_at' }, body: {} }),
+ () => ({ enabled: true })
+ ),
+ }));
+ try {
+ fake.executions[0].next(
+ graphqlSoupPage({ items: [{ id: 'a' }], next_cursor: 'next' })
+ );
+ const next = query.fetchNextPage();
+ await vi.waitFor(() => expect(fake.executions).toHaveLength(2));
+ fake.executions[1].next(
+ graphqlSoupPage({
+ items: [{ id: 'b' }, { id: 'keep' }],
+ next_cursor: null,
+ })
+ );
+ await next;
+ let reject!: (error: Error) => void;
+ const mutation = queryClient.getMutationCache().build(queryClient, {
+ mutationKey: GRAPHQL_SOUP_DELETE_MUTATION_KEY,
+ onMutate: () => ({
+ graphqlDeletion: createGraphqlSoupDeletion(['a', 'b']),
+ }),
+ onSettled: (_data, _error, _vars, context) =>
+ context?.graphqlDeletion.release(),
+ mutationFn: () =>
+ new Promise<void>((_resolve, fail) => {
+ reject = fail;
+ }),
+ });
+ const failed = expect(mutation.execute(undefined)).rejects.toThrow(
+ 'rejected'
+ );
+ const ids = () => query.data()?.entities.map((entity) => entity.id);
+ await vi.waitFor(() => expect(ids()).toEqual(['keep']));
+ // A late query response must neither resurrect pending deletes nor be
+ // overwritten by a whole-page snapshot rollback.
+ fake.executions[1].next(
+ graphqlSoupPage({
+ items: [{ id: 'b', name: 'updated' }, { id: 'new' }],
+ next_cursor: null,
+ })
+ );
+ expect(ids()).toEqual(['new']);
+ reject(new Error('rejected'));
+ await failed;
+ await vi.waitFor(() => expect(ids()).toEqual(['a', 'b', 'new']));
+ } finally {
+ dispose();
+ }
+ });
+
it('retains the page projection when only query activity changes', () => {
const fake = makeFakeClient();
getGraphqlSoupClientMock.mockReturnValue(fake.client);
diff --git a/apps/web/src/lib/queries/soup/graphql/items.ts b/apps/web/src/lib/queries/soup/graphql/items.ts
index 6bb6787d66..9c0a5fdc01 100644
--- a/apps/web/src/lib/queries/soup/graphql/items.ts
+++ b/apps/web/src/lib/queries/soup/graphql/items.ts
@@ -56,6 +56,10 @@ import {
} from '../transform-utils';
import { makeGraphqlSoupInput } from './ast';
import { isCachedMailView, materializeMailView } from './mail-view';
+import {
+ usePendingGraphqlSoupDeleteIds,
+ withoutPendingGraphqlSoupDeletes,
+} from './optimistic-deletions';
import {
materializeReconciledSoup,
soupItemKey,
@@ -98,6 +102,7 @@ export function createGraphqlSoupAstItemsQuery(
options: Accessor<GraphqlSoupAstItemsQueryOptions>
): GraphqlSoupAstItemsQuery {
const instructionsIdQuery = useInstructionsMdIdQuery();
+ const pendingDeleteIds = usePendingGraphqlSoupDeleteIds();
const [offline, setOffline] = createSignal(
typeof navigator !== 'undefined' && !navigator.onLine
);
@@ -622,11 +627,12 @@ export function createGraphqlSoupAstItemsQuery(
return {
data: createMemo(() => {
const data = displayData();
- return (
+ return withoutPendingGraphqlSoupDeletes(
data && {
...data,
oldestFetchedTimestamp: query.data?.data.oldestFetchedTimestamp,
- }
+ },
+ pendingDeleteIds()
);
}),
error,
diff --git a/apps/web/src/lib/queries/soup/graphql/keys.ts b/apps/web/src/lib/queries/soup/graphql/keys.ts
new file mode 100644
index 0000000000..286f9d0820
--- /dev/null
+++ b/apps/web/src/lib/queries/soup/graphql/keys.ts
@@ -0,0 +1,6 @@
+import { createQueryKeys } from '@lukemorales/query-key-factory';
+
+/** Ephemeral client state, outside the persisted REST Soup query namespace. */
+export const graphqlSoupKeys = createQueryKeys('graphql-soup', {
+ retainedDeletions: null,
+});
diff --git a/apps/web/src/lib/queries/soup/graphql/optimistic-deletions.test.ts b/apps/web/src/lib/queries/soup/graphql/optimistic-deletions.test.ts
new file mode 100644
index 0000000000..b782cf0627
--- /dev/null
+++ b/apps/web/src/lib/queries/soup/graphql/optimistic-deletions.test.ts
@@ -0,0 +1,268 @@
+import { QueryClient } from '@tanstack/solid-query';
+import { createRoot } from 'solid-js';
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+import type { SoupAstItemsData } from '../items';
+
+const refresh = vi.hoisted(() => vi.fn<() => Promise<void>>());
+vi.mock('./active-queries', () => ({
+ refreshActiveGraphqlSoupQueries: refresh,
+}));
+
+vi.mock('@queries/client', () => ({
+ get queryClient() {
+ return client;
+ },
+}));
+
+import {
+ createGraphqlSoupDeletion,
+ GRAPHQL_SOUP_DELETE_MUTATION_KEY,
+ GRAPHQL_SOUP_DELETE_RETENTION_MS,
+ usePendingGraphqlSoupDeleteIds,
+ withoutPendingGraphqlSoupDeletes,
+ withoutPendingSoupEntities,
+} from './optimistic-deletions';
+
+let client: QueryClient;
+let dispose: (() => void) | undefined;
+beforeEach(() => {
+ refresh.mockReset().mockResolvedValue(undefined);
+ client = new QueryClient();
+});
+afterEach(() => {
+ dispose?.();
+ client.clear();
+ vi.useRealTimers();
+});
+
+function data(): SoupAstItemsData {
+ return {
+ entities: ['a', 'b', 'c'].map(
+ (id) => ({ id, type: 'document' }) as SoupAstItemsData['entities'][number]
+ ),
+ groups: [
+ {
+ key: 'one',
+ label: 'One',
+ displayOrder: 0,
+ totalCount: 10,
+ itemIds: ['a', 'b'],
+ nextCursor: 'next',
+ },
+ {
+ key: 'two',
+ label: 'Two',
+ displayOrder: 1,
+ totalCount: 1,
+ itemIds: ['c'],
+ nextCursor: null,
+ },
+ ],
+ itemsById: Object.fromEntries(
+ ['a', 'b', 'c'].map((id) => [id, { tag: 'document', data: { id } }])
+ ) as SoupAstItemsData['itemsById'],
+ oldestFetchedTimestamp: 123,
+ };
+}
+
+describe('GraphQL Soup deletion lifecycle', () => {
+ it('retains confirmed ids after their completed mutation is garbage collected', async () => {
+ let finishRefresh!: () => void;
+ refresh.mockReturnValue(
+ new Promise<void>((resolve) => {
+ finishRefresh = resolve;
+ })
+ );
+ const deletion = createGraphqlSoupDeletion(['deleted']);
+ const ids = createRoot((cleanup) => {
+ dispose = cleanup;
+ return usePendingGraphqlSoupDeleteIds();
+ });
+ const mutation = client.getMutationCache().build(client, {
+ mutationKey: GRAPHQL_SOUP_DELETE_MUTATION_KEY,
+ onMutate: () => ({ graphqlDeletion: deletion }),
+ mutationFn: async () => true,
+ onSettled: () => deletion.settle(['deleted']),
+ });
+ try {
+ await mutation.execute(undefined);
+ expect(mutation.state.status).toBe('success');
+ client.getMutationCache().remove(mutation);
+ expect([...ids()]).toEqual(['deleted']);
+ finishRefresh();
+ await vi.waitFor(() => expect(ids().size).toBe(0));
+ } finally {
+ deletion.release();
+ }
+ });
+
+ it('bounds retries and expires retained ids even when every refresh fails', async () => {
+ vi.useFakeTimers();
+ refresh.mockRejectedValue(new Error('offline'));
+ const deletion = createGraphqlSoupDeletion(['deleted', 'failed']);
+ deletion.settle(['deleted']);
+ await vi.advanceTimersByTimeAsync(3000);
+ expect(refresh).toHaveBeenCalledTimes(3);
+ expect(deletion.ids()).toEqual(['deleted']);
+ await vi.advanceTimersByTimeAsync(GRAPHQL_SOUP_DELETE_RETENTION_MS - 3001);
+ expect(deletion.ids()).toEqual(['deleted']);
+ await vi.advanceTimersByTimeAsync(1);
+ expect(deletion.ids()).toEqual([]);
+ await vi.advanceTimersByTimeAsync(GRAPHQL_SOUP_DELETE_RETENTION_MS);
+ expect(refresh).toHaveBeenCalledTimes(3);
+ });
+
+ it('expires ids even if a refresh never settles and does not restart after a late failure', async () => {
+ vi.useFakeTimers();
+ let reject!: (error: Error) => void;
+ refresh.mockReturnValue(
+ new Promise<void>((_resolve, fail) => {
+ reject = fail;
+ })
+ );
+ const deletion = createGraphqlSoupDeletion(['deleted']);
+ deletion.settle(['deleted']);
+ await vi.advanceTimersByTimeAsync(GRAPHQL_SOUP_DELETE_RETENTION_MS);
+ expect(deletion.ids()).toEqual([]);
+ reject(new Error('late failure'));
+ await vi.advanceTimersByTimeAsync(GRAPHQL_SOUP_DELETE_RETENTION_MS);
+ expect(refresh).toHaveBeenCalledOnce();
+ });
+
+ it('a late success only releases its own overlapping operation', async () => {
+ let finishFirst!: () => void;
+ let finishSecond!: () => void;
+ refresh
+ .mockReturnValueOnce(
+ new Promise<void>((resolve) => {
+ finishFirst = resolve;
+ })
+ )
+ .mockReturnValueOnce(
+ new Promise<void>((resolve) => {
+ finishSecond = resolve;
+ })
+ );
+ const first = createGraphqlSoupDeletion(['shared']);
+ const second = createGraphqlSoupDeletion(['shared']);
+ try {
+ first.settle(['shared']);
+ second.settle(['shared']);
+ finishFirst();
+ await vi.waitFor(() => expect(first.ids()).toEqual([]));
+ expect(second.ids()).toEqual(['shared']);
+ finishSecond();
+ await vi.waitFor(() => expect(second.ids()).toEqual([]));
+ } finally {
+ first.release();
+ second.release();
+ }
+ });
+});
+
+describe('GraphQL Soup pending delete projection', () => {
+ it('preserves unchanged data identity', () => {
+ const original = data();
+ expect(withoutPendingGraphqlSoupDeletes(original, new Set())).toBe(
+ original
+ );
+ expect(withoutPendingGraphqlSoupDeletes(original, new Set(['other']))).toBe(
+ original
+ );
+ expect(
+ withoutPendingGraphqlSoupDeletes(undefined, new Set(['a']))
+ ).toBeUndefined();
+ });
+
+ it('filters grouped membership, row pools, and known counts without changing cursors or source data', () => {
+ const original = data();
+ const filtered = withoutPendingGraphqlSoupDeletes(
+ original,
+ new Set(['a', 'b'])
+ )!;
+ expect(filtered.entities).toEqual([original.entities[2]]);
+ expect(filtered.groups?.[0]).toEqual({
+ ...original.groups![0],
+ itemIds: [],
+ totalCount: 8,
+ });
+ expect(filtered.groups?.[1]).toBe(original.groups?.[1]);
+ expect(Object.keys(filtered.itemsById!)).toEqual(['c']);
+ expect(filtered.oldestFetchedTimestamp).toBe(123);
+ expect(original.entities).toHaveLength(3);
+ expect(original.groups?.[0].itemIds).toEqual(['a', 'b']);
+ expect(Object.keys(original.itemsById!)).toEqual(['a', 'b', 'c']);
+ expect(withoutPendingGraphqlSoupDeletes(original, new Set())).toBe(
+ original
+ );
+ });
+
+ it('filters flat/local-reconciled rows while preserving unrelated rows and metadata', () => {
+ const original = {
+ ...data(),
+ groups: undefined,
+ itemsById: undefined,
+ cachedMail: true,
+ };
+ const filtered = withoutPendingGraphqlSoupDeletes(
+ original,
+ new Set(['b'])
+ )!;
+ expect(filtered.entities).toEqual([
+ original.entities[0],
+ original.entities[2],
+ ]);
+ expect(filtered.entities[0]).toBe(original.entities[0]);
+ expect(filtered.cachedMail).toBe(true);
+ expect(filtered.oldestFetchedTimestamp).toBe(123);
+ expect(withoutPendingSoupEntities(original.entities, new Set())).toBe(
+ original.entities
+ );
+ });
+
+ it('keeps overlapping deletes hidden until each operation releases its ids', async () => {
+ const ids = createRoot((cleanup) => {
+ dispose = cleanup;
+ return usePendingGraphqlSoupDeleteIds();
+ });
+ let releaseFirst!: () => void;
+ let releaseSecond!: () => void;
+ const first = client
+ .getMutationCache()
+ .build(client, {
+ mutationKey: GRAPHQL_SOUP_DELETE_MUTATION_KEY,
+ onMutate: () => ({
+ graphqlDeletion: createGraphqlSoupDeletion(['a', 'b']),
+ }),
+ onSettled: (_data, _error, _vars, context) =>
+ context?.graphqlDeletion.release(),
+ mutationFn: () =>
+ new Promise<void>((resolve) => {
+ releaseFirst = resolve;
+ }),
+ })
+ .execute(undefined);
+ const second = client
+ .getMutationCache()
+ .build(client, {
+ mutationKey: GRAPHQL_SOUP_DELETE_MUTATION_KEY,
+ onMutate: () => ({
+ graphqlDeletion: createGraphqlSoupDeletion(['b', 'c']),
+ }),
+ onSettled: (_data, _error, _vars, context) =>
+ context?.graphqlDeletion.release(),
+ mutationFn: () =>
+ new Promise<void>((resolve) => {
+ releaseSecond = resolve;
+ }),
+ })
+ .execute(undefined);
+ await vi.waitFor(() => expect([...ids()]).toEqual(['a', 'b', 'c']));
+ releaseFirst();
+ await first;
+ await vi.waitFor(() => expect([...ids()]).toEqual(['b', 'c']));
+ releaseSecond();
+ await second;
+ await vi.waitFor(() => expect(ids().size).toBe(0));
+ });
+});
diff --git a/apps/web/src/lib/queries/soup/graphql/optimistic-deletions.ts b/apps/web/src/lib/queries/soup/graphql/optimistic-deletions.ts
new file mode 100644
index 0000000000..30cb417cd8
--- /dev/null
+++ b/apps/web/src/lib/queries/soup/graphql/optimistic-deletions.ts
@@ -0,0 +1,176 @@
+import { skipToken, useMutationState, useQuery } from '@tanstack/solid-query';
+import { type Accessor, createMemo, createSignal } from 'solid-js';
+import { queryClient } from '../../client';
+import type { SoupAstItemsData } from '../items';
+import { refreshActiveGraphqlSoupQueries } from './active-queries';
+import { graphqlSoupKeys } from './keys';
+
+export const GRAPHQL_SOUP_DELETE_MUTATION_KEY = [
+ 'graphql-soup',
+ 'delete',
+] as const;
+
+export const GRAPHQL_SOUP_DELETE_RETENTION_MS = 60_000;
+const REVALIDATION_ATTEMPTS = 3;
+const RETRY_DELAY_MS = 1_000;
+const RETAINED_DELETIONS_KEY = graphqlSoupKeys.retainedDeletions.queryKey;
+type RetainedDeletionIds = Array<Accessor<readonly string[]>>;
+
+export type GraphqlSoupDeletion = {
+ ids: Accessor<readonly string[]>;
+ /** Restore failed ids immediately and revalidate the confirmed deletions. */
+ settle: (deletedIds: readonly string[]) => void;
+ release: () => void;
+};
+
+/** Per-operation state outlives the mutation and its observers until revalidation. */
+export function createGraphqlSoupDeletion(
+ ids: readonly string[]
+): GraphqlSoupDeletion {
+ const [deletedIds, setDeletedIds] = createSignal(ids);
+ let settled = false;
+ let released = false;
+ let expiryTimer: ReturnType<typeof setTimeout> | undefined;
+ let retryTimer: ReturnType<typeof setTimeout> | undefined;
+ const release = () => {
+ if (released) return;
+ released = true;
+ clearTimeout(expiryTimer);
+ clearTimeout(retryTimer);
+ setDeletedIds([]);
+ queryClient.setQueryData<RetainedDeletionIds>(
+ RETAINED_DELETIONS_KEY,
+ (retained) => retained?.filter((ids) => ids !== deletedIds)
+ );
+ };
+ const revalidate = async (attempt: number): Promise<void> => {
+ try {
+ await refreshActiveGraphqlSoupQueries({ throwOnError: true });
+ release();
+ } catch {
+ if (released || attempt + 1 >= REVALIDATION_ATTEMPTS) return;
+ retryTimer = setTimeout(
+ () => {
+ retryTimer = undefined;
+ void revalidate(attempt + 1);
+ },
+ RETRY_DELAY_MS * 2 ** attempt
+ );
+ }
+ };
+ return {
+ ids: deletedIds,
+ release,
+ settle: (successfulIds) => {
+ if (settled || released) return;
+ settled = true;
+ setDeletedIds([...new Set(successfulIds)]);
+ if (successfulIds.length === 0) {
+ release();
+ // Reconcile uncertain failures without retaining a failed deletion.
+ if (ids.length > 0) void revalidate(0);
+ return;
+ }
+ // A mutation observer can detach before the API finishes, starting its
+ // GC clock early. Keep confirmed ids outside that mutation's lifetime.
+ queryClient.setQueryData<RetainedDeletionIds>(
+ RETAINED_DELETIONS_KEY,
+ (retained) => [...(retained ?? []), deletedIds]
+ );
+ // Cleanup is independent of hung refresh promises and query lifetime.
+ expiryTimer = setTimeout(release, GRAPHQL_SOUP_DELETE_RETENTION_MS);
+ void revalidate(0);
+ },
+ };
+}
+
+/** Only GraphQL-enabled bulk deletes opt into the display overlay. */
+export type GraphqlSoupDeleteContext = {
+ graphqlDeletion?: GraphqlSoupDeletion;
+};
+
+/** Reflect pending and confirmed REST deletions in GraphQL views. Completed
+ * mutations retain their own tombstones until strict revalidation or expiry.
+ */
+export function usePendingGraphqlSoupDeleteIds() {
+ const retained = useQuery(
+ () => ({
+ queryKey: RETAINED_DELETIONS_KEY,
+ queryFn: skipToken,
+ initialData: [] as RetainedDeletionIds,
+ // One empty index may live for the client session; its entries always have
+ // bounded lifetimes, even when no view is mounted to observe this query.
+ gcTime: Infinity,
+ }),
+ () => queryClient
+ );
+ const deletions = useMutationState(
+ () => ({
+ filters: {
+ mutationKey: GRAPHQL_SOUP_DELETE_MUTATION_KEY,
+ exact: true,
+ },
+ select: (mutation) =>
+ (mutation.state.context as GraphqlSoupDeleteContext | undefined)
+ ?.graphqlDeletion?.ids,
+ }),
+ () => queryClient
+ );
+ return createMemo<ReadonlySet<string>>(
+ () =>
+ new Set([
+ ...deletions().flatMap((ids) => ids?.() ?? []),
+ ...(retained.isSuccess ? retained.data.flatMap((ids) => ids()) : []),
+ ])
+ );
+}
+
+/** Preserve unaffected rows and array identity, including when no delete is pending. */
+export function withoutPendingSoupEntities<T extends { id: string }>(
+ entities: T[],
+ deletedIds: ReadonlySet<string>
+): T[] {
+ if (
+ deletedIds.size === 0 ||
+ !entities.some((entity) => deletedIds.has(entity.id))
+ )
+ return entities;
+ return entities.filter((entity) => !deletedIds.has(entity.id));
+}
+
+/** Applies only at the GraphQL display boundary, after local/server reconciliation. */
+export function withoutPendingGraphqlSoupDeletes(
+ data: SoupAstItemsData | undefined,
+ deletedIds: ReadonlySet<string>
+): SoupAstItemsData | undefined {
+ if (!data || deletedIds.size === 0) return data;
+ const entities = withoutPendingSoupEntities(data.entities, deletedIds);
+ const groups = data.groups?.map((group) => {
+ const itemIds = group.itemIds.filter((id) => !deletedIds.has(id));
+ const removed = group.itemIds.length - itemIds.length;
+ return removed === 0
+ ? group
+ : {
+ ...group,
+ itemIds,
+ totalCount: Math.max(0, group.totalCount - removed),
+ };
+ });
+ const itemsById = data.itemsById;
+ const removedItemIds = itemsById
+ ? Object.keys(itemsById).filter((id) => deletedIds.has(id))
+ : [];
+ if (
+ entities === data.entities &&
+ groups?.every((group, index) => group === data.groups?.[index]) !== false &&
+ removedItemIds.length === 0
+ ) {
+ return data;
+ }
+ const remainingItems =
+ removedItemIds.length > 0 ? { ...itemsById } : itemsById;
+ if (remainingItems && removedItemIds.length > 0) {
+ for (const id of removedItemIds) delete remainingItems[id];
+ }
+ return { ...data, entities, groups, itemsById: remainingItems };
+}
diff --git a/apps/web/src/lib/queries/soup/grouped/graphql/create-graphql-grouped-soup-queries.test.ts b/apps/web/src/lib/queries/soup/grouped/graphql/create-graphql-grouped-soup-queries.test.ts
index b18e2f26f4..fbd668d40c 100644
--- a/apps/web/src/lib/queries/soup/grouped/graphql/create-graphql-grouped-soup-queries.test.ts
+++ b/apps/web/src/lib/queries/soup/grouped/graphql/create-graphql-grouped-soup-queries.test.ts
@@ -45,7 +45,17 @@ vi.mock('@service-storage/graphql-soup', () => ({
mapGraphqlGroupedSoupPage: mapGraphqlGroupedSoupPageMock,
}));
+vi.mock('@queries/client', async () => {
+ const { QueryClient } = await import('@tanstack/solid-query');
+ return { queryClient: new QueryClient() };
+});
+
+import { queryClient } from '@queries/client';
import { createGraphqlGroupedSoupAstItemsQuery } from '@queries/soup/graphql/grouped-items';
+import {
+ createGraphqlSoupDeletion,
+ GRAPHQL_SOUP_DELETE_MUTATION_KEY,
+} from '@queries/soup/graphql/optimistic-deletions';
import {
groupedSoupInputKey,
groupedSoupLogicalViewKey,
@@ -142,6 +152,7 @@ afterEach(() => {
});
beforeEach(() => {
+ queryClient.clear();
vi.clearAllMocks();
});
@@ -188,6 +199,47 @@ function setup() {
}
describe('createGraphqlGroupedSoupQueries', () => {
+ it('hides pending deletes from both initial and continuation pages without changing pagination', async () => {
+ const { fake, grouped } = setup();
+ fake.executions[0].next(page([group('a', ['a-1'], 'next')], [item('a-1')]));
+ const a = grouped.map().get('a')!;
+ const next = a.fetchNextPage();
+ fake.executions[1].next(
+ page([group('a', ['a-2', 'keep'], 'more')], [item('a-2'), item('keep')])
+ );
+ await next;
+ let finish!: () => void;
+ const mutation = queryClient
+ .getMutationCache()
+ .build(queryClient, {
+ mutationKey: GRAPHQL_SOUP_DELETE_MUTATION_KEY,
+ onMutate: () => ({
+ graphqlDeletion: createGraphqlSoupDeletion(['a-1', 'a-2']),
+ }),
+ onSettled: (_data, _error, _vars, context) =>
+ context?.graphqlDeletion.release(),
+ mutationFn: () =>
+ new Promise<void>((resolve) => {
+ finish = resolve;
+ }),
+ })
+ .execute(undefined);
+ await vi.waitFor(() => expect(entityIds(a)).toEqual(['keep']));
+ expect(grouped.map().get('a')).toBe(a);
+ expect(a.hasNextPage()).toBe(true);
+ expect(fake.executions).toHaveLength(2);
+ // Refreshes arriving before the deletes settle must stay filtered too.
+ fake.executions[0].next(
+ page([group('a', ['a-1', 'new'], 'next')], [item('a-1'), item('new')])
+ );
+ expect(entityIds(a)).toEqual(['new', 'keep']);
+ finish();
+ await mutation;
+ await vi.waitFor(() =>
+ expect(entityIds(a)).toEqual(['a-1', 'new', 'a-2', 'keep'])
+ );
+ });
+
it('uses only the parent initial subscription and reacts to parent group moves', () => {
const { fake, grouped } = setup();
diff --git a/apps/web/src/lib/queries/soup/grouped/graphql/create-graphql-grouped-soup-queries.ts b/apps/web/src/lib/queries/soup/grouped/graphql/create-graphql-grouped-soup-queries.ts
index 95010a5408..e58215b8ba 100644
--- a/apps/web/src/lib/queries/soup/grouped/graphql/create-graphql-grouped-soup-queries.ts
+++ b/apps/web/src/lib/queries/soup/grouped/graphql/create-graphql-grouped-soup-queries.ts
@@ -38,6 +38,11 @@ import {
onCleanup,
} from 'solid-js';
+import {
+ usePendingGraphqlSoupDeleteIds,
+ withoutPendingSoupEntities,
+} from '../../graphql/optimistic-deletions';
+
export type GroupQueryData = {
entities: EntityData[];
};
@@ -107,6 +112,7 @@ export function createGraphqlGroupedSoupQueries(
resetToInitialPage: () => void;
} {
const instructionsIdQuery = useInstructionsMdIdQuery();
+ const pendingDeleteIds = usePendingGraphqlSoupDeleteIds();
const mapItems = (
items: SoupAstItemsGroupedPage['items'],
@@ -282,8 +288,14 @@ export function createGraphqlGroupedSoupQueries(
const initial = initialData();
if (!initial) return;
const continued = getContinuation()?.query.data;
- if (!continued) return initial;
- return { entities: [...initial.entities, ...continued.entities] };
+ const combined = continued
+ ? { entities: [...initial.entities, ...continued.entities] }
+ : initial;
+ const entities = withoutPendingSoupEntities(
+ combined.entities,
+ pendingDeleteIds()
+ );
+ return entities === combined.entities ? combined : { entities };
});
const trackFirstPage = (action: Promise<unknown>): Promise<void> => {
diff --git a/crates/client/cache-wasm/src/shell.rs b/crates/client/cache-wasm/src/shell.rs
index 3b1310e4b5..6e2a6abcf4 100644
--- a/crates/client/cache-wasm/src/shell.rs
+++ b/crates/client/cache-wasm/src/shell.rs
@@ -776,6 +776,29 @@ pub fn schema_hash() -> String {
cache_core::meta::SCHEMA_HASH.to_string()
}
+#[derive(Serialize)]
+#[serde(rename_all = "camelCase")]
+struct CacheBuildInfo {
+ package_version: &'static str,
+ schema_hash: &'static str,
+ schema_compatibility_epoch: u32,
+ format_version: u32,
+ storage_schema_version: u32,
+}
+
+/// Read-only metadata embedded in this WASM binary, available without opening storage.
+/// Recovery fixtures compare both artifacts before invoking any destructive hook.
+#[wasm_bindgen(js_name = cacheBuildInfo)]
+pub fn cache_build_info() -> Result<JsValue, JsValue> {
+ to_js(&CacheBuildInfo {
+ package_version: env!("CARGO_PKG_VERSION"),
+ schema_hash: cache_core::meta::SCHEMA_HASH,
+ schema_compatibility_epoch: cache_core::codec::CACHE_SCHEMA_COMPATIBILITY_EPOCH,
+ format_version: cache_core::codec::CACHE_FORMAT_VERSION,
+ storage_schema_version: cache_turso::STORAGE_SCHEMA_VERSION,
+ })
+}
+
fn to_js<T: Serialize>(value: &T) -> Result<JsValue, JsValue> {
value
.serialize(&serde_wasm_bindgen::Serializer::json_compatible())
diff --git a/crates/client/cache-wasm/src/shell/test.rs b/crates/client/cache-wasm/src/shell/test.rs
index df4a65a676..26ea8cf31f 100644
--- a/crates/client/cache-wasm/src/shell/test.rs
+++ b/crates/client/cache-wasm/src/shell/test.rs
@@ -11,6 +11,26 @@ wasm_bindgen_test_configure!(run_in_dedicated_worker);
mod mail_projection;
+#[wasm_bindgen_test]
+fn build_info_reports_compiled_versions_without_opening_storage() {
+ let info: serde_json::Value =
+ serde_wasm_bindgen::from_value(cache_build_info().unwrap()).unwrap();
+ assert_eq!(info["packageVersion"], env!("CARGO_PKG_VERSION"));
+ assert_eq!(info["schemaHash"], schema_hash());
+ assert_eq!(
+ info["schemaCompatibilityEpoch"],
+ cache_core::codec::CACHE_SCHEMA_COMPATIBILITY_EPOCH
+ );
+ assert_eq!(
+ info["formatVersion"],
+ cache_core::codec::CACHE_FORMAT_VERSION
+ );
+ assert_eq!(
+ info["storageSchemaVersion"],
+ cache_turso::STORAGE_SCHEMA_VERSION
+ );
+}
+
const QUERY: &str = r#"query Soup($input: SoupInput!) {
user {
id
diff --git a/docs/AGENT_GUIDE/tasks.md b/docs/AGENT_GUIDE/tasks.md
index 5d85be1a11..fd55d9784a 100644
--- a/docs/AGENT_GUIDE/tasks.md
+++ b/docs/AGENT_GUIDE/tasks.md
@@ -46,6 +46,35 @@ filter sheets. Desktop uses the centered composer dialog.
Tasks are documents under the hood (creation hits `POST /dss/documents/create_task`), so they
also show up in Files/`All` and in AI-chat document listings.
+## Bulk delete
+
+Select task rows with their leading checkboxes, choose **Actions → Delete items**,
+then confirm **Delete**. With GraphQL Soup enabled, selected rows disappear while
+requests are pending, including rows loaded through grouped pagination. The
+confirmation closes when the whole batch succeeds. After a partial failure it
+reports how many items were deleted and keeps only failed items in the dialog for
+retry; successful items must not be submitted again. Confirmed deletions are
+removed from split histories immediately, even if the dialog is then canceled.
+Cancel clears stale selection and focuses a surviving failed item or a live
+neighbor; it does not undo successful deletions or run deferred email deletions.
+After a partial deletion is retried successfully, focus uses a surviving neighbor
+captured before deletion (next, then previous), rather than restarting at the top
+of the list. If both neighbors disappeared, it falls back near the original list
+position, skipping group headers and load-more rows.
+Failed items return to Soup and search immediately, while successful removals
+remain absent from both.
+Successfully deleted items stay hidden until all enabled GraphQL Soup lists
+revalidate successfully, even if the first refresh fails. Refresh is attempted
+at most three times (one- and two-second retry delays); suppression expires one
+minute after deletion finishes if revalidation remains unavailable. The
+GraphQL-disabled path retains its existing behavior.
+
+For verification, use disposable tasks and delay only their DELETE requests:
+rows should disappear before those requests complete. Then fail a Soup refresh:
+successful deletes should remain hidden after the confirmation closes and clear
+their suppression after a successful retry. Partial deletion failures restore
+only the failed items.
+
## View and edit task properties
An open task shows Status, Priority, and Assignees as property pills below its title. Task