Skip to content

armgit / macro

Public Imported from GitHub
0

Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.

Seanaye/fix/mail cache reconciliation (#9098)

Sean Aye5d7381aca70a
diff --git a/apps/web/AGENTS.md b/apps/web/AGENTS.md
index 0928c99822..5d2ce84d77 100644
--- a/apps/web/AGENTS.md
+++ b/apps/web/AGENTS.md
@@ -423,6 +423,16 @@ resolved session ID; keep the existing `AgentSessionProvider` subscription to
properties do not prove prior membership. Supply complete prior multi-value
memberships when adjusting counts: a row loaded in one bin may already be an
unloaded member of another.
+- Reconcile authoritative Mail first pages in the shared Soup projection adapter,
+ not a UI exclusion list: invalidate contradicted facts within the supported
+ page range without deleting normalized threads or treating backfill gaps as
+ deletions. The OPFS revisit/reload regression is
+ `bunx playwright test --config src/lib/graphql-cache/worker/browser-test/playwright.config.ts mail-reconciliation --project chromium`.
+ For transient row flicker, run
+ `bunx playwright test --config src/lib/queries/soup/graphql/browser-test/playwright.config.ts`:
+ it uses the real Soup source and exchange, holds local/network completion,
+ and asserts every observed row-ID state on revisits and an OPFS reload.
+ It also runs in `just test-cache-wasm-browser`.
- Reactive GraphQL reads use `createLiveQuery` for a document and
`useSoupAstItemsQuery` (backed by `createSoupLiveQuery`) for Soup lists; see
[choosing a reactive read](docs/graphql-normalized-cache-plan.md#choosing-a-reactive-read).
diff --git a/apps/web/docs/graphql-cache-testing.md b/apps/web/docs/graphql-cache-testing.md
index afd552ce78..e9aab73b78 100644
--- a/apps/web/docs/graphql-cache-testing.md
+++ b/apps/web/docs/graphql-cache-testing.md
@@ -21,6 +21,8 @@ flashes, stale subscribers, and notifications to unrelated fields.
| [Rename settlement](../src/features/entity/queries/rename.test.tsx) and [initiative transport](../src/lib/service-clients/service-storage/initiative.test.ts) | Queued partial responses, background rejection, permanent GraphQL permission failures | Legacy projections do not retain failed optimistic names. Queued responses are acknowledgements. Domain error codes survive optimistic settlement. |
| [Project creation](../src/features/projects/queries/create-project.test.ts) and [task assignment](../src/features/projects/queries/project-sources.test.ts) | Viewer/host/reset changes during creation; setting, clearing, and creating a task with its Project property | Old operations cannot publish into a replacement cache. Task assignment uses the shared optimistic property path and preserves per-task failures. |
| [Coordinator catch-up](../src/lib/graphql-cache/worker/coordinator-cache-changes.test.ts) and [parked browser writes](../src/lib/graphql-cache/worker/browser-test/parking.browser.e2e.ts) | Separate queued commit messages; failed/no-op tails; resets; hydration; continuous traffic; owner/caller departure | One broad refresh for a drained catch-up batch, with all writes retained. Search scopes union without promoting hydration. Reset/dependency/settlement pushes stay immediate; old revisions never leak across epochs. A real worker/WASM backlog of five commits produces one visible-cache refresh and reports its batch size. |
+| [Mail membership](../../../crates/soup_filter_cache_adapter/src/mail/test/reconciliation.rs) and [OPFS revisits](../src/lib/graphql-cache/worker/browser-test/mail-reconciliation.browser.e2e.ts) | Authoritative first-page gaps, timestamp/ID boundaries, empty pages, partial/backfill responses, optimistic edits, view revisits, engine/browser reopen | Contradicted facts stop matching without deleting normalized threads; later pages and unrelated filters remain intact. Backfill gaps and pending edits are not negative membership evidence. |
+| [Mail render transitions](../src/lib/queries/soup/graphql/browser-test/mail-flicker.browser.e2e.ts) | Real `createSoupLiveQuery` and normalized exchange over worker/WASM/OPFS; local evaluation and HTTP released separately; repeated Drafts → Inbox → Noise visits and reload | A `MutationObserver` records every row-ID change from cached baseline through local evaluation and network settlement. Repaired visits retain exactly the server IDs throughout, not merely at the end. The unrepaired first visit is a positive control. Each run attaches `rendered-row-transitions.json`. Only app singleton composition is replaced; the fixture uses a small Solid row renderer rather than the full Mail shell. |
| [Refresh ordering](../src/lib/queries/trailing-refetch.test.ts) | Concurrent invalidations, failed earlier reads, owner disposal | A change during a pending read gets a trailing read; disposed owners stop requesting work. |
The shared [Rust oracle](../../../crates/client/cache-core/tests/support/optimistic_model.rs)
diff --git a/apps/web/docs/graphql-normalized-cache-plan.md b/apps/web/docs/graphql-normalized-cache-plan.md
index 99e017b5f0..e166d3d1a4 100644
--- a/apps/web/docs/graphql-normalized-cache-plan.md
+++ b/apps/web/docs/graphql-normalized-cache-plan.md
@@ -954,6 +954,21 @@ Background hydration retains its existing opt-in notification contract. Soup's
local collection source opts in; ordinary foreground queries refresh on their
normal affected-operation notifications.
+## Authoritative Mail membership repair
+
+The shared Soup adapter reconciles supported `UPDATED_AT` Mail first pages
+inside `properties::augment_authoritative`, under the same engine lock and
+transaction as their normalized write. It compares up to 500 local matches
+against returned IDs, bounded by the last canonical view timestamp and ID when
+`nextCursor` exists. Exhausted/empty first pages cover the whole matching slice.
+Missing matches become durable incomplete projections, not deleted records or
+invented archive/signal facts. Full canonical snapshots restore their proof;
+partial patches cannot. Backfills, cursor pages, incomplete responses,
+unsupported predicates, identity switches, and pending optimistic rows do not
+supply this negative evidence. The existing authoritative-write telemetry covers
+this preparation/write path; no new worker command or database migration is needed.
+See [cached Mail verification](../../../docs/agent-guide/email.md#cached-mail-filtering).
+
## Cached email Quick Access
The disposable Quick Access projection is version 3. Email rows index the thread
diff --git a/apps/web/justfile b/apps/web/justfile
index 95a39ef4b8..5c50487acb 100644
--- a/apps/web/justfile
+++ b/apps/web/justfile
@@ -302,6 +302,7 @@ build-cache-wasm-browser-test-hooks:
# fresh-scope startup, DB-ready, first-ready, and linear-memory budgets.
test-cache-wasm-browser:
bunx --bun playwright test --config src/lib/graphql-cache/worker/browser-test/playwright.config.ts --project chromium --project firefox
+ bunx --bun playwright test --config src/lib/queries/soup/graphql/browser-test/playwright.config.ts --project chromium
bunx --bun playwright test --config src/lib/graphql-cache/worker/browser-test/playwright.production.config.ts --project chromium-production --project firefox-production
# Local profiling build: retain Rust symbols and DWARF source locations so an
diff --git a/apps/web/src/lib/graphql-cache/worker/browser-test/mail-reconciliation.browser.e2e.ts b/apps/web/src/lib/graphql-cache/worker/browser-test/mail-reconciliation.browser.e2e.ts
new file mode 100644
index 0000000000..d8e3d53aab
--- /dev/null
+++ b/apps/web/src/lib/graphql-cache/worker/browser-test/mail-reconciliation.browser.e2e.ts
@@ -0,0 +1,30 @@
+import { expect, test } from '@playwright/test';
+
+test('authoritative Mail gaps stay absent on revisits and an OPFS reload', async ({
+ page,
+}) => {
+ const scope = `mail-reconciliation-${crypto.randomUUID()}`;
+ await page.goto(`/mail-reconciliation.html?scope=${scope}`);
+ const rows = page.locator('#rows li');
+ await expect(rows).toHaveCount(8);
+ await page
+ .getByRole('button', { name: 'Apply authoritative Noise page' })
+ .click();
+ const expected = [20, 16, 8, 6, 4].map(
+ (n) =>
+ `GraphqlSoupEmailThread:00000000-0000-0000-0000-${String(n).padStart(12, '0')}`
+ );
+ await expect(rows).toHaveText(expected);
+ for (let visit = 0; visit < 3; visit += 1) {
+ await page.getByRole('button', { name: 'Drafts', exact: true }).click();
+ await expect(page.getByRole('status')).toHaveText('DRAFTS');
+ await page.getByRole('button', { name: 'Noise', exact: true }).click();
+ await expect(page.getByRole('status')).toHaveText('INBOX');
+ await expect(rows).toHaveText(expected);
+ }
+ // No reseeding and no socket deletion: persisted projection invalidation
+ // must survive closing the old worker's engine and reopening the OPFS file.
+ await page.goto('about:blank');
+ await page.goto(`/mail-reconciliation.html?scope=${scope}&seed=false`);
+ await expect(rows).toHaveText(expected);
+});
diff --git a/apps/web/src/lib/graphql-cache/worker/browser-test/mail-reconciliation.html b/apps/web/src/lib/graphql-cache/worker/browser-test/mail-reconciliation.html
new file mode 100644
index 0000000000..7f584ce49e
--- /dev/null
+++ b/apps/web/src/lib/graphql-cache/worker/browser-test/mail-reconciliation.html
@@ -0,0 +1,14 @@
+<!doctype html>
+<html lang="en">
+<head><meta charset="utf-8"><title>Mail page reconciliation</title></head>
+<body>
+<h1>Mail page reconciliation</h1>
+<p>Production worker/WASM/OPFS cache with synthetic mailbox data.</p>
+<button id="noise">Noise</button>
+<button id="drafts">Drafts</button>
+<button id="network">Apply authoritative Noise page</button>
+<p id="result" role="status">Loading</p>
+<ol id="rows"></ol>
+<script type="module" src="./mail-reconciliation.ts"></script>
+</body>
+</html>
diff --git a/apps/web/src/lib/graphql-cache/worker/browser-test/mail-reconciliation.ts b/apps/web/src/lib/graphql-cache/worker/browser-test/mail-reconciliation.ts
new file mode 100644
index 0000000000..10ff830d2c
--- /dev/null
+++ b/apps/web/src/lib/graphql-cache/worker/browser-test/mail-reconciliation.ts
@@ -0,0 +1,140 @@
+import { createWorkerCacheHost } from '../../host/worker-host';
+import { mailProjectionCapsules } from './mail-projection-capsules';
+
+const parameters = new URLSearchParams(location.search);
+const host = createWorkerCacheHost({
+ scope: parameters.get('scope') ?? `mail-reconcile-${crypto.randomUUID()}`,
+ requestTimeoutMs: 30_000,
+});
+const VIEWER = 'offline-mail-viewer';
+const id = (n: number) =>
+ `00000000-0000-0000-0000-${String(n).padStart(12, '0')}`;
+const NIL = id(0);
+const FILTERS = {
+ documentFilter: { literal: { id: NIL } },
+ projectFilter: { literal: { projectIdSelf: NIL } },
+ chatFilter: { literal: { chatId: NIL } },
+ calendarEventFilter: { literal: { id: NIL } },
+ channelFilter: { literal: { channelId: NIL } },
+ channelThreadFilter: { literal: { threadId: NIL } },
+ callFilter: { literal: { callId: NIL } },
+ crmCompanyFilter: { literal: { id: NIL } },
+ foreignEntityFilter: { literal: { id: NIL } },
+ emailFilter: { tree: { literal: { importance: false } } },
+};
+const QUERY = `query Soup($input: SoupInput!) {
+ user {
+ id emailLinks { id }
+ soup(input: $input) {
+ nextCursor
+ items {
+ __typename id cacheProjection isFavorited
+ ... on GraphqlSoupEmailThread {
+ linkId ownerId inboxVisible isRead isSignal updatedAt
+ latestInboundMessageTs reminderReturnedAt properties { id }
+ mailAllPreview { id } mailDraftPreview { id } mailSentPreview { id }
+ }
+ }
+ }
+ }
+}`;
+const VARIABLES = {
+ input: {
+ initial: {
+ filters: FILTERS,
+ emailView: 'INBOX',
+ sortMethod: 'UPDATED_AT',
+ sortDirection: 'DESC',
+ limit: 3,
+ },
+ },
+};
+const data = (ids: number[]) => ({
+ user: {
+ id: VIEWER,
+ emailLinks: [{ id: id(1000) }],
+ soup: {
+ nextCursor: 'server-continuation',
+ items: ids.map((n) => ({
+ __typename: 'GraphqlSoupEmailThread',
+ id: id(n),
+ linkId: id(1000),
+ ownerId: VIEWER,
+ inboxVisible: true,
+ isRead: false,
+ isSignal: false,
+ updatedAt: '2025-01-04T00:00:00Z',
+ latestInboundMessageTs: '2025-01-02T00:00:00Z',
+ reminderReturnedAt: null,
+ cacheProjection: mailProjectionCapsules[n - 1],
+ isFavorited: false,
+ properties: [],
+ mailAllPreview: { id: id(n + 10000) },
+ mailDraftPreview: { id: id(n + 20000) },
+ mailSentPreview: null,
+ })),
+ },
+ },
+});
+const result = document.querySelector<HTMLParagraphElement>('#result')!;
+const rows = document.querySelector<HTMLOListElement>('#rows')!;
+async function show(view: 'INBOX' | 'DRAFTS') {
+ const page = await host.entityFilter({
+ filters: FILTERS,
+ sortMethod: 'UPDATED_AT',
+ sortDirection: 'DESC',
+ limit: 100,
+ mail: { view },
+ });
+ if (page.kind !== 'mail-page') throw new Error(JSON.stringify(page));
+ rows.replaceChildren(
+ ...page.keys.map((key) => {
+ const row = document.createElement('li');
+ row.textContent = key;
+ return row;
+ })
+ );
+ result.textContent = view;
+}
+const failed = (error: unknown) => {
+ result.textContent = String(error);
+};
+document.querySelector('#noise')!.addEventListener('click', async () => {
+ try {
+ await show('INBOX');
+ } catch (error) {
+ failed(error);
+ }
+});
+document.querySelector('#drafts')!.addEventListener('click', async () => {
+ try {
+ await show('DRAFTS');
+ } catch (error) {
+ failed(error);
+ }
+});
+document.querySelector('#network')!.addEventListener('click', async () => {
+ try {
+ await host.writeQuery({
+ query: QUERY,
+ operationName: 'Soup',
+ variables: VARIABLES,
+ data: data([20, 16, 8]),
+ identity: VIEWER,
+ });
+ await show('INBOX');
+ } catch (error) {
+ failed(error);
+ }
+});
+if (parameters.get('seed') !== 'false') {
+ await host.writeQuery({
+ query: QUERY.replace('query Soup(', 'query SoupMailBackfill('),
+ operationName: 'SoupMailBackfill',
+ variables: VARIABLES,
+ data: data([20, 18, 16, 14, 10, 8, 6, 4]),
+ identity: VIEWER,
+ });
+}
+await show('INBOX');
+window.addEventListener('pagehide', () => host.dispose(), { once: true });
diff --git a/apps/web/src/lib/graphql-cache/worker/browser-test/playwright.config.ts b/apps/web/src/lib/graphql-cache/worker/browser-test/playwright.config.ts
index 2cdfa7afd0..7041918e14 100644
--- a/apps/web/src/lib/graphql-cache/worker/browser-test/playwright.config.ts
+++ b/apps/web/src/lib/graphql-cache/worker/browser-test/playwright.config.ts
@@ -14,6 +14,7 @@ export default defineConfig({
'cache-wasm-packaging.browser.e2e.ts',
'notification-projection.browser.e2e.ts',
'mail-projection.browser.e2e.ts',
+ 'mail-reconciliation.browser.e2e.ts',
'mail-tabs.browser.e2e.ts',
'query-write-scope.browser.e2e.ts',
'email-participants.browser.e2e.ts',
diff --git a/apps/web/src/lib/graphql-cache/worker/browser-test/vite.config.ts b/apps/web/src/lib/graphql-cache/worker/browser-test/vite.config.ts
index 9cccf35e0f..b6ab3f3040 100644
--- a/apps/web/src/lib/graphql-cache/worker/browser-test/vite.config.ts
+++ b/apps/web/src/lib/graphql-cache/worker/browser-test/vite.config.ts
@@ -55,6 +55,7 @@ export default defineConfig(({ command }) => ({
'cache-recovery.html',
'notification-projection.html',
'mail-projection.html',
+ 'mail-reconciliation.html',
'query-write-scope.html',
'email-participants.html',
'search-buckets.html',
diff --git a/apps/web/src/lib/queries/soup/graphql/browser-test/fixtures.ts b/apps/web/src/lib/queries/soup/graphql/browser-test/fixtures.ts
new file mode 100644
index 0000000000..3c93a5002f
--- /dev/null
+++ b/apps/web/src/lib/queries/soup/graphql/browser-test/fixtures.ts
@@ -0,0 +1,108 @@
+import { mailProjectionCapsules } from '../../../../graphql-cache/worker/browser-test/mail-projection-capsules';
+import type {
+ MailItemFieldsFragment,
+ SoupQuery,
+} from '../../../../service-clients/service-storage/graphql/generated/graphql';
+import type { SoupLiveQueryInput } from '../create-soup-live-query';
+
+export type MailTab = 'noise' | 'signal' | 'drafts';
+export const VIEWER = 'offline-mail-viewer';
+export const SERVER_IDS = [20, 16, 8];
+export const STALE_IDS = [18, 14, 10];
+export const CACHED_IDS = [20, 18, 16, 14, 10, 8];
+export const id = (n: number) =>
+ `00000000-0000-0000-0000-${String(n).padStart(12, '0')}`;
+const NIL = id(0);
+
+export function input(tab: MailTab): SoupLiveQueryInput {
+ return {
+ filters: {
+ documentFilter: { literal: { id: NIL } },
+ projectFilter: { literal: { projectIdSelf: NIL } },
+ chatFilter: { literal: { chatId: NIL } },
+ calendarEventFilter: { literal: { id: NIL } },
+ channelFilter: { literal: { channelId: NIL } },
+ channelThreadFilter: { literal: { threadId: NIL } },
+ callFilter: { literal: { callId: NIL } },
+ crmCompanyFilter: { literal: { id: NIL } },
+ foreignEntityFilter: { literal: { id: NIL } },
+ emailFilter: { tree: { literal: { importance: tab === 'signal' } } },
+ },
+ emailView: tab === 'drafts' ? 'DRAFTS' : 'INBOX',
+ sortMethod: 'UPDATED_AT',
+ sortDirection: 'DESC',
+ limit: 100,
+ };
+}
+
+function thread(n: number): MailItemFieldsFragment {
+ return {
+ __typename: 'GraphqlSoupEmailThread',
+ id: id(n),
+ entityType: 'EMAIL_THREAD',
+ displayName: `Thread ${n}`,
+ emailName: `Thread ${n}`,
+ providerId: null,
+ linkId: id(1000),
+ ownerId: VIEWER,
+ inboxVisible: true,
+ isRead: false,
+ isSignal: false,
+ isImportant: false,
+ isDraft: false,
+ projectId: null,
+ createdAt: '2025-01-02T00:00:00Z',
+ updatedAt: '2025-01-04T00:00:00Z',
+ viewedAt: null,
+ latestInboundMessageTs: '2025-01-02T00:00:00Z',
+ latestOutboundMessageTs: null,
+ latestNonSpamMessageTs: '2025-01-03T00:00:00Z',
+ reminderReturnedAt: null,
+ cacheProjection: mailProjectionCapsules[n - 1],
+ frecencyScore: null,
+ isFavorited: false,
+ properties: [],
+ notifications: [],
+ participants: [],
+ attachments: [],
+ labels: [],
+ snippet: null,
+ senderEmail: null,
+ senderName: null,
+ senderPhotoUrl: null,
+ mailDraftState: null,
+ mailAllPreview: {
+ id: id(n + 10000),
+ subject: `Thread ${n}`,
+ snippet: null,
+ isDraft: false,
+ senderEmail: null,
+ senderName: null,
+ senderPhotoUrl: null,
+ },
+ mailDraftPreview: null,
+ mailSentPreview: null,
+ };
+}
+
+export function response(ids: readonly number[]): SoupQuery {
+ return {
+ user: {
+ id: VIEWER,
+ emailLinks: [
+ {
+ id: id(1000),
+ macroId: VIEWER,
+ provider: 'GMAIL',
+ emailAddress: '[email protected]',
+ photoUrl: null,
+ isPrimary: true,
+ needsReauth: false,
+ draftIsSignal: false,
+ settings: { signature: null, signatureOnRepliesForwards: false },
+ },
+ ],
+ soup: { nextCursor: null, items: ids.map(thread) },
+ },
+ };
+}
diff --git a/apps/web/src/lib/queries/soup/graphql/browser-test/harness.tsx b/apps/web/src/lib/queries/soup/graphql/browser-test/harness.tsx
new file mode 100644
index 0000000000..17aadb3098
--- /dev/null
+++ b/apps/web/src/lib/queries/soup/graphql/browser-test/harness.tsx
@@ -0,0 +1,130 @@
+import { stringifyDocument } from '@urql/core';
+import { createSignal, For, Show } from 'solid-js';
+import { render } from 'solid-js/web';
+import { SoupMailBackfillDocument } from '../../../../service-clients/service-storage/graphql/generated/graphql';
+import { createSoupLiveQuery } from '../create-soup-live-query';
+import { CACHED_IDS, input, type MailTab, response, VIEWER } from './fixtures';
+import {
+ cacheErrors,
+ cacheRevision,
+ holdLocalReads,
+ host,
+ pendingWrites,
+ releaseLocalReads,
+ setCacheRevision,
+} from './runtime';
+
+const container = document.querySelector<HTMLDivElement>('#app')!;
+const timeline: Array<{ ids: string[]; source: string | null }> = [];
+let recording = false;
+function recordRows() {
+ if (!recording) return;
+ const ids = [
+ ...container.querySelectorAll<HTMLElement>('[data-thread-id]'),
+ ].map((row) => row.dataset.threadId!);
+ const previous = timeline.at(-1);
+ if (previous && JSON.stringify(previous.ids) === JSON.stringify(ids)) return;
+ timeline.push({
+ ids,
+ source:
+ container.querySelector('#mail-list')?.getAttribute('data-source') ??
+ null,
+ });
+}
+const observer = new MutationObserver(recordRows);
+observer.observe(container, {
+ childList: true,
+ subtree: true,
+ attributes: true,
+});
+
+const api = {
+ releaseLocalReads,
+ startRecording() {
+ timeline.length = 0;
+ recording = true;
+ recordRows();
+ },
+ stopRecording() {
+ recordRows();
+ recording = false;
+ return {
+ timeline: structuredClone(timeline),
+ cacheErrors: [...cacheErrors],
+ };
+ },
+};
+declare global {
+ interface Window {
+ mailFlicker: typeof api;
+ }
+}
+window.mailFlicker = api;
+
+function MailList(props: { tab: MailTab }) {
+ const source = createSoupLiveQuery(() => input(props.tab));
+ const phase = () => {
+ if (
+ source.localRevision() === undefined ||
+ source.localRevision() !== cacheRevision()
+ )
+ return 'pending';
+ if (source.cachedMail()) return 'local';
+ if (!source.isFetching() && pendingWrites() === 0) return 'network';
+ return 'pending';
+ };
+ return (
+ <section id="mail-list" data-tab={props.tab} data-source={phase()}>
+ <p role="status">{source.error()?.message ?? phase()}</p>
+ <ol>
+ <For each={source.data()}>
+ {(thread) => <li data-thread-id={thread.id}>{thread.displayName}</li>}
+ </For>
+ </ol>
+ </section>
+ );
+}
+
+function App() {
+ const [tab, setTab] = createSignal<MailTab>('noise');
+ const navigate = (next: MailTab) => {
+ if (next === 'noise') holdLocalReads();
+ else releaseLocalReads();
+ setTab(next);
+ };
+ return (
+ <>
+ <h1>Mail cache/network flicker regression</h1>
+ <p>
+ Real Soup query source, normalized exchange, and worker/WASM/OPFS cache.
+ </p>
+ <button onClick={() => navigate('drafts')}>Drafts</button>
+ <button onClick={() => navigate('signal')}>Inbox</button>
+ <button onClick={() => navigate('noise')}>Noise</button>
+ <Show when={tab()} keyed>
+ {(current) => <MailList tab={current} />}
+ </Show>
+ </>
+ );
+}
+
+if (new URLSearchParams(location.search).get('seed') !== 'false') {
+ await host.writeQuery({
+ query: stringifyDocument(SoupMailBackfillDocument),
+ operationName: 'SoupMailBackfill',
+ variables: { input: { initial: { ...input('noise'), expand: true } } },
+ data: response(CACHED_IDS),
+ identity: VIEWER,
+ });
+}
+setCacheRevision(await host.currentRevision());
+const dispose = render(() => <App />, container);
+window.addEventListener(
+ 'pagehide',
+ () => {
+ observer.disconnect();
+ dispose();
+ host.dispose();
+ },
+ { once: true }
+);
diff --git a/apps/web/src/lib/queries/soup/graphql/browser-test/index.html b/apps/web/src/lib/queries/soup/graphql/browser-test/index.html
new file mode 100644
index 0000000000..d655861155
--- /dev/null
+++ b/apps/web/src/lib/queries/soup/graphql/browser-test/index.html
@@ -0,0 +1,5 @@
+<!doctype html>
+<html lang="en">
+<head><meta charset="utf-8"><title>Mail cache/network flicker regression</title></head>
+<body><div id="app"></div><script type="module" src="./harness.tsx"></script></body>
+</html>
diff --git a/apps/web/src/lib/queries/soup/graphql/browser-test/mail-flicker.browser.e2e.ts b/apps/web/src/lib/queries/soup/graphql/browser-test/mail-flicker.browser.e2e.ts
new file mode 100644
index 0000000000..621f520aa0
--- /dev/null
+++ b/apps/web/src/lib/queries/soup/graphql/browser-test/mail-flicker.browser.e2e.ts
@@ -0,0 +1,110 @@
+import { writeFile } from 'node:fs/promises';
+import { expect, test } from '@playwright/test';
+import type { SoupQueryVariables } from '../../../../service-clients/service-storage/graphql/generated/graphql';
+import { CACHED_IDS, id, response, SERVER_IDS, STALE_IDS } from './fixtures';
+
+function heldResponse() {
+ return {
+ requested: Promise.withResolvers<void>(),
+ release: Promise.withResolvers<void>(),
+ };
+}
+
+test('Noise never reintroduces repaired stale threads between cached and network renders', async ({
+ page,
+}, testInfo) => {
+ const scope = `mail-flicker-${crypto.randomUUID()}`;
+ const errors: string[] = [];
+ page.on('pageerror', (error) => errors.push(error.message));
+ let network = heldResponse();
+ let noiseRequests = 0;
+ await page.route('**/__mail-flicker/graphql', async (route) => {
+ const request: { variables: SoupQueryVariables } = route
+ .request()
+ .postDataJSON();
+ const initial = request.variables.input.initial;
+ const noise =
+ initial?.emailView === 'INBOX' &&
+ initial.filters?.emailFilter?.tree?.literal?.importance === false;
+ if (noise) {
+ noiseRequests += 1;
+ const current = network;
+ current.requested.resolve();
+ await current.release.promise;
+ }
+ await route.fulfill({ json: { data: response(noise ? SERVER_IDS : []) } });
+ });
+ await page.goto(`/?scope=${scope}`);
+ const rows = page.locator('[data-thread-id]');
+ const list = page.locator('#mail-list');
+ const expected = SERVER_IDS.map(id);
+
+ // Positive control: on an unrepaired first visit, the observer must see the
+ // stale local rows and their removal. This proves it can detect the flash.
+ await network.requested.promise;
+ await expect(rows).toHaveCount(CACHED_IDS.length);
+ await page.evaluate(() => window.mailFlicker.startRecording());
+ await page.evaluate(() => window.mailFlicker.releaseLocalReads());
+ await expect(list).toHaveAttribute('data-source', 'local');
+ network.release.resolve();
+ await expect(list).toHaveAttribute('data-source', 'network');
+ await expect(rows).toHaveCount(SERVER_IDS.length);
+ const initial = await page.evaluate(() => window.mailFlicker.stopRecording());
+ expect(
+ initial.timeline.some((snapshot) =>
+ STALE_IDS.every((n) => snapshot.ids.includes(id(n)))
+ )
+ ).toBe(true);
+ expect(initial.timeline.at(-1)?.ids).toEqual(expected);
+
+ const visits = [];
+ for (const visit of ['first-return', 'second-return', 'opfs-reload']) {
+ network = heldResponse();
+ if (visit === 'opfs-reload') {
+ await page.goto('about:blank');
+ await page.goto(`/?scope=${scope}&seed=false`);
+ } else {
+ await page.getByRole('button', { name: 'Drafts', exact: true }).click();
+ await expect(list).toHaveAttribute('data-tab', 'drafts');
+ await expect(list).toHaveAttribute('data-source', 'network');
+ await page.getByRole('button', { name: 'Inbox', exact: true }).click();
+ await expect(list).toHaveAttribute('data-tab', 'signal');
+ await expect(list).toHaveAttribute('data-source', 'network');
+ await page.getByRole('button', { name: 'Noise', exact: true }).click();
+ }
+ await network.requested.promise;
+ // First show the persisted server baseline. Then allow the real local
+ // evaluation, while HTTP is still held. No fixed delay or frame sampling.
+ await expect(rows).toHaveText(['Thread 20', 'Thread 16', 'Thread 8']);
+ await page.evaluate(() => window.mailFlicker.startRecording());
+ await page.evaluate(() => window.mailFlicker.releaseLocalReads());
+ await expect(list).toHaveAttribute('data-source', 'local');
+ const localIds = await rows.evaluateAll((elements) =>
+ elements.map((element) => element.getAttribute('data-thread-id'))
+ );
+ network.release.resolve();
+ await expect(list).toHaveAttribute('data-source', 'network');
+ const observation = await page.evaluate(() =>
+ window.mailFlicker.stopRecording()
+ );
+ visits.push({ visit, localIds, ...observation });
+ }
+ const evidence = testInfo.outputPath('rendered-row-transitions.json');
+ await writeFile(evidence, JSON.stringify({ initial, visits }, null, 2));
+ await testInfo.attach('rendered-row-transitions', {
+ path: evidence,
+ contentType: 'application/json',
+ });
+ expect(errors).toEqual([]);
+ expect(noiseRequests).toBe(4);
+ for (const observation of visits) {
+ expect(observation.cacheErrors).toEqual([]);
+ // Assert every recorded DOM state, not just an eventually correct list.
+ // Without the Rust repair this is [server, server+stale, server].
+ expect(
+ observation.timeline.map((snapshot) => snapshot.ids),
+ JSON.stringify(observation, null, 2)
+ ).toEqual([expected]);
+ expect(observation.localIds).toEqual(expected);
+ }
+});
diff --git a/apps/web/src/lib/queries/soup/graphql/browser-test/playwright.config.ts b/apps/web/src/lib/queries/soup/graphql/browser-test/playwright.config.ts
new file mode 100644
index 0000000000..66da8cbfff
--- /dev/null
+++ b/apps/web/src/lib/queries/soup/graphql/browser-test/playwright.config.ts
@@ -0,0 +1,31 @@
+import { fileURLToPath } from 'node:url';
+import { defineConfig, devices } from '@playwright/test';
+
+const DIRECTORY = fileURLToPath(new URL('.', import.meta.url));
+const WEB = fileURLToPath(new URL('../../../../../../', import.meta.url));
+export default defineConfig({
+ testDir: DIRECTORY,
+ testMatch: '*.browser.e2e.ts',
+ timeout: 90_000,
+ workers: 1,
+ reporter: 'line',
+ use: { baseURL: 'http://127.0.0.1:4196', headless: true },
+ webServer: {
+ command: `just build-cache-wasm && bunx --bun vite --config ${JSON.stringify(`${DIRECTORY}/vite.config.ts`)}`,
+ cwd: WEB,
+ url: 'http://127.0.0.1:4196',
+ reuseExistingServer: false,
+ timeout: 180_000,
+ },
+ projects: [
+ {
+ name: 'chromium',
+ use: {
+ ...devices['Desktop Chrome'],
+ launchOptions: {
+ executablePath: process.env.PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH,
+ },
+ },
+ },
+ ],
+});
diff --git a/apps/web/src/lib/queries/soup/graphql/browser-test/runtime.ts b/apps/web/src/lib/queries/soup/graphql/browser-test/runtime.ts
new file mode 100644
index 0000000000..ce5c010c04
--- /dev/null
+++ b/apps/web/src/lib/queries/soup/graphql/browser-test/runtime.ts
@@ -0,0 +1,60 @@
+import { QueryClient } from '@tanstack/solid-query';
+import { createClient, fetchExchange } from '@urql/core';
+import { createSignal } from 'solid-js';
+import { normalizedCacheExchange } from '../../../../graphql-cache/exchange/normalized-cache-exchange';
+import type { CacheHost } from '../../../../graphql-cache/host/types';
+import { createWorkerCacheHost } from '../../../../graphql-cache/host/worker-host';
+import { VIEWER } from './fixtures';
+
+// Only application composition is replaced: the query source, exchange,
+// fragments, worker and persisted cache all run their production implementations.
+export const queryClient = new QueryClient();
+export const cacheErrors: string[] = [];
+const PARAMETERS = new URLSearchParams(location.search);
+const PHYSICAL_HOST = createWorkerCacheHost({
+ scope: PARAMETERS.get('scope') ?? `mail-flicker-${crypto.randomUUID()}`,
+ requestTimeoutMs: 30_000,
+});
+let localReadGate = Promise.withResolvers<void>();
+export function holdLocalReads() {
+ localReadGate.resolve();
+ localReadGate = Promise.withResolvers<void>();
+}
+export function releaseLocalReads() {
+ localReadGate.resolve();
+}
+export const [pendingWrites, setPendingWrites] = createSignal(0);
+export const host: CacheHost = {
+ ...PHYSICAL_HOST,
+ async writeQuery(args) {
+ setPendingWrites((count) => count + 1);
+ try {
+ return await PHYSICAL_HOST.writeQuery(args);
+ } finally {
+ setPendingWrites((count) => count - 1);
+ }
+ },
+ async entityFilter(args) {
+ // Delay only transport; never fabricate a local result or its authority.
+ await localReadGate.promise;
+ return PHYSICAL_HOST.entityFilter(args);
+ },
+};
+export const [cacheRevision, setCacheRevision] = createSignal<string>();
+host.onCacheChanged(setCacheRevision, { includeHydration: true });
+const CLIENT = createClient({
+ url: `${location.origin}/__mail-flicker/graphql`,
+ exchanges: [
+ normalizedCacheExchange(host, {
+ extractIdentity: () => VIEWER,
+ onCacheError: (error) => cacheErrors.push(String(error)),
+ }),
+ fetchExchange,
+ ],
+});
+
+// Vite redirects only the source's app singleton import to these real instances.
+export const getGraphqlSoupClient = () => CLIENT;
+export const getGraphqlSoupCacheHost = () => host;
+export const hasGraphqlCacheHost = () => true;
+export const graphqlSoupProjectionSupported = () => true;
diff --git a/apps/web/src/lib/queries/soup/graphql/browser-test/vite.config.ts b/apps/web/src/lib/queries/soup/graphql/browser-test/vite.config.ts
new file mode 100644
index 0000000000..04f510f0d6
--- /dev/null
+++ b/apps/web/src/lib/queries/soup/graphql/browser-test/vite.config.ts
@@ -0,0 +1,35 @@
+import { fileURLToPath } from 'node:url';
+import { defineConfig } from 'vite';
+import solid from 'vite-plugin-solid';
+import tsconfigPaths from 'vite-tsconfig-paths';
+
+const DIRECTORY = fileURLToPath(new URL('.', import.meta.url));
+const WEB = fileURLToPath(new URL('../../../../../../', import.meta.url));
+const RUNTIME = fileURLToPath(new URL('./runtime.ts', import.meta.url));
+
+export default defineConfig({
+ root: DIRECTORY,
+ plugins: [
+ {
+ name: 'mail-flicker-composition',
+ enforce: 'pre',
+ resolveId(source, importer) {
+ if (
+ (importer?.endsWith('/graphql/create-soup-live-query.ts') &&
+ source ===
+ '../../../service-clients/service-storage/graphql-soup') ||
+ (importer?.endsWith('/graphql/optimistic-deletions.ts') &&
+ source === '../../client')
+ )
+ return RUNTIME;
+ },
+ },
+ solid(),
+ tsconfigPaths({ projects: [`${WEB}/tsconfig.json`] }),
+ ],
+ resolve: { dedupe: ['solid-js'] },
+ define: { __CACHE_WASM_BUILD_MODE__: JSON.stringify('development') },
+ server: { host: '127.0.0.1', port: 4196, strictPort: true },
+ worker: { format: 'es' },
+ build: { target: 'esnext' },
+});
diff --git a/crates/soup_filter_cache_adapter/AGENTS.md b/crates/soup_filter_cache_adapter/AGENTS.md
index 1c4e549148..0ef70aacdc 100644
--- a/crates/soup_filter_cache_adapter/AGENTS.md
+++ b/crates/soup_filter_cache_adapter/AGENTS.md
@@ -18,6 +18,12 @@ understand. Runs both natively (Tauri) and compiled to wasm (web).
and notification projection updates applied without a server round-trip.
- `src/channels.rs`, `src/database_rows.rs`, `src/mail.rs`, `src/properties.rs`:
per-entity-kind projection/patch logic.
+- `src/mail/reconciliation.rs`: invalidate contradicted Mail facts during an
+ authoritative `Soup` first-page write. Use the supported view/filter and
+ microsecond timestamp/key boundary; never infer thread deletion. Skip
+ backfills, partial/cursor pages, identity switches, and pending optimistic
+ rows. At most 500 local matches are checked per write; later full snapshots
+ restore completeness.
## Scope
diff --git a/crates/soup_filter_cache_adapter/src/mail.rs b/crates/soup_filter_cache_adapter/src/mail.rs
index 16f780ea01..7d828a6340 100644
--- a/crates/soup_filter_cache_adapter/src/mail.rs
+++ b/crates/soup_filter_cache_adapter/src/mail.rs
@@ -17,6 +17,8 @@ use soup_filter_projection::{
};
use std::borrow::Cow;
+pub(crate) mod reconciliation;
+
#[cfg(all(test, not(target_arch = "wasm32")))]
mod test;
diff --git a/crates/soup_filter_cache_adapter/src/mail/reconciliation.rs b/crates/soup_filter_cache_adapter/src/mail/reconciliation.rs
new file mode 100644
index 0000000000..1e197adcaf
--- /dev/null
+++ b/crates/soup_filter_cache_adapter/src/mail/reconciliation.rs
@@ -0,0 +1,187 @@
+//! Negative membership evidence from a complete, authoritative Mail first page.
+//! Invalidate contradicted projections, never infer entity deletion or a new view.
+
+use super::*;
+use item_filter_index::properties::{compile_mail, mail_profile};
+use predicate_index::{IndexQuery, PredicateExpr};
+use std::collections::HashMap;
+
+/// Called only for the canonical Soup operation, after composing its current
+/// projections. Backfills and cursor pages are not negative membership evidence.
+/// Preparation and the eventual write run under the host's engine lock.
+pub(crate) async fn first_page_updates<S: PredicateIndexStorage>(
+ storage: &S,
+ variables: &Map<String, Value>,
+ data: &Value,
+ incoming: &[ProjectionMutation],
+) -> Result<Vec<ProjectionMutation>, ProjectionError<S::Error>> {
+ let Some(initial) = variables
+ .get("input")
+ .and_then(|input| input.get("initial"))
+ else {
+ return Ok(vec![]);
+ };
+ // Canonical Mail order is view-specific activity time, not created/viewed time.
+ if initial.get("sortMethod").and_then(Value::as_str) != Some("UPDATED_AT") {
+ return Ok(vec![]);
+ }
+ let Some(view) = initial.get("emailView").and_then(Value::as_str) else {
+ return Ok(vec![]);
+ };
+ let direction = match initial.get("sortDirection").and_then(Value::as_str) {
+ Some("ASC") => SortDirection::Asc,
+ Some("DESC") | None => SortDirection::Desc,
+ _ => return Ok(vec![]),
+ };
+ let Some(user) = data.get("user") else {
+ return Ok(vec![]);
+ };
+ let Some(viewer) = user.get("id").and_then(Value::as_str) else {
+ return Ok(vec![]);
+ };
+ let Some(soup) = user.get("soup") else {
+ return Ok(vec![]);
+ };
+ let Some(items) = soup.get("items").and_then(Value::as_array) else {
+ return Ok(vec![]);
+ };
+ let exhausted = match soup.get("nextCursor") {
+ Some(Value::Null) => true,
+ Some(Value::String(_)) if !items.is_empty() => false,
+ _ => return Ok(vec![]),
+ };
+ let Some(keys) = items
+ .iter()
+ .map(|item| {
+ if item.get("__typename").and_then(Value::as_str) != Some(TYPE)
+ || !FIELDS.iter().all(|field| item.get(*field).is_some())
+ {
+ return None;
+ }
+ let id = uuid::Uuid::parse_str(item.get("id")?.as_str()?).ok()?;
+ RecordKey::new(format!("{TYPE}:{id}")).ok()
+ })
+ .collect::<Option<Vec<_>>>()
+ else {
+ return Ok(vec![]);
+ };
+ let records = storage
+ .get_batch(&[EntityKey::entity("GraphqlUser", &[viewer])])
+ .await
+ .map_err(ProjectionError::Storage)?;
+ let Some(Some(user)) = records.first() else {
+ return Ok(vec![]);
+ };
+ let Some(CacheValue::List(links)) = user.fields.get("emailLinks") else {
+ return Ok(vec![]);
+ };
+ let Some(links) = links
+ .iter()
+ .map(|link| match link {
+ CacheValue::Ref(key) => {
+ uuid::Uuid::parse_str(key.as_ref().strip_prefix("GraphqlEmailLink:")?).ok()
+ }
+ _ => None,
+ })
+ .collect::<Option<Vec<_>>>()
+ else {
+ return Ok(vec![]);
+ };
+ let Ok(ast) = materialize_graphql_filter(
+ initial
+ .get("filters")
+ .cloned()
+ .unwrap_or_else(|| Value::Object(Map::new())),
+ ) else {
+ return Ok(vec![]);
+ };
+ let Ok(LocalCompileOutcome::Supported(query)) = compile_mail(
+ &ast,
+ SoupFlatRequest {
+ sort: SoupIndexSort::UpdatedAt,
+ direction,
+ limit: predicate_index::MAX_QUERY_LIMIT,
+ has_cursor: false,
+ },
+ view,
+ &links,
+ viewer,
+ ) else {
+ return Ok(vec![]);
+ };
+ let documents: HashMap<_, _> = incoming
+ .iter()
+ .filter_map(|mutation| match mutation {
+ ProjectionMutation::Replace(document) if document.profile == mail_profile() => {
+ Some((&document.record_key, document))
+ }
+ _ => None,
+ })
+ .collect();
+ // Every server row must have complete canonical facts. A partial response
+ // cannot establish a safe boundary, even when nextCursor happens to be null.
+ let Some(positions) = keys
+ .iter()
+ .map(|key| {
+ let document = documents.get(key)?;
+ let value = document
+ .sort_facts
+ .iter()
+ .find(|fact| fact.attribute == query.as_query().sort_attribute)?
+ .value;
+ Some((value, key))
+ })
+ .collect::<Option<Vec<_>>>()
+ else {
+ return Ok(vec![]);
+ };
+ if positions
+ .windows(2)
+ .any(|pair| !query.sorts_after(pair[1], pair[0]))
+ {
+ return Ok(vec![]);
+ }
+ let mut bounded: IndexQuery = query.as_query().clone();
+ if !exhausted && let Some((value, key)) = positions.last() {
+ for partition in &mut bounded.partitions {
+ partition.predicate = PredicateExpr::And(
+ Box::new(partition.predicate.clone()),
+ Box::new(PredicateExpr::Not(Box::new(PredicateExpr::After {
+ attribute: bounded.sort_attribute.clone(),
+ value: *value,
+ key: (*key).clone(),
+ direction: bounded.sort_direction,
+ tie_direction: bounded.tie_break_direction,
+ }))),
+ );
+ }
+ }
+ let bounded = ValidatedIndexQuery::new(bounded).map_err(error)?;
+ // Bound each authoritative write's work to one index page. A severely stale
+ // cache converges over subsequent refreshes rather than scanning the mailbox.
+ let local = storage
+ .reconcile_predicate_index(&bounded, &[])
+ .await
+ .map_err(ProjectionError::Storage)?;
+ let returned: HashSet<_> = keys.into_iter().collect();
+ let missing: Vec<_> = local
+ .keys
+ .into_iter()
+ .filter(|key| !returned.contains(key))
+ .collect();
+ let pending = storage
+ .load_optimistic_projections(&missing)
+ .await
+ .map_err(ProjectionError::Storage)?;
+ Ok(missing
+ .into_iter()
+ .zip(pending)
+ .filter(|(_, pending)| pending.is_none())
+ .map(|(record_key, _)| ProjectionMutation::MarkIncomplete {
+ record_key,
+ profile: mail_profile(),
+ partition: vocabulary::partition(),
+ kind: ProjectionIncompleteKind::Dirty,
+ })
+ .collect())
+}
diff --git a/crates/soup_filter_cache_adapter/src/mail/test.rs b/crates/soup_filter_cache_adapter/src/mail/test.rs
index 4cdfaed434..d3095ca1a7 100644
--- a/crates/soup_filter_cache_adapter/src/mail/test.rs
+++ b/crates/soup_filter_cache_adapter/src/mail/test.rs
@@ -11,6 +11,7 @@ use soup_filter_projection::{
mod drafts;
mod optimistic;
+mod reconciliation;
mod reminders;
mod through;
diff --git a/crates/soup_filter_cache_adapter/src/mail/test/reconciliation.graphql b/crates/soup_filter_cache_adapter/src/mail/test/reconciliation.graphql
new file mode 100644
index 0000000000..d1d038c961
--- /dev/null
+++ b/crates/soup_filter_cache_adapter/src/mail/test/reconciliation.graphql
@@ -0,0 +1,29 @@
+query Soup($input: SoupInput!) {
+ user {
+ id
+ emailLinks { id }
+ soup(input: $input) {
+ items {
+ __typename
+ id
+ cacheProjection
+ isFavorited
+ ... on GraphqlSoupEmailThread {
+ linkId
+ ownerId
+ inboxVisible
+ isRead
+ isSignal
+ latestInboundMessageTs
+ reminderReturnedAt
+ mailAllPreview { id }
+ mailDraftPreview { id }
+ mailSentPreview { id }
+ updatedAt
+ properties { id }
+ }
+ }
+ nextCursor
+ }
+ }
+}
diff --git a/crates/soup_filter_cache_adapter/src/mail/test/reconciliation.rs b/crates/soup_filter_cache_adapter/src/mail/test/reconciliation.rs
new file mode 100644
index 0000000000..77fbf0e8f9
--- /dev/null
+++ b/crates/soup_filter_cache_adapter/src/mail/test/reconciliation.rs
@@ -0,0 +1,448 @@
+use super::*;
+
+const SOUP: &str = include_str!("reconciliation.graphql");
+
+fn noise() -> Value {
+ let mut filters = filters();
+ filters["emailFilter"] = json!({"tree":{"literal":{"importance":false}}});
+ filters
+}
+fn variables(direction: &str) -> Map<String, Value> {
+ json!({"input":{"initial":{
+ "filters":noise(), "sortMethod":"UPDATED_AT", "sortDirection":direction,
+ "emailView":"INBOX", "limit":3
+ }}})
+ .as_object()
+ .unwrap()
+ .clone()
+}
+fn response(ids: &[u128], next: Value) -> Value {
+ let items: Vec<_> = ids
+ .iter()
+ .map(|n| {
+ let mut item = row(*n);
+ item["properties"] = json!([]);
+ item["isFavorited"] = json!(false);
+ item["isSignal"] = json!(*n == 12);
+ item
+ })
+ .collect();
+ json!({"user":{"id":VIEWER,"emailLinks":[{"id":id(1000)}],"soup":{"items":items,"nextCursor":next}}})
+}
+async fn persist<S: PredicateIndexStorage>(
+ engine: &mut Engine<S>,
+ operation: &str,
+ vars: &Map<String, Value>,
+ data: &Value,
+) {
+ let query = SOUP.replace("query Soup(", &format!("query {operation}("));
+ let projections = projection_updates(
+ engine.schema(),
+ engine.storage(),
+ &query,
+ Some(operation),
+ vars,
+ data,
+ )
+ .await
+ .unwrap();
+ let projections = crate::properties::augment_authoritative(
+ engine.schema(),
+ engine.storage(),
+ &query,
+ Some(operation),
+ vars,
+ data,
+ true,
+ projections,
+ )
+ .await
+ .unwrap();
+ engine
+ .write_query_with_registration_and_projections(
+ None,
+ None,
+ NetworkWrite {
+ query: &query,
+ operation_name: Some(operation),
+ variables: vars,
+ data,
+ identity: Some(VIEWER),
+ },
+ projections,
+ )
+ .await
+ .unwrap();
+}
+async fn current_keys<S: PredicateIndexStorage>(
+ engine: &mut Engine<S>,
+ filter: Value,
+ direction: &str,
+) -> Vec<String> {
+ let PageResult::MailPage { keys, .. } = page_current(
+ engine,
+ "generation",
+ filter,
+ "UPDATED_AT",
+ direction,
+ 100,
+ PageRequest {
+ view: "INBOX".into(),
+ cursor: None,
+ through: None,
+ },
+ )
+ .await
+ .unwrap() else {
+ panic!("complete mail page")
+ };
+ keys
+}
+
+async fn bounded_reconciliation<S: PredicateIndexStorage>(storage: S) {
+ let mut engine = Engine::new(storage);
+ let vars = variables("DESC");
+ persist(
+ &mut engine,
+ "SoupMailBackfill",
+ &vars,
+ &response(&[20, 18, 16, 14, 12, 10, 8, 6, 4], Value::Null),
+ )
+ .await;
+ assert_eq!(current_keys(&mut engine, noise(), "DESC").await.len(), 8);
+ let network = response(&[20, 16, 8], json!("next-page"));
+ persist(&mut engine, "Soup", &vars, &network).await;
+ let expected: Vec<_> = [20, 16, 8, 6, 4]
+ .map(|n| format!("{TYPE}:{}", id(n)))
+ .into();
+ assert_eq!(current_keys(&mut engine, noise(), "DESC").await, expected);
+ // Re-evaluation, a repeated write and an engine restart cannot resurrect gaps.
+ persist(&mut engine, "Soup", &vars, &network).await;
+ let mut engine = Engine::new(engine.into_storage());
+ assert_eq!(current_keys(&mut engine, noise(), "DESC").await, expected);
+ assert!(
+ current_keys(&mut engine, filters(), "DESC")
+ .await
+ .contains(&format!("{TYPE}:{}", id(12)))
+ );
+ assert!(
+ engine
+ .storage()
+ .get_batch(&[EntityKey::entity(TYPE, &[&id(18)])])
+ .await
+ .unwrap()[0]
+ .is_some(),
+ "negative membership is not a normalized entity deletion"
+ );
+ // A fresh complete snapshot restores the facts; a partial patch cannot.
+ let partial = json!({"user":{"id":VIEWER,"soup":{"items":[{
+ "__typename":TYPE,"id":id(18),"isRead":true
+ }]}}});
+ persist(&mut engine, "Partial", &vars, &partial).await;
+ assert_eq!(current_keys(&mut engine, noise(), "DESC").await, expected);
+ persist(
+ &mut engine,
+ "SoupMailBackfill",
+ &vars,
+ &response(&[18], Value::Null),
+ )
+ .await;
+ assert!(
+ current_keys(&mut engine, noise(), "DESC")
+ .await
+ .contains(&format!("{TYPE}:{}", id(18)))
+ );
+}
+
+#[test]
+fn authoritative_mail_page_invalidates_gaps_without_deleting_threads() {
+ pollster::block_on(bounded_reconciliation(InMemoryStorage::new()));
+}
+#[test]
+fn turso_authoritative_mail_page_invalidates_gaps_without_deleting_threads() {
+ pollster::block_on(bounded_reconciliation(
+ cache_turso::TursoStorage::open_in_memory("mail-reconciliation").unwrap(),
+ ));
+}
+#[test]
+fn ascending_boundary_preserves_rows_after_the_server_page() {
+ pollster::block_on(async {
+ let mut engine = Engine::new(InMemoryStorage::new());
+ let vars = variables("ASC");
+ persist(
+ &mut engine,
+ "SoupMailBackfill",
+ &vars,
+ &response(&[4, 6, 8, 10, 14, 16], Value::Null),
+ )
+ .await;
+ persist(
+ &mut engine,
+ "Soup",
+ &vars,
+ &response(&[4, 8, 14], json!("next")),
+ )
+ .await;
+ assert_eq!(
+ current_keys(&mut engine, noise(), "ASC").await,
+ [4, 8, 14, 16].map(|n| format!("{TYPE}:{}", id(n)))
+ );
+ });
+}
+#[test]
+fn exhausted_and_empty_first_pages_revoke_older_matches() {
+ pollster::block_on(async {
+ let mut engine = Engine::new(InMemoryStorage::new());
+ let vars = variables("DESC");
+ persist(
+ &mut engine,
+ "SoupMailBackfill",
+ &vars,
+ &response(&[20, 18, 16, 12, 4], Value::Null),
+ )
+ .await;
+ persist(&mut engine, "Soup", &vars, &response(&[20], Value::Null)).await;
+ assert_eq!(
+ current_keys(&mut engine, noise(), "DESC").await,
+ [format!("{TYPE}:{}", id(20))]
+ );
+ persist(&mut engine, "Soup", &vars, &response(&[], Value::Null)).await;
+ assert!(current_keys(&mut engine, noise(), "DESC").await.is_empty());
+ assert_eq!(
+ current_keys(&mut engine, filters(), "DESC").await,
+ [format!("{TYPE}:{}", id(12))]
+ );
+ });
+}
+#[test]
+fn other_mail_views_use_their_own_timestamp_and_membership() {
+ pollster::block_on(async {
+ for view in ["ALL", "SENT", "DRAFTS"] {
+ let mut engine = Engine::new(InMemoryStorage::new());
+ let mut vars = variables("DESC");
+ vars["input"]["initial"]["emailView"] = json!(view);
+ let ids = if view == "DRAFTS" {
+ [36, 30, 24, 18, 6]
+ } else {
+ [28, 24, 20, 16, 8]
+ };
+ persist(
+ &mut engine,
+ "SoupMailBackfill",
+ &vars,
+ &response(&ids, Value::Null),
+ )
+ .await;
+ persist(
+ &mut engine,
+ "Soup",
+ &vars,
+ &response(&[ids[0], ids[2], ids[3]], json!("next")),
+ )
+ .await;
+ let PageResult::MailPage { keys, .. } = page_current(
+ &mut engine,
+ "generation",
+ noise(),
+ "UPDATED_AT",
+ "DESC",
+ 100,
+ PageRequest {
+ view: view.into(),
+ cursor: None,
+ through: None,
+ },
+ )
+ .await
+ .unwrap() else {
+ panic!("supported {view}")
+ };
+ assert_eq!(
+ keys,
+ [ids[0], ids[2], ids[3], ids[4]].map(|n| format!("{TYPE}:{}", id(n))),
+ "{view}"
+ );
+ }
+ });
+}
+
+#[test]
+fn pending_local_edits_survive_negative_server_membership() {
+ pollster::block_on(async {
+ let mut engine = Engine::new(InMemoryStorage::new());
+ let vars = variables("DESC");
+ persist(
+ &mut engine,
+ "SoupMailBackfill",
+ &vars,
+ &response(&[20, 18, 16, 14, 8], Value::Null),
+ )
+ .await;
+ let data = json!({"user":{"id":VIEWER,"soup":{"items":[{
+ "__typename":TYPE,"id":id(18),"isRead":true
+ }]}}});
+ let projections = optimistic_updates(
+ projection_updates(
+ engine.schema(),
+ engine.storage(),
+ PARTIAL,
+ None,
+ &Map::new(),
+ &data,
+ )
+ .await
+ .unwrap(),
+ );
+ let projections = crate::properties::augment_optimistic(
+ engine.schema(),
+ engine.storage(),
+ PARTIAL,
+ None,
+ &Map::new(),
+ &data,
+ projections,
+ )
+ .await
+ .unwrap();
+ engine
+ .begin_optimistic_write_with_projections(
+ None,
+ BeginOptimisticWrite {
+ client_metadata: None,
+ uuid: "00000000-0000-0000-0000-000000009000",
+ query: PARTIAL,
+ operation_name: None,
+ variables: &Map::new(),
+ data: &data,
+ link_patches: &[],
+ revalidations: &[],
+ created_at_ms: 1,
+ identity_bindings: &[],
+ },
+ projections,
+ )
+ .await
+ .unwrap();
+ persist(
+ &mut engine,
+ "Soup",
+ &vars,
+ &response(&[20, 16, 8], json!("next")),
+ )
+ .await;
+ assert_eq!(
+ current_keys(&mut engine, noise(), "DESC").await,
+ [20, 18, 16, 8].map(|n| format!("{TYPE}:{}", id(n)))
+ );
+ let states = engine
+ .storage()
+ .load_projection_states(&[RecordKey::new(format!("{TYPE}:{}", id(18))).unwrap()])
+ .await
+ .unwrap();
+ assert!(
+ matches!(states[0], Some(ProjectionState::Complete(_))),
+ "the authoritative base must survive rollback of a pending edit"
+ );
+ });
+}
+
+#[test]
+fn view_timestamps_not_preview_updated_at_bound_the_page() {
+ pollster::block_on(async {
+ let mut engine = Engine::new(InMemoryStorage::new());
+ let vars = variables("DESC");
+ let mut seed = response(&[20, 18, 16, 14], Value::Null);
+ for (item, timestamp) in seed["user"]["soup"]["items"]
+ .as_array_mut()
+ .unwrap()
+ .iter_mut()
+ .zip([
+ "2025-01-02T00:00:00.000003Z",
+ "2025-01-02T00:00:00.000002Z",
+ "2025-01-02T00:00:00.000001Z",
+ "2025-01-02T00:00:00Z",
+ ])
+ {
+ item["latestInboundMessageTs"] = json!(timestamp);
+ }
+ persist(&mut engine, "SoupMailBackfill", &vars, &seed).await;
+ let mut network = seed.clone();
+ network["user"]["soup"]["items"] = json!([
+ seed["user"]["soup"]["items"][0],
+ seed["user"]["soup"]["items"][2]
+ ]);
+ network["user"]["soup"]["nextCursor"] = json!("next");
+ persist(&mut engine, "Soup", &vars, &network).await;
+ assert_eq!(
+ current_keys(&mut engine, noise(), "DESC").await,
+ [20, 16, 14].map(|n| format!("{TYPE}:{}", id(n)))
+ );
+ });
+}
+
+#[test]
+fn partial_backfill_cursor_and_unsupported_queries_are_not_negative_evidence() {
+ pollster::block_on(async {
+ for case in [
+ "backfill",
+ "cursor",
+ "unsupported",
+ "partial",
+ "no-cursor",
+ "unordered",
+ "created-at",
+ "no-mail",
+ ] {
+ let mut engine = Engine::new(InMemoryStorage::new());
+ let mut vars = variables("DESC");
+ persist(
+ &mut engine,
+ "SoupMailBackfill",
+ &vars,
+ &response(&[20, 18, 16, 4], Value::Null),
+ )
+ .await;
+ let mut data = response(&[20, 16], json!("next"));
+ let mut operation = "Soup";
+ match case {
+ "backfill" => operation = "SoupMailBackfill",
+ "cursor" => {
+ vars = json!({"input":{"cursor":"server-cursor"}})
+ .as_object()
+ .unwrap()
+ .clone()
+ }
+ "unsupported" => {
+ vars["input"]["initial"]["filters"]["emailFilter"] =
+ json!({"tree":{"literal":{"sender":{"partial":"test"}}}})
+ }
+ "partial" => data["user"]["soup"]["items"][1]
+ .as_object_mut()
+ .unwrap()
+ .remove("latestInboundMessageTs")
+ .map(|_| ())
+ .unwrap(),
+ "no-cursor" => data["user"]["soup"]
+ .as_object_mut()
+ .unwrap()
+ .remove("nextCursor")
+ .map(|_| ())
+ .unwrap(),
+ "unordered" => data["user"]["soup"]["items"]
+ .as_array_mut()
+ .unwrap()
+ .reverse(),
+ "created-at" => vars["input"]["initial"]["sortMethod"] = json!("CREATED_AT"),
+ "no-mail" => vars["input"]["initial"]["emailView"] = Value::Null,
+ _ => unreachable!(),
+ }
+ persist(&mut engine, operation, &vars, &data).await;
+ assert!(
+ current_keys(&mut engine, noise(), "DESC")
+ .await
+ .contains(&format!("{TYPE}:{}", id(18))),
+ "{case}"
+ );
+ }
+ });
+}
diff --git a/crates/soup_filter_cache_adapter/src/properties.rs b/crates/soup_filter_cache_adapter/src/properties.rs
index d3df8a47e0..87fde06443 100644
--- a/crates/soup_filter_cache_adapter/src/properties.rs
+++ b/crates/soup_filter_cache_adapter/src/properties.rs
@@ -200,6 +200,12 @@ pub async fn augment_authoritative<S: cache_core::predicate::PredicateIndexStora
{
return Err(error("Soup backfill contains an incomplete property projection").into());
}
+ if reuse_stored_identity && operation == Some("Soup") {
+ result.extend(
+ crate::mail::reconciliation::first_page_updates(storage, variables, data, &result)
+ .await?,
+ );
+ }
Ok(result)
}
diff --git a/docs/agent-guide/email.md b/docs/agent-guide/email.md
index b72dafd708..f4a3378b0e 100644
--- a/docs/agent-guide/email.md
+++ b/docs/agent-guide/email.md
@@ -335,6 +335,29 @@ when a newer normal message is the ALL preview; no message bodies are needed.
Sent also requires a canonical outbound timestamp. Trashed messages cannot supply
any preview. Calendar uses the authoritative thread calendar-attachment flag.
+A successful authoritative `Soup` first page also invalidates cached Mail facts
+that match its supported view/filter but are missing inside its covered sort
+range (including timestamp/ID ties). An exhausted page covers the remaining
+matching range, including an empty result. This checks at most 500 local matches
+per write; larger stale sets converge over refreshes. It preserves normalized
+threads and pending optimistic edits, and does not use cursor pages or backfill
+gaps as negative evidence. Invalidated facts stay unqueryable until a full
+canonical snapshot arrives, rather than guessing whether a thread was moved or
+deleted. Verification: seed stale Noise matches between returned rows, apply a
+first page, switch Drafts → Noise repeatedly, then reload without reseeding;
+stale rows must stay absent while rows beyond the page remain. This runs against
+real browser worker/WASM/OPFS storage in `mail-reconciliation.browser.e2e.ts`.
+The stronger `queries/soup/graphql/browser-test/mail-flicker.browser.e2e.ts`
+mounts the production Soup source and normalized exchange with a small Solid row
+renderer. It holds local evaluation until the saved server list renders, then
+holds HTTP until local evaluation finishes. Every observed DOM row-ID state must
+stay identical through network settlement, repeated Drafts → Inbox → Noise visits,
+and an OPFS reload. The unrepaired first visit must expose and remove the seeded
+stale rows as a positive control. Each run saves `rendered-row-transitions.json`;
+removing the adapter's `first_page_updates` call and rebuilding WASM must make
+this test fail on the transient extra rows, not on final membership or a timeout.
+Native persistence reconciliation requires a rebuilt native cache adapter.
+
Shared requires a last-known thread grant through the viewer, a team, or an active
channel, plus the existing Mail UI rule excluding viewer-owned threads. Merely
having a different owner or a delegated inbox does not qualify. Shared metadata has