Skip to content

armgit / macro

Public Imported from GitHub
0

Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.

Seanaye/feat/local filter from baseline (#6271)

Sean Ayec286bf4a23ca
diff --git a/Cargo.lock b/Cargo.lock
index fa68973a6c..873940d601 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -3000,7 +3000,7 @@ dependencies = [
[[package]]
name = "cache-wasm"
-version = "0.6.4"
+version = "0.6.5"
dependencies = [
"async-lock",
"cache-core",
diff --git a/apps/web/docs/local-predicate-pagination-plan.md b/apps/web/docs/local-predicate-pagination-plan.md
index 6cb2c06855..9d78cce810 100644
--- a/apps/web/docs/local-predicate-pagination-plan.md
+++ b/apps/web/docs/local-predicate-pagination-plan.md
@@ -1,6 +1,12 @@
# Revision-Safe Local Predicate Pagination Plan
-Status: **optimistic predicate-index prerequisite complete; ready for implementation**
+Status: **local pagination deferred; flat views now use server-baseline reconciliation**
+
+The implementation now reconciles loaded server-page membership with bounded local
+candidates, retaining unknown baseline rows and removing confirmed non-matches/deletions.
+It preserves the server cursor chain and labels the cache result `reconciled`, not
+`complete`. This is distinct from the exact local page-chain proposal below; that
+proposal's all-or-network authority assumptions must be revisited before implementation.
## Objective
@@ -23,7 +29,8 @@ Local predicate pagination does not exist today:
- `soup-filter-cache-adapter` always compiles with `has_cursor: false`;
- Turso predicate SQL orders and limits matches but has no keyset boundary;
- the frontend calls `entityFilter` only for GraphQL `initial` inputs;
-- once a local projection becomes authoritative, `fetchNextPage` discards it and returns to the stale server page chain.
+- `entityFilter` accepts optional baseline membership/sort evidence and returns revision-scoped reconciled keys;
+- `fetchNextPage` extends the unchanged server page chain, then reconciles all loaded baseline rows again.
After the optimistic fact-index prerequisite is complete, the index has the required effective document universe and ordering basis: one integer sort fact plus a stable normalized-record-key tie-breaker. The remaining work is cursor representation, keyset execution, revision validation, browser transport, and local page-chain ownership.
diff --git a/apps/web/src/lib/graphql-cache/protocol.ts b/apps/web/src/lib/graphql-cache/protocol.ts
index 4980f5f72d..889f45d100 100644
--- a/apps/web/src/lib/graphql-cache/protocol.ts
+++ b/apps/web/src/lib/graphql-cache/protocol.ts
@@ -77,15 +77,28 @@ export type SearchCachePage = {
nextCursor: SearchCursor | null;
};
-/** Exact initial-page request over the canonical GraphQL Soup filter input. */
+/** Maximum server-page membership evidence in one reconciliation request. */
+export const MAX_RECONCILIATION_BASELINE = 5_000;
+
+/** Initial-page candidates, optionally reconciled with same-query server pages. */
export type EntityFilterCacheArgs = {
filters: Record<string, unknown>;
sortMethod: 'CREATED_AT' | 'UPDATED_AT' | 'VIEWED_AT' | 'VIEWED_UPDATED';
sortDirection: 'ASC' | 'DESC';
limit: number;
+ /** Omit for exact evaluation; even an empty array requests reconciliation. */
+ baseline?: Array<{ key: string; sortTimestamp: string }>;
};
export type EntityFilterCacheResult =
+ | {
+ kind: 'reconciled';
+ revision: CacheRevision;
+ keys: string[];
+ /** Survivors backed only by previous server membership evidence. */
+ retainedKeys: string[];
+ optimistic: boolean;
+ }
| {
kind: 'complete';
revision: CacheRevision;
diff --git a/apps/web/src/lib/graphql-cache/worker/coordinator-protocol.test.ts b/apps/web/src/lib/graphql-cache/worker/coordinator-protocol.test.ts
index 17238e950c..8dc150cad1 100644
--- a/apps/web/src/lib/graphql-cache/worker/coordinator-protocol.test.ts
+++ b/apps/web/src/lib/graphql-cache/worker/coordinator-protocol.test.ts
@@ -24,6 +24,32 @@ const enginePort = {
} as unknown as MessagePort;
describe('coordinator runtime protocol', () => {
+ it('validates bounded reconciliation evidence without changing exact requests', () => {
+ const request = {
+ filters: {},
+ sortMethod: 'UPDATED_AT',
+ sortDirection: 'DESC',
+ limit: 20,
+ };
+ const valid = (baseline?: unknown) =>
+ isCacheRequest({
+ id: 1,
+ kind: 'entity-filter',
+ request: { ...request, baseline },
+ });
+ const entry = {
+ key: 'GraphqlSoupDocument:one',
+ sortTimestamp: '2026-01-01T00:00:00.123456Z',
+ };
+ expect(valid()).toBe(true);
+ expect(valid([])).toBe(true);
+ expect(valid([entry])).toBe(true);
+ expect(valid(Array(5001).fill(entry))).toBe(false);
+ expect(valid([{ ...entry, key: 'not-a-normalized-key' }])).toBe(false);
+ expect(valid([{ ...entry, sortTimestamp: 123 }])).toBe(false);
+ expect(valid([{ ...entry, unexpected: true }])).toBe(false);
+ });
+
it('validates cache RPCs and rejects unknown fields or kinds', () => {
expect(isCacheRequest({ id: 0, kind: 'clear' })).toBe(true);
expect(isCacheRequest({ id: 1, kind: 'current-revision' })).toBe(true);
diff --git a/apps/web/src/lib/graphql-cache/worker/coordinator-protocol.ts b/apps/web/src/lib/graphql-cache/worker/coordinator-protocol.ts
index 3494990bf0..d057663e4c 100644
--- a/apps/web/src/lib/graphql-cache/worker/coordinator-protocol.ts
+++ b/apps/web/src/lib/graphql-cache/worker/coordinator-protocol.ts
@@ -12,6 +12,7 @@ import {
isValidCacheSearchQuery,
isValidNormalizedRecordKey,
isWorkerMessage,
+ MAX_RECONCILIATION_BASELINE,
MAX_RECORD_SELECTION_PAGE_SIZE,
type WorkerMessage,
} from '../protocol';
@@ -557,13 +558,25 @@ export function isCacheRequest(value: unknown): value is CacheRequest {
'sortMethod',
'sortDirection',
'limit',
+ 'baseline',
]) &&
isRecord(request.filters) &&
['CREATED_AT', 'UPDATED_AT', 'VIEWED_AT', 'VIEWED_UPDATED'].includes(
request.sortMethod as string
) &&
(request.sortDirection === 'ASC' || request.sortDirection === 'DESC') &&
- isValidCacheSearchLimit(request.limit)
+ isValidCacheSearchLimit(request.limit) &&
+ (request.baseline === undefined ||
+ (Array.isArray(request.baseline) &&
+ request.baseline.length <= MAX_RECONCILIATION_BASELINE &&
+ request.baseline.every(
+ (entry) =>
+ isRecord(entry) &&
+ hasOnlyKeys(entry, ['key', 'sortTimestamp']) &&
+ isValidNormalizedRecordKey(entry.key) &&
+ typeof entry.sortTimestamp === 'string' &&
+ entry.sortTimestamp.length <= 64
+ )))
);
}
case 'inspect-query':
diff --git a/apps/web/src/lib/queries/soup/graphql/items.test.ts b/apps/web/src/lib/queries/soup/graphql/items.test.ts
index 4484a2ce05..2b5ac384c3 100644
--- a/apps/web/src/lib/queries/soup/graphql/items.test.ts
+++ b/apps/web/src/lib/queries/soup/graphql/items.test.ts
@@ -5,7 +5,7 @@ import type {
OperationContext,
OperationResult,
} from '@urql/core';
-import { createRoot } from 'solid-js';
+import { createRoot, createSignal } from 'solid-js';
import { beforeEach, describe, expect, it, vi } from 'vitest';
import { makeSubject } from 'wonka';
@@ -68,6 +68,7 @@ vi.mock('../transform-utils', () => ({
import { createGraphqlSoupAstItemsQuery } from './items';
type FakeExecution = {
+ variables: Record<string, unknown>;
next(
data: unknown,
metadata?: {
@@ -86,7 +87,12 @@ function graphqlSoupPage(page: SoupPageFixture) {
return {
user: {
soup: {
- items: page.items,
+ items: page.items.map((item) => ({
+ __typename: 'GraphqlSoupDocument',
+ createdAt: '2026-01-01T00:00:00Z',
+ updatedAt: '2026-01-01T00:00:00Z',
+ ...(item as object),
+ })),
nextCursor: page.next_cursor,
},
},
@@ -109,6 +115,7 @@ function makeFakeClient(): {
context,
} as Operation<unknown, Record<string, unknown>>;
executions.push({
+ variables: _request.variables,
next: (
data,
metadata = { source: 'live-network', revision: REVISION_1 }
@@ -174,7 +181,8 @@ describe('createGraphqlSoupAstItemsQuery', () => {
onCacheGenerationChanged: () => () => undefined,
});
entityFilterMock.mockResolvedValue({
- kind: 'complete',
+ kind: 'reconciled',
+ retainedKeys: [],
revision: REVISION_0,
keys: ['GraphqlSoupDocument:task-1'],
optimistic: false,
@@ -237,13 +245,15 @@ describe('createGraphqlSoupAstItemsQuery', () => {
});
entityFilterMock
.mockResolvedValueOnce({
- kind: 'complete',
+ kind: 'reconciled',
+ retainedKeys: [],
revision: REVISION_0,
keys: ['GraphqlSoupDocument:task-1'],
optimistic: true,
})
.mockResolvedValueOnce({
- kind: 'complete',
+ kind: 'reconciled',
+ retainedKeys: [],
revision: REVISION_2,
keys: ['GraphqlSoupDocument:task-1'],
optimistic: false,
@@ -312,6 +322,7 @@ describe('createGraphqlSoupAstItemsQuery', () => {
let notifyCacheChanged: (revision: string) => void = () => undefined;
let notifyGenerationChanged: () => void = () => undefined;
let resolveRevision4!: (value: unknown) => void;
+ let revision4Started = false;
const revision4Filter = new Promise((resolve) => {
resolveRevision4 = resolve;
});
@@ -328,7 +339,10 @@ describe('createGraphqlSoupAstItemsQuery', () => {
entityFilter: entityFilterMock,
});
entityFilterMock.mockImplementation(async () => {
- if (currentRevision === '4') return await revision4Filter;
+ if (currentRevision === '4') {
+ revision4Started = true;
+ return await revision4Filter;
+ }
const names =
currentRevision === REVISION_2
? ['Network item', 'Realtime item']
@@ -336,7 +350,8 @@ describe('createGraphqlSoupAstItemsQuery', () => {
? ['Latest local item']
: ['Replacement durable item'];
return {
- kind: 'complete',
+ kind: 'reconciled',
+ retainedKeys: [],
revision: currentRevision,
keys: names.map((_, index) => `GraphqlSoupDocument:item-${index}`),
optimistic: false,
@@ -408,24 +423,28 @@ describe('createGraphqlSoupAstItemsQuery', () => {
currentRevision = '4';
notifyCacheChanged('4');
- await vi.waitFor(() => expect(entityFilterMock).toHaveBeenCalledTimes(2));
+ await vi.waitFor(() => expect(revision4Started).toBe(true));
+ const callsBeforePendingChanges = entityFilterMock.mock.calls.length;
currentRevision = '5';
notifyCacheChanged('5');
currentRevision = '6';
notifyCacheChanged('6');
await Promise.resolve();
- expect(entityFilterMock).toHaveBeenCalledTimes(2);
+ expect(entityFilterMock).toHaveBeenCalledTimes(callsBeforePendingChanges);
expect(query.data()?.entities.map((item) => item.name)).toEqual([
'New network item',
]);
resolveRevision4({
- kind: 'complete',
+ kind: 'reconciled',
+ retainedKeys: [],
revision: '4',
keys: ['GraphqlSoupDocument:stale'],
optimistic: false,
});
await vi.waitFor(() => {
- expect(entityFilterMock).toHaveBeenCalledTimes(3);
+ expect(entityFilterMock).toHaveBeenCalledTimes(
+ callsBeforePendingChanges + 1
+ );
expect(query.data()?.entities.map((item) => item.name)).toEqual([
'Latest local item',
]);
@@ -442,6 +461,275 @@ describe('createGraphqlSoupAstItemsQuery', () => {
dispose();
});
+ it('merges baseline survivors with candidates without resetting server pagination', async () => {
+ const fake = makeFakeClient();
+ getGraphqlSoupClientMock.mockReturnValue(fake.client);
+ let revision = REVISION_1;
+ let notify: (revision: string) => void = () => {};
+ getGraphqlSoupCacheHostMock.mockReturnValue({
+ currentRevision: async () => revision,
+ entityFilter: entityFilterMock,
+ onCacheChanged: (callback: typeof notify) => {
+ notify = callback;
+ return () => {};
+ },
+ onCacheGenerationChanged: () => () => {},
+ });
+ makeGraphqlSoupInputMock.mockImplementation(({ cursor }) =>
+ cursor
+ ? { continuation: { cursor } }
+ : { initial: { sortMethod: 'UPDATED_AT', limit: 2 } }
+ );
+ entityFilterMock.mockImplementation(async ({ baseline }) => ({
+ kind: 'reconciled',
+ revision,
+ optimistic: false,
+ // 'removed' is a confirmed non-match, 'kept' is unknown. The latter
+ // need not materialize from current normalized storage to survive.
+ keys: [
+ 'GraphqlSoupDocument:new',
+ ...baseline
+ .filter((entry: { key: string }) => !entry.key.endsWith(':removed'))
+ .map((entry: { key: string }) => entry.key),
+ ],
+ retainedKeys: ['GraphqlSoupDocument:kept'],
+ }));
+ readRecordsByKeysMock.mockImplementation(async () => ({
+ revision,
+ records: [
+ {
+ recordKey: 'GraphqlSoupDocument:new',
+ record: { id: 'new', name: 'New candidate' },
+ },
+ ],
+ }));
+ let dispose!: () => void;
+ let query!: ReturnType<typeof createGraphqlSoupAstItemsQuery>;
+ createRoot((stop) => {
+ dispose = stop;
+ query = createGraphqlSoupAstItemsQuery(
+ () => ({ params: {}, body: {} }),
+ () => ({ enabled: true })
+ );
+ });
+ try {
+ fake.executions[0].next(
+ graphqlSoupPage({
+ items: [
+ { id: 'kept', name: 'Baseline survivor' },
+ { id: 'removed', name: 'No longer matches' },
+ ],
+ next_cursor: 'server-cursor-2',
+ })
+ );
+ revision = REVISION_2;
+ notify(revision);
+ await vi.waitFor(() =>
+ expect(query.data()?.entities.map((item) => item.name)).toEqual([
+ 'New candidate',
+ 'Baseline survivor',
+ ])
+ );
+ expect(fake.executions).toHaveLength(1);
+ expect(query.hasNextPage()).toBe(true);
+ const nextPage = query.fetchNextPage();
+ await vi.waitFor(() => expect(fake.executions).toHaveLength(2));
+ expect(fake.executions[1].variables).toEqual({
+ input: { continuation: { cursor: 'server-cursor-2' } },
+ });
+ revision = '3';
+ notify(revision);
+ fake.executions[1].next(
+ graphqlSoupPage({
+ items: [{ id: 'page-2', name: 'Second page' }],
+ next_cursor: null,
+ }),
+ { source: 'live-network', revision }
+ );
+ await nextPage;
+ await vi.waitFor(() =>
+ expect(query.data()?.entities.map((item) => item.name)).toEqual([
+ 'New candidate',
+ 'Baseline survivor',
+ 'Second page',
+ ])
+ );
+ expect(query.hasNextPage()).toBe(false);
+ expect(fake.executions).toHaveLength(2);
+ } finally {
+ dispose();
+ }
+ });
+
+ it('never supplies another filter or cache generation as baseline evidence', async () => {
+ const fake = makeFakeClient();
+ getGraphqlSoupClientMock.mockReturnValue(fake.client);
+ let revision = REVISION_1;
+ let notifyGeneration: () => void = () => {};
+ getGraphqlSoupCacheHostMock.mockReturnValue({
+ currentRevision: async () => revision,
+ entityFilter: entityFilterMock,
+ onCacheChanged: () => () => {},
+ onCacheGenerationChanged: (callback: () => void) => {
+ notifyGeneration = callback;
+ return () => {};
+ },
+ });
+ const [scope, setScope] = createSignal('one');
+ makeGraphqlSoupInputMock.mockImplementation(({ body }) => ({
+ initial: { sortMethod: 'UPDATED_AT', filters: body, limit: 100 },
+ }));
+ entityFilterMock.mockImplementation(async () => ({
+ kind: 'reconciled',
+ revision,
+ keys: [],
+ retainedKeys: [],
+ optimistic: false,
+ }));
+ readRecordsByKeysMock.mockImplementation(async () => ({
+ revision,
+ records: [],
+ }));
+ let dispose!: () => void;
+ let query!: ReturnType<typeof createGraphqlSoupAstItemsQuery>;
+ createRoot((stop) => {
+ dispose = stop;
+ query = createGraphqlSoupAstItemsQuery(
+ () => ({ params: {}, body: { scope: scope() } }) as never,
+ () => ({ enabled: true })
+ );
+ });
+ try {
+ fake.executions[0].next(
+ graphqlSoupPage({
+ items: [{ id: 'old', name: 'Old filter' }],
+ next_cursor: null,
+ })
+ );
+ setScope('two');
+ await vi.waitFor(() => expect(fake.executions).toHaveLength(2));
+ await vi.waitFor(() =>
+ expect(entityFilterMock).toHaveBeenCalledWith(
+ expect.objectContaining({ filters: { scope: 'two' }, baseline: [] })
+ )
+ );
+ expect(query.data()?.entities.some((item) => item.id === 'old')).not.toBe(
+ true
+ );
+ fake.executions[1].next(
+ graphqlSoupPage({
+ items: [{ id: 'new', name: 'New filter' }],
+ next_cursor: null,
+ })
+ );
+ revision = REVISION_0;
+ notifyGeneration();
+ await vi.waitFor(() =>
+ expect(entityFilterMock).toHaveBeenLastCalledWith(
+ expect.objectContaining({ baseline: [] })
+ )
+ );
+ await vi.waitFor(() => expect(query.data()?.entities).toEqual([]));
+ } finally {
+ dispose();
+ }
+ });
+
+ it.each(['unsupported', 'incomplete'])(
+ 'retains the network baseline when reconciliation is %s',
+ async (kind) => {
+ const fake = makeFakeClient();
+ getGraphqlSoupClientMock.mockReturnValue(fake.client);
+ let revision = REVISION_1;
+ let notify: (revision: string) => void = () => {};
+ getGraphqlSoupCacheHostMock.mockReturnValue({
+ currentRevision: async () => revision,
+ entityFilter: entityFilterMock,
+ onCacheChanged: (callback: typeof notify) => {
+ notify = callback;
+ return () => {};
+ },
+ onCacheGenerationChanged: () => () => {},
+ });
+ entityFilterMock.mockImplementation(async () => ({ kind, revision }));
+ let dispose!: () => void;
+ let query!: ReturnType<typeof createGraphqlSoupAstItemsQuery>;
+ createRoot((stop) => {
+ dispose = stop;
+ query = createGraphqlSoupAstItemsQuery(
+ () => ({ params: {}, body: {} }),
+ () => ({ enabled: true })
+ );
+ });
+ try {
+ fake.executions[0].next(
+ graphqlSoupPage({
+ items: [{ id: 'server', name: 'Server row' }],
+ next_cursor: null,
+ })
+ );
+ revision = REVISION_2;
+ notify(revision);
+ await vi.waitFor(() => expect(entityFilterMock).toHaveBeenCalled());
+ expect(query.data()?.entities.map((item) => item.name)).toEqual([
+ 'Server row',
+ ]);
+ } finally {
+ dispose();
+ }
+ }
+ );
+
+ it('batches large overlays and rejects mixed-revision materialization', async () => {
+ const fake = makeFakeClient();
+ getGraphqlSoupClientMock.mockReturnValue(fake.client);
+ getGraphqlSoupCacheHostMock.mockReturnValue({
+ currentRevision: async () => REVISION_2,
+ entityFilter: entityFilterMock,
+ onCacheChanged: () => () => {},
+ onCacheGenerationChanged: () => () => {},
+ });
+ const keys = Array.from(
+ { length: 501 },
+ (_, i) => `GraphqlSoupDocument:${i}`
+ );
+ entityFilterMock.mockResolvedValue({
+ kind: 'reconciled',
+ revision: REVISION_2,
+ keys,
+ retainedKeys: [],
+ optimistic: false,
+ });
+ let reads = 0;
+ readRecordsByKeysMock.mockImplementation(
+ async (_host, _selection, chunk: string[]) => ({
+ revision: reads++ === 0 ? REVISION_1 : REVISION_2,
+ records: chunk.map((recordKey) => ({
+ recordKey,
+ record: { id: recordKey, name: recordKey },
+ })),
+ })
+ );
+ let dispose!: () => void;
+ let query!: ReturnType<typeof createGraphqlSoupAstItemsQuery>;
+ createRoot((stop) => {
+ dispose = stop;
+ query = createGraphqlSoupAstItemsQuery(
+ () => ({ params: {}, body: {} }),
+ () => ({ enabled: true })
+ );
+ });
+ try {
+ await vi.waitFor(() => expect(query.data()?.entities).toHaveLength(501));
+ expect(entityFilterMock).toHaveBeenCalledTimes(2);
+ expect(
+ readRecordsByKeysMock.mock.calls.map((call) => call[2].length)
+ ).toEqual([500, 1, 500, 1]);
+ } finally {
+ dispose();
+ }
+ });
+
it('retains identical page projections across cache re-executions', () => {
const firstPage = {
items: [{ id: 'task-1', type: 'document', name: 'Task' }],
@@ -462,7 +750,7 @@ describe('createGraphqlSoupAstItemsQuery', () => {
const initial = query.data();
const initialEntity = initial?.entities[0];
- expect(initialEntity).toEqual(firstPage.items[0]);
+ expect(initialEntity).toMatchObject(firstPage.items[0]);
fake.executions[0]?.next(graphqlSoupPage(structuredClone(firstPage)));
expect(query.data()).toBe(initial);
diff --git a/apps/web/src/lib/queries/soup/graphql/items.ts b/apps/web/src/lib/queries/soup/graphql/items.ts
index 0bc2981d84..4e4439eb95 100644
--- a/apps/web/src/lib/queries/soup/graphql/items.ts
+++ b/apps/web/src/lib/queries/soup/graphql/items.ts
@@ -1,7 +1,8 @@
/*
* Reactive urql-backed Soup items. Every loaded page remains subscribed to
- * its normalized GraphQL cache operation so cache writes update the list
- * directly. The public REST/GraphQL facade lives in ../items.ts.
+ * its normalized GraphQL cache operation. Supported flat queries reconcile
+ * server membership with local evidence without replacing the server cursor
+ * chain. The public REST/GraphQL facade lives in ../items.ts.
*/
import {
@@ -19,7 +20,10 @@ import {
type SoupQuery,
type SoupQueryVariables,
} from '@service-storage/graphql/generated/graphql';
-import type { GraphqlSoupInput } from '@service-storage/graphql-soup';
+import type {
+ GraphqlSoupInput,
+ GraphqlSoupItem,
+} from '@service-storage/graphql-soup';
import {
getGraphqlSoupCacheHost,
getGraphqlSoupClient,
@@ -41,6 +45,10 @@ import {
import type { SoupAstBody, SoupAstItemsData, SoupAstParams } from '../items';
import { mapSoupPageToEntityList } from '../transform-utils';
import { makeGraphqlSoupInput } from './ast';
+import {
+ materializeReconciledSoup,
+ soupReconciliationBaseline,
+} from './reconciliation';
export type GraphqlSoupAstItemsQueryArgs = {
params: SoupAstParams;
@@ -92,10 +100,14 @@ export function createGraphqlSoupAstItemsQuery(
const firstPageInput = createMemo(() => inputForCursor(null));
const isSupported = () => firstPageInput() !== undefined;
+ type ServerProjection = {
+ data: SoupAstItemsData;
+ records: GraphqlSoupItem[];
+ };
type LocalProjection = {
revision: CacheRevision;
data: SoupAstItemsData;
- optimistic: boolean;
+ baseline: readonly GraphqlSoupItem[];
};
const [currentCacheRevision, setCurrentCacheRevision] = createSignal<
CacheRevision | undefined
@@ -112,8 +124,9 @@ export function createGraphqlSoupAstItemsQuery(
let localEvaluationRunning = false;
let localEvaluationPending = false;
let cacheGeneration = 0;
- let resetContinuationPages: (() => void) | undefined;
+ const [baselineGeneration, setBaselineGeneration] = createSignal<number>();
let previousInitialInput: GraphqlSoupInput | undefined;
+ let networkAuthorityInput: GraphqlSoupInput | undefined;
let staleFallbackSpan: ReturnType<typeof Telemetry.span> | undefined;
const recordAuthority = (source: 'network' | 'local' | 'stale-fallback') => {
@@ -136,6 +149,7 @@ export function createGraphqlSoupAstItemsQuery(
setCurrentCacheRevision(undefined);
setNetworkAuthorityRevision(undefined);
setLocalProjection(undefined);
+ setBaselineGeneration(undefined);
};
const observeCurrentRevision = () => {
const observedGeneration = cacheGeneration;
@@ -185,6 +199,7 @@ export function createGraphqlSoupAstItemsQuery(
const input = firstPageInput();
const queryOptions = options();
const host = getGraphqlSoupCacheHost();
+ const records = serverRecords();
const requestId = ++localRequest;
if (input !== previousInitialInput) {
previousInitialInput = input;
@@ -192,7 +207,8 @@ export function createGraphqlSoupAstItemsQuery(
}
if (
revision === undefined ||
- networkAuthorityRevision() === revision ||
+ (networkAuthorityInput === input &&
+ networkAuthorityRevision() === revision) ||
!queryOptions.enabled ||
!graphqlSoupProjectionSupported() ||
!input ||
@@ -209,7 +225,13 @@ export function createGraphqlSoupAstItemsQuery(
}
const filters = initial.filters ?? {};
const sortMethod = initial.sortMethod;
- if (!sortMethod || sortMethod === 'VIEWED_AT') {
+ if (sortMethod !== 'CREATED_AT' && sortMethod !== 'UPDATED_AT') {
+ localEvaluationPending = false;
+ return;
+ }
+ const baseline = soupReconciliationBaseline(records, sortMethod);
+ if (!baseline) {
+ setLocalProjection(undefined);
localEvaluationPending = false;
return;
}
@@ -236,24 +258,36 @@ export function createGraphqlSoupAstItemsQuery(
sortMethod,
sortDirection,
limit,
+ baseline,
});
- if (result.kind !== 'complete') return;
+ if (result.kind !== 'reconciled') return;
if (requestId !== localRequest) {
discarded = true;
return;
}
- const selected = await readRecordsByKeys(
- host,
- soupItemSelection,
- result.keys
- );
+ // Loaded server pages may exceed the bounded fragment-read API.
+ // Every chunk must still belong to the same reconciliation revision.
+ const chunks = [];
+ for (
+ let offset = 0;
+ offset < Math.max(1, result.keys.length);
+ offset += 500
+ ) {
+ chunks.push(
+ await readRecordsByKeys(
+ host,
+ soupItemSelection,
+ result.keys.slice(offset, offset + 500)
+ )
+ );
+ }
const latestRevision = await host.currentRevision();
if (requestId !== localRequest) {
discarded = true;
return;
}
if (
- result.revision !== selected.revision ||
+ chunks.some((chunk) => result.revision !== chunk.revision) ||
result.revision !== latestRevision ||
result.revision !== expectedRevision
) {
@@ -263,15 +297,17 @@ export function createGraphqlSoupAstItemsQuery(
setCurrentCacheRevision(latestRevision);
continue;
}
- if (selected.records.length !== result.keys.length) return;
- const items = selected.records.flatMap(({ record }) => {
+ const items = materializeReconciledSoup(
+ result.keys,
+ chunks.flatMap((chunk) => chunk.records),
+ records
+ ).flatMap((record) => {
const item = mapGraphqlSoupItem(record);
return item ? [item] : [];
});
- if (items.length !== result.keys.length) return;
setLocalProjection({
revision: latestRevision,
- optimistic: result.optimistic,
+ baseline: records,
data: {
entities: mapSoupPageToEntityList(
{ items, next_cursor: undefined },
@@ -287,7 +323,7 @@ export function createGraphqlSoupAstItemsQuery(
outcome = 'success';
recordAuthority('local');
finishStaleFallback('local');
- resetContinuationPages?.();
+ span.setAttr('evaluation.retained_count', result.retainedKeys.length);
return;
}
} catch {
@@ -312,7 +348,7 @@ export function createGraphqlSoupAstItemsQuery(
SoupQuery,
SoupQueryVariables,
string | null,
- SoupAstItemsData
+ ServerProjection
>(() => {
const firstInput = firstPageInput();
const queryOptions = options();
@@ -336,9 +372,11 @@ export function createGraphqlSoupAstItemsQuery(
requestPolicy: 'cache-and-network',
keepPreviousData: false,
onResult: (result, page) => {
+ if (result.data) setBaselineGeneration(cacheGeneration);
if (page.pageIndex !== 0) return;
const metadata = normalizedCacheResultMetadata(result);
if (metadata?.source !== 'live-network' || !metadata.revision) return;
+ networkAuthorityInput = firstInput;
batch(() => {
setCurrentCacheRevision(metadata.revision);
setNetworkAuthorityRevision(metadata.revision);
@@ -348,25 +386,40 @@ export function createGraphqlSoupAstItemsQuery(
finishStaleFallback('network');
},
select: ({ pages }) => ({
- entities: pages.flatMap((page) =>
- mapSoupPageToEntityList(mapGraphqlSoupPage(page), {
- instructionsIdQuery,
- showSupportedForeignEntities,
- })
- ),
- groups: undefined,
+ records: pages.flatMap((page) => page.user.soup.items),
+ data: {
+ entities: pages.flatMap((page) =>
+ mapSoupPageToEntityList(mapGraphqlSoupPage(page), {
+ instructionsIdQuery,
+ showSupportedForeignEntities,
+ })
+ ),
+ groups: undefined,
+ },
}),
};
});
- resetContinuationPages = query.resetToInitialPage;
+ // Snapshot the reactive normalized rows: their object identities may remain
+ // stable across writes, but baseline membership/sort evidence must not mutate
+ // beneath an in-flight reconciliation.
+ const serverRecords = createMemo(() =>
+ baselineGeneration() === cacheGeneration
+ ? (query.data?.records ?? []).map((record) => ({ ...record }))
+ : []
+ );
const authoritativeLocalProjection = (): LocalProjection | undefined => {
const local = localProjection();
const revision = currentCacheRevision();
- return local?.revision === revision ? local : undefined;
+ return local &&
+ local.revision === revision &&
+ local.baseline === serverRecords()
+ ? local
+ : undefined;
};
const networkIsAuthoritative = (): boolean =>
+ networkAuthorityInput === firstPageInput() &&
networkAuthorityRevision() !== undefined &&
networkAuthorityRevision() === currentCacheRevision();
@@ -381,9 +434,9 @@ export function createGraphqlSoupAstItemsQuery(
return {
data: () => {
- if (networkIsAuthoritative()) return query.data;
+ if (networkIsAuthoritative()) return query.data?.data;
const local = authoritativeLocalProjection();
- return local?.data ?? query.data ?? localProjection()?.data;
+ return local?.data ?? query.data?.data ?? localProjection()?.data;
},
error,
isSupported,
@@ -396,9 +449,7 @@ export function createGraphqlSoupAstItemsQuery(
!networkIsAuthoritative() && authoritativeLocalProjection() !== undefined,
hasNextPage: () => query.hasNextPage,
fetchNextPage: async () => {
- // Local predicate pagination is not revision-safe yet. Return to the
- // stale server page chain before requesting its continuation cursor.
- setLocalProjection(undefined);
+ // The display overlay never owns or resets the server cursor chain.
await query.fetchNextPage();
},
resetToInitialPage: query.resetToInitialPage,
diff --git a/apps/web/src/lib/queries/soup/graphql/reconciliation.test.ts b/apps/web/src/lib/queries/soup/graphql/reconciliation.test.ts
new file mode 100644
index 0000000000..162ad0bef3
--- /dev/null
+++ b/apps/web/src/lib/queries/soup/graphql/reconciliation.test.ts
@@ -0,0 +1,72 @@
+import type { GraphqlSoupItem } from '@service-storage/graphql-soup';
+import { describe, expect, it } from 'vitest';
+import {
+ materializeReconciledSoup,
+ soupItemKey,
+ soupReconciliationBaseline,
+} from './reconciliation';
+
+const item = (id: string, typename = 'GraphqlSoupDocument') =>
+ ({
+ __typename: typename,
+ id,
+ documentName: id,
+ createdAt: '2026-01-01T00:00:00.123456Z',
+ updatedAt: '2026-02-01T00:00:00.654321Z',
+ }) as GraphqlSoupItem;
+
+describe('flat Soup baseline reconciliation', () => {
+ it('uses normalized identities across entity kinds and preserves timestamp precision', () => {
+ const document = item('same');
+ const project = item('same', 'GraphqlSoupProject');
+ const chat = item('same', 'GraphqlSoupChat');
+ expect(
+ soupReconciliationBaseline(
+ [document, project, chat, document],
+ 'UPDATED_AT'
+ )
+ ).toEqual(
+ [document, project, chat].map((record) => ({
+ key: soupItemKey(record),
+ sortTimestamp: '2026-02-01T00:00:00.654321Z',
+ }))
+ );
+ expect(
+ soupReconciliationBaseline([document], 'CREATED_AT')?.[0].sortTimestamp
+ ).toBe('2026-01-01T00:00:00.123456Z');
+ });
+
+ it('refuses missing sort evidence and oversized baselines rather than dropping server rows', () => {
+ expect(
+ soupReconciliationBaseline(
+ [{ ...item('a'), updatedAt: undefined } as unknown as GraphqlSoupItem],
+ 'UPDATED_AT'
+ )
+ ).toBeUndefined();
+ expect(
+ soupReconciliationBaseline(
+ Array.from({ length: 5001 }, (_, i) => item(String(i))),
+ 'UPDATED_AT'
+ )
+ ).toBeUndefined();
+ });
+
+ it('retains unknown baseline display data, omits unrenderable new items and never resurrects removals', () => {
+ const kept = item('kept');
+ const removed = item('removed');
+ const newItem = item('new');
+ const selected = [{ recordKey: soupItemKey(newItem), record: newItem }];
+ expect(
+ materializeReconciledSoup(
+ [
+ soupItemKey(newItem),
+ soupItemKey(kept),
+ 'GraphqlSoupDocument:unrenderable',
+ soupItemKey(kept),
+ ],
+ selected,
+ [kept, removed]
+ )
+ ).toEqual([newItem, kept]);
+ });
+});
diff --git a/apps/web/src/lib/queries/soup/graphql/reconciliation.ts b/apps/web/src/lib/queries/soup/graphql/reconciliation.ts
new file mode 100644
index 0000000000..4fa37c9ef3
--- /dev/null
+++ b/apps/web/src/lib/queries/soup/graphql/reconciliation.ts
@@ -0,0 +1,44 @@
+import { MAX_RECONCILIATION_BASELINE } from '@app/lib/graphql-cache/protocol';
+import type { GraphqlSoupItem } from '@service-storage/graphql-soup';
+
+/** Normalized identity, shared by every Soup entity variant. */
+export const soupItemKey = (item: GraphqlSoupItem): string =>
+ `${item.__typename}:${item.id}`;
+
+/** Same-query server membership and sort evidence; filtering stays in Rust. */
+export function soupReconciliationBaseline(
+ records: readonly GraphqlSoupItem[],
+ sortMethod: 'CREATED_AT' | 'UPDATED_AT'
+): Array<{ key: string; sortTimestamp: string }> | undefined {
+ const entries = new Map<string, { key: string; sortTimestamp: string }>();
+ for (const record of records) {
+ const sortTimestamp =
+ sortMethod === 'CREATED_AT'
+ ? 'createdAt' in record && record.createdAt
+ : 'updatedAt' in record && record.updatedAt;
+ // Do not silently drop baseline rows whose sort evidence is unavailable.
+ if (typeof sortTimestamp !== 'string') return undefined;
+ const key = soupItemKey(record);
+ entries.set(key, { key, sortTimestamp });
+ }
+ return entries.size <= MAX_RECONCILIATION_BASELINE
+ ? [...entries.values()]
+ : undefined;
+}
+
+/** Materialize only reconciled survivors, preserving baseline display data if
+ * a selected record is incomplete. Unrenderable new candidates are omitted. */
+export function materializeReconciledSoup(
+ keys: readonly string[],
+ selected: readonly { recordKey: string; record: GraphqlSoupItem }[],
+ baseline: readonly GraphqlSoupItem[]
+): GraphqlSoupItem[] {
+ const records = new Map(
+ baseline.map((record) => [soupItemKey(record), record])
+ );
+ for (const { recordKey, record } of selected) records.set(recordKey, record);
+ return [...new Set(keys)].flatMap((key) => {
+ const record = records.get(key);
+ return record ? [record] : [];
+ });
+}
diff --git a/crates/client/cache-core/src/engine.rs b/crates/client/cache-core/src/engine.rs
index fb3ce4908c..6547176388 100644
--- a/crates/client/cache-core/src/engine.rs
+++ b/crates/client/cache-core/src/engine.rs
@@ -20,6 +20,9 @@ use crate::normalize::{
DependencyCompleteness, NormalizeError, RecordUpdates, normalize, normalize_with_dependencies,
project_hydration_response,
};
+use crate::predicate::reconciliation::{
+ MAX_RECONCILIATION_BASELINE, PredicateBaselineEntry, PredicateReconciliation,
+};
use crate::predicate::{
OptimisticShadowReconciliation, OptimisticUpsertReconciliation, PredicateIndexStorage,
PredicateQueryResult, ProjectionMutation, ProjectionMutationLayer, ProjectionState,
@@ -2414,6 +2417,27 @@ impl<S: PredicateIndexStorage> Engine<S> {
Ok(self.revisioned(affected))
}
+ /// Reconcile server-page membership and local candidates at one engine revision.
+ pub async fn reconcile_predicate_index(
+ &mut self,
+ query: &ValidatedIndexQuery,
+ baseline: &[PredicateBaselineEntry],
+ ) -> Result<Revisioned<PredicateReconciliation>, EngineError<S::Error>> {
+ if baseline.len() > MAX_RECONCILIATION_BASELINE {
+ return Err(RecordSelectionError::TooManyKeys {
+ count: baseline.len(),
+ max: MAX_RECONCILIATION_BASELINE,
+ }
+ .into());
+ }
+ let value = self
+ .storage
+ .reconcile_predicate_index(query, baseline)
+ .await
+ .map_err(EngineError::Storage)?;
+ Ok(self.revisioned(value))
+ }
+
/// Execute a complete generic exact-index query over authoritative and optimistic projections.
pub async fn query_predicate_index(
&mut self,
diff --git a/crates/client/cache-core/src/predicate.rs b/crates/client/cache-core/src/predicate.rs
index 07ce13802b..5cfc11229f 100644
--- a/crates/client/cache-core/src/predicate.rs
+++ b/crates/client/cache-core/src/predicate.rs
@@ -1,5 +1,7 @@
//! Generic predicate-projection lifecycle and exact-query storage port.
+pub mod reconciliation;
+
use crate::{
queue::{MutationId, MutationQueueSnapshot},
store::Storage,
@@ -645,6 +647,14 @@ pub trait PredicateIndexStorage: Storage {
projection_keys: &[RecordKey],
) -> impl Future<Output = Result<(), Self::Error>> + MaybeSend;
+ /// Reconcile bounded server membership with complete local candidates.
+ /// Unlike an exact query, unrelated incomplete projections do not block it.
+ fn reconcile_predicate_index(
+ &self,
+ query: &ValidatedIndexQuery,
+ baseline: &[reconciliation::PredicateBaselineEntry],
+ ) -> impl Future<Output = Result<reconciliation::PredicateReconciliation, Self::Error>> + MaybeSend;
+
/// Execute a validated generic query or report an incomplete local scope.
fn query_predicate_index(
&self,
diff --git a/crates/client/cache-core/src/predicate/reconciliation.rs b/crates/client/cache-core/src/predicate/reconciliation.rs
new file mode 100644
index 0000000000..52056e6471
--- /dev/null
+++ b/crates/client/cache-core/src/predicate/reconciliation.rs
@@ -0,0 +1,166 @@
+//! Server-baseline reconciliation over known local predicate facts.
+
+use super::ProjectionState;
+use predicate_index::{
+ EffectiveOptimisticProjection, IndexDocument, OptimisticProjectionState, RecordKey,
+ ReferenceHit, SortDirection, ValidatedIndexQuery,
+};
+use std::collections::{HashMap, HashSet};
+
+/// Maximum number of server-page rows reconciled in one request.
+pub const MAX_RECONCILIATION_BASELINE: usize = 5_000;
+
+/// Positive server membership evidence, scoped by the caller to this exact query.
+#[derive(Debug, Clone)]
+pub struct PredicateBaselineEntry {
+ /// Normalized record key.
+ pub record_key: RecordKey,
+ /// Previous sort value, used only while current local facts are unknown.
+ pub sort_value: i64,
+}
+
+/// Best-effort local reconciliation, not an exhaustive collection or local page.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub struct PredicateReconciliation {
+ /// Deduplicated sorted baseline survivors and bounded matching candidates.
+ pub keys: Vec<RecordKey>,
+ /// Baseline survivors whose current membership could not be evaluated.
+ pub retained_keys: Vec<RecordKey>,
+ /// Whether a query-relevant optimistic shadow was observed.
+ pub optimistic: bool,
+}
+
+/// Current record-local evidence. Unknown is never treated as a non-match.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum PredicateMembership {
+ /// Confirmed membership and its current sort value.
+ Match(i64),
+ /// Confirmed non-membership, including deleted records.
+ NonMatch,
+ /// Insufficient or uncertain projection facts.
+ Unknown,
+}
+
+/// Classify one effective record without applying the candidate page limit.
+/// A shadow always suppresses authority, including uncertain and deleted shadows.
+pub fn predicate_membership(
+ query: &ValidatedIndexQuery,
+ authority: Option<&ProjectionState>,
+ shadow: Option<&EffectiveOptimisticProjection>,
+ record_present: bool,
+) -> PredicateMembership {
+ let document = if let Some(shadow) = shadow {
+ if matches!(shadow.state, OptimisticProjectionState::Deleted { .. }) {
+ return PredicateMembership::NonMatch;
+ }
+ if query.includes_scope(shadow.state.profile(), shadow.state.partition())
+ && query
+ .dependent_attributes(shadow.state.partition())
+ .iter()
+ .any(|attribute| shadow.uncertainty.affects(attribute))
+ {
+ return PredicateMembership::Unknown;
+ }
+ match &shadow.state {
+ OptimisticProjectionState::Complete(document) => document,
+ _ => return PredicateMembership::Unknown,
+ }
+ } else {
+ // Baselines came from normalized server pages. A removed durable record
+ // must not be resurrected from that old membership evidence.
+ if !record_present {
+ return PredicateMembership::NonMatch;
+ }
+ match authority {
+ Some(ProjectionState::Complete(document)) => document,
+ _ => return PredicateMembership::Unknown,
+ }
+ };
+ document_membership(query, document)
+}
+
+fn document_membership(
+ query: &ValidatedIndexQuery,
+ document: &IndexDocument,
+) -> PredicateMembership {
+ let descriptor = query.as_query();
+ if document.profile != descriptor.profile {
+ return PredicateMembership::Unknown;
+ }
+ let Some(partition) = descriptor
+ .partitions
+ .iter()
+ .find(|partition| partition.partition == document.partition)
+ else {
+ return PredicateMembership::NonMatch;
+ };
+ if !document.matches(&partition.predicate) {
+ return PredicateMembership::NonMatch;
+ }
+ match document
+ .sort_facts
+ .iter()
+ .find(|fact| fact.attribute == descriptor.sort_attribute)
+ {
+ Some(fact) => PredicateMembership::Match(fact.value),
+ None => PredicateMembership::Unknown,
+ }
+}
+
+/// Merge already bounded candidates with independently evaluated baseline rows.
+/// Baseline matches outside the candidate limit survive; unknown rows retain
+/// their prior sort values. This result deliberately has no pagination cursor.
+pub fn reconcile_predicate_baseline(
+ query: &ValidatedIndexQuery,
+ baseline: &[PredicateBaselineEntry],
+ membership: impl IntoIterator<Item = PredicateMembership>,
+ candidates: Vec<ReferenceHit>,
+ optimistic: bool,
+) -> PredicateReconciliation {
+ let mut hits: HashMap<_, _> = candidates
+ .into_iter()
+ .map(|hit| (hit.record_key, hit.sort_value))
+ .collect();
+ let mut retained = HashSet::new();
+ for (entry, membership) in baseline.iter().zip(membership) {
+ match membership {
+ PredicateMembership::Match(value) => {
+ hits.insert(entry.record_key.clone(), value);
+ retained.remove(&entry.record_key);
+ }
+ PredicateMembership::NonMatch => {
+ hits.remove(&entry.record_key);
+ retained.remove(&entry.record_key);
+ }
+ PredicateMembership::Unknown => {
+ hits.insert(entry.record_key.clone(), entry.sort_value);
+ retained.insert(entry.record_key.clone());
+ }
+ }
+ }
+ let descriptor = query.as_query();
+ let mut hits = hits.into_iter().collect::<Vec<_>>();
+ hits.sort_by(|(left_key, left_sort), (right_key, right_sort)| {
+ let sort = left_sort.cmp(right_sort);
+ let tie = left_key.cmp(right_key);
+ let directed = |order: std::cmp::Ordering, direction| match direction {
+ SortDirection::Asc => order,
+ SortDirection::Desc => order.reverse(),
+ };
+ directed(sort, descriptor.sort_direction)
+ .then_with(|| directed(tie, descriptor.tie_break_direction))
+ });
+ let keys: Vec<_> = hits.into_iter().map(|(key, _)| key).collect();
+ PredicateReconciliation {
+ retained_keys: keys
+ .iter()
+ .filter(|key| retained.contains(*key))
+ .cloned()
+ .collect(),
+ keys,
+ optimistic,
+ }
+}
+
+#[cfg(test)]
+mod test;
diff --git a/crates/client/cache-core/src/predicate/reconciliation/test.rs b/crates/client/cache-core/src/predicate/reconciliation/test.rs
new file mode 100644
index 0000000000..cce9d4eb42
--- /dev/null
+++ b/crates/client/cache-core/src/predicate/reconciliation/test.rs
@@ -0,0 +1,193 @@
+use super::*;
+use crate::predicate::ProjectionIncompleteKind;
+use predicate_index::{
+ ExactFact, ExactValue, IndexQuery, IntegerFact, OptimisticUncertainty, PartitionPredicate,
+ PredicateExpr, Profile, Token,
+};
+
+fn token(value: &str) -> Token {
+ Token::new(value).unwrap()
+}
+fn key(value: &str) -> RecordKey {
+ RecordKey::new(format!("Thing:{value}")).unwrap()
+}
+fn query() -> ValidatedIndexQuery {
+ ValidatedIndexQuery::new(IndexQuery {
+ profile: Profile::new(token("test")),
+ partitions: vec![PartitionPredicate {
+ partition: token("thing"),
+ predicate: PredicateExpr::Exact {
+ attribute: token("owner"),
+ value: ExactValue::utf8("me").unwrap(),
+ },
+ }],
+ sort_attribute: token("updated"),
+ sort_direction: SortDirection::Desc,
+ tie_break_direction: SortDirection::Desc,
+ limit: 1,
+ })
+ .unwrap()
+}
+fn document() -> IndexDocument {
+ IndexDocument {
+ record_key: key("a"),
+ profile: query().as_query().profile.clone(),
+ partition: token("thing"),
+ exact_facts: vec![ExactFact {
+ attribute: token("owner"),
+ value: ExactValue::utf8("me").unwrap(),
+ }],
+ integer_facts: vec![],
+ sort_facts: vec![IntegerFact {
+ attribute: token("updated"),
+ value: 10,
+ }],
+ }
+}
+
+#[test]
+fn unknown_is_distinct_from_nonmatch_and_deletion() {
+ let q = query();
+ let authority = ProjectionState::Complete(document());
+ assert_eq!(
+ predicate_membership(&q, Some(&authority), None, true),
+ PredicateMembership::Match(10)
+ );
+ assert_eq!(
+ predicate_membership(&q, Some(&authority), None, false),
+ PredicateMembership::NonMatch
+ );
+ assert_eq!(
+ predicate_membership(&q, None, None, true),
+ PredicateMembership::Unknown
+ );
+ let incomplete = ProjectionState::Incomplete {
+ record_key: key("a"),
+ profile: q.as_query().profile.clone(),
+ partition: token("thing"),
+ kind: ProjectionIncompleteKind::Missing,
+ };
+ assert_eq!(
+ predicate_membership(&q, Some(&incomplete), None, true),
+ PredicateMembership::Unknown
+ );
+ let mut changed = document();
+ changed.exact_facts.clear();
+ assert_eq!(
+ predicate_membership(&q, Some(&ProjectionState::Complete(changed)), None, true),
+ PredicateMembership::NonMatch
+ );
+}
+
+#[test]
+fn shadows_never_resurrect_authority_and_only_relevant_uncertainty_blocks() {
+ let q = query();
+ let authority = ProjectionState::Complete(document());
+ let mut shadow = EffectiveOptimisticProjection {
+ owner: 1,
+ state: OptimisticProjectionState::Complete(document()),
+ uncertainty: OptimisticUncertainty::default(),
+ };
+ shadow.uncertainty.mark(&[token("name")]);
+ assert_eq!(
+ predicate_membership(&q, Some(&authority), Some(&shadow), true),
+ PredicateMembership::Match(10)
+ );
+ shadow.uncertainty.mark(&[token("owner")]);
+ assert_eq!(
+ predicate_membership(&q, Some(&authority), Some(&shadow), true),
+ PredicateMembership::Unknown
+ );
+ shadow.state = OptimisticProjectionState::Deleted {
+ record_key: key("a"),
+ profile: q.as_query().profile.clone(),
+ partition: token("thing"),
+ };
+ assert_eq!(
+ predicate_membership(&q, Some(&authority), Some(&shadow), true),
+ PredicateMembership::NonMatch
+ );
+}
+
+#[test]
+fn baseline_is_not_truncated_by_candidate_limit_and_unknowns_keep_sort_evidence() {
+ let baseline = vec![
+ PredicateBaselineEntry {
+ record_key: key("a"),
+ sort_value: 20,
+ },
+ PredicateBaselineEntry {
+ record_key: key("b"),
+ sort_value: 15,
+ },
+ PredicateBaselineEntry {
+ record_key: key("c"),
+ sort_value: 40,
+ },
+ ];
+ let candidates = vec![ReferenceHit {
+ record_key: key("d"),
+ sort_value: 30,
+ }];
+ let result = reconcile_predicate_baseline(
+ &query(),
+ &baseline,
+ [
+ PredicateMembership::Match(10),
+ PredicateMembership::Unknown,
+ PredicateMembership::NonMatch,
+ ],
+ candidates,
+ false,
+ );
+ assert_eq!(result.keys, vec![key("d"), key("b"), key("a")]);
+ assert_eq!(result.retained_keys, vec![key("b")]);
+}
+
+#[test]
+fn in_memory_reconciles_missing_stubs_without_changing_exact_api() {
+ use crate::{
+ predicate::{PredicateIndexStorage, PredicateQueryResult, ProjectionMutation},
+ store::{InMemoryStorage, Storage},
+ value::{EntityKey, Record},
+ };
+ // These futures are immediately ready with the in-memory adapter.
+ pollster::block_on(async {
+ let mut storage = InMemoryStorage::default();
+ let doc = document();
+ storage
+ .put_batch_with_projections(
+ vec![
+ (EntityKey("Thing:a".into()), Record::default()),
+ (EntityKey("Thing:stub".into()), Record::default()),
+ ],
+ vec![
+ ProjectionMutation::Replace(doc),
+ ProjectionMutation::MarkIncomplete {
+ record_key: key("stub"),
+ profile: query().as_query().profile.clone(),
+ partition: token("thing"),
+ kind: ProjectionIncompleteKind::Missing,
+ },
+ ],
+ )
+ .await
+ .unwrap();
+ assert_eq!(
+ storage.query_predicate_index(&query()).await.unwrap(),
+ PredicateQueryResult::Incomplete
+ );
+ let result = storage
+ .reconcile_predicate_index(
+ &query(),
+ &[PredicateBaselineEntry {
+ record_key: key("stub"),
+ sort_value: 15,
+ }],
+ )
+ .await
+ .unwrap();
+ assert_eq!(result.keys, vec![key("stub"), key("a")]);
+ assert_eq!(result.retained_keys, vec![key("stub")]);
+ });
+}
diff --git a/crates/client/cache-core/src/store.rs b/crates/client/cache-core/src/store.rs
index 84f9a94f9e..7106f06e61 100644
--- a/crates/client/cache-core/src/store.rs
+++ b/crates/client/cache-core/src/store.rs
@@ -6,6 +6,10 @@
//! wasm — wasm futures aren't
//! `Send`.
+use crate::predicate::reconciliation::{
+ PredicateBaselineEntry, PredicateMembership, PredicateReconciliation, predicate_membership,
+ reconcile_predicate_baseline,
+};
use crate::predicate::{
OptimisticShadowReconciliation, OptimisticUpsertReconciliation, PredicateIndexStorage,
PredicateQueryResult, ProjectionMutation, ProjectionState, StagedOptimisticProjection,
@@ -738,6 +742,62 @@ impl Storage for InMemoryStorage {
}
impl PredicateIndexStorage for InMemoryStorage {
+ async fn reconcile_predicate_index(
+ &self,
+ query: &predicate_index::ValidatedIndexQuery,
+ baseline: &[PredicateBaselineEntry],
+ ) -> Result<PredicateReconciliation, Self::Error> {
+ let present = |key: &PredicateRecordKey| {
+ self.records
+ .contains_key(&EntityKey(key.as_str().to_owned().into()))
+ };
+ let membership = |key: &PredicateRecordKey| {
+ predicate_membership(
+ query,
+ self.projections.get(key),
+ self.optimistic_projections.get(key),
+ present(key),
+ )
+ };
+ let keys: HashSet<_> = self
+ .projections
+ .keys()
+ .chain(self.optimistic_projections.keys())
+ .collect();
+ let documents = keys
+ .into_iter()
+ .filter_map(|key| {
+ if !matches!(membership(key), PredicateMembership::Match(_)) {
+ return None;
+ }
+ match self.optimistic_projections.get(key) {
+ Some(projection) => match &projection.state {
+ predicate_index::OptimisticProjectionState::Complete(document) => {
+ Some(document.clone())
+ }
+ _ => None,
+ },
+ None => match self.projections.get(key) {
+ Some(ProjectionState::Complete(document)) => Some(document.clone()),
+ _ => None,
+ },
+ }
+ })
+ .collect::<Vec<_>>();
+ Ok(reconcile_predicate_baseline(
+ query,
+ baseline,
+ baseline.iter().map(|entry| membership(&entry.record_key)),
+ evaluate_reference(query, &documents),
+ self.optimistic_projections.iter().any(|(key, shadow)| {
+ query.includes_scope(shadow.state.profile(), shadow.state.partition())
+ || self.projections.get(key).is_some_and(|authority| {
+ query.includes_scope(authority.profile(), authority.partition())
+ })
+ }),
+ ))
+ }
+
async fn delete_batch_with_projections(
&mut self,
keys: &[EntityKey<'static>],
diff --git a/crates/client/cache-turso/src/storage.rs b/crates/client/cache-turso/src/storage.rs
index 0eaff1da49..c472c1b184 100644
--- a/crates/client/cache-turso/src/storage.rs
+++ b/crates/client/cache-turso/src/storage.rs
@@ -4,6 +4,10 @@ use crate::{PhysicalResetReason, TursoStorageError};
use cache_core::codec::{
cache_namespace, decode_record, decode_record_updates, encode_record, encode_record_updates,
};
+use cache_core::predicate::reconciliation::{
+ PredicateBaselineEntry, PredicateReconciliation, predicate_membership,
+ reconcile_predicate_baseline,
+};
use cache_core::predicate::{
OptimisticShadowReconciliation, OptimisticUpsertReconciliation, PredicateIndexStorage,
PredicateQueryResult, ProjectionIncompleteKind, ProjectionMutation, ProjectionState,
@@ -1414,6 +1418,62 @@ impl Storage for TursoStorage {
}
impl PredicateIndexStorage for TursoStorage {
+ async fn reconcile_predicate_index(
+ &self,
+ query: &ValidatedIndexQuery,
+ baseline: &[PredicateBaselineEntry],
+ ) -> Result<PredicateReconciliation, Self::Error> {
+ self.require_healthy()?;
+ let connection = self.connection();
+ let result = driver::read_transaction(&connection, || {
+ let optimistic = optimistic_query_status(&connection, query)?;
+ // Incomplete authority and unknown shadows are not candidates, but
+ // they must not prevent unrelated known-good rows from updating.
+ let (sql, parameters) =
+ compile_predicate_selection(query, &optimistic.uncertain_ids, true);
+ let candidates = driver::query(&connection, &sql, parameters)?
+ .into_iter()
+ .map(|row| {
+ if row.len() != 2 {
+ return Err(invariant());
+ }
+ Ok(predicate_index::ReferenceHit {
+ record_key: PredicateRecordKey::new(required_text(&row, 0)?)
+ .map_err(|_| invariant())?,
+ sort_value: required_i64(&row, 1)?,
+ })
+ })
+ .collect::<Result<Vec<_>, TursoStorageError>>()?;
+ let keys: Vec<_> = baseline
+ .iter()
+ .map(|entry| entry.record_key.clone())
+ .collect();
+ let authority = load_projection_states(&connection, &keys)?;
+ let shadows = load_optimistic_projections(&connection, &keys)?;
+ let present = present_predicate_records(&connection, &keys)?;
+ let membership =
+ keys.iter()
+ .zip(&authority)
+ .zip(&shadows)
+ .map(|((key, authority), shadow)| {
+ predicate_membership(
+ query,
+ authority.as_ref(),
+ shadow.as_ref(),
+ present.contains(key),
+ )
+ });
+ Ok(reconcile_predicate_baseline(
+ query,
+ baseline,
+ membership,
+ candidates,
+ optimistic.has_shadow,
+ ))
+ });
+ self.latch_result(result)
+ }
+
async fn delete_batch_with_projections(
&mut self,
keys: &[EntityKey<'static>],
@@ -2270,6 +2330,40 @@ fn load_index_documents(
Ok(keys.iter().map(|key| documents.get(key).cloned()).collect())
}
+fn present_predicate_records(
+ connection: &Arc<Connection>,
+ keys: &[PredicateRecordKey],
+) -> Result<BTreeSet<PredicateRecordKey>, TursoStorageError> {
+ let mut present = BTreeSet::new();
+ // Point lookups through records' composite primary key; never scan/decode
+ // normalized blobs just to distinguish an unknown baseline from a deletion.
+ for batch in keys.chunks(500) {
+ let mut parameters = Vec::with_capacity(batch.len() * 2);
+ for key in batch {
+ let parsed = RecordKey::from_entity(&EntityKey(key.as_str().into()))?;
+ parameters.extend([text(&parsed.typename), text(&parsed.id)]);
+ }
+ let values = vec!["(?, ?)"; batch.len()].join(", ");
+ let sql = format!(
+ "WITH requested(typename, id) AS (VALUES {values}) SELECT r.__typename, r.id FROM requested AS q JOIN records AS r ON r.__typename = q.typename AND r.id = q.id"
+ );
+ for row in driver::query(connection, &sql, parameters)? {
+ if row.len() != 2 {
+ return Err(invariant());
+ }
+ present.insert(
+ PredicateRecordKey::new(format!(
+ "{}:{}",
+ required_text(&row, 0)?,
+ required_text(&row, 1)?
+ ))
+ .map_err(|_| invariant())?,
+ );
+ }
+ }
+ Ok(present)
+}
+
fn predicate_scope_is_incomplete(
connection: &Arc<Connection>,
query: &ValidatedIndexQuery,
@@ -2292,10 +2386,11 @@ fn predicate_scope_is_incomplete(
Ok(!driver::query(connection, &sql, parameters)?.is_empty())
}
-#[derive(Clone, Copy, Debug, Default)]
+#[derive(Clone, Debug, Default)]
struct OptimisticQueryStatus {
has_shadow: bool,
incomplete: bool,
+ uncertain_ids: Vec<i64>,
}
fn optimistic_query_status(
@@ -2329,6 +2424,7 @@ fn optimistic_query_status(
let mut status = OptimisticQueryStatus {
has_shadow: !rows.is_empty(),
incomplete: false,
+ uncertain_ids: Vec::new(),
};
let mut uncertainty: HashMap<(i64, Token), Vec<String>> = HashMap::new();
for row in rows {
@@ -2341,7 +2437,6 @@ fn optimistic_query_status(
let state = OptimisticIndexDocumentState::try_from(required_i64(&row, 3)?)?;
if current_scope && state == OptimisticIndexDocumentState::Incomplete {
status.incomplete = true;
- return Ok(status);
}
if current_scope && let Some(attribute) = nullable_text(&row, 4)? {
uncertainty
@@ -2350,7 +2445,7 @@ fn optimistic_query_status(
.push(attribute);
}
}
- for ((_, partition), attributes) in uncertainty {
+ for ((id, partition), attributes) in uncertainty {
let uncertainty = parse_optimistic_uncertainty(attributes)?;
if query
.dependent_attributes(&partition)
@@ -2358,15 +2453,24 @@ fn optimistic_query_status(
.any(|attribute| uncertainty.affects(attribute))
{
status.incomplete = true;
- break;
+ status.uncertain_ids.push(id);
}
}
+ status.uncertain_ids.sort_unstable();
Ok(status)
}
fn compile_predicate_sql(query: &ValidatedIndexQuery) -> (String, Vec<Value>) {
+ compile_predicate_selection(query, &[], false)
+}
+
+fn compile_predicate_selection(
+ query: &ValidatedIndexQuery,
+ excluded_optimistic: &[i64],
+ include_sort: bool,
+) -> (String, Vec<Value>) {
let descriptor = query.as_query();
- let mut compiler = SqlPredicateCompiler::new();
+ let mut compiler = SqlPredicateCompiler::new(excluded_optimistic);
let roots = descriptor
.partitions
.iter()
@@ -2408,8 +2512,9 @@ fn compile_predicate_sql(query: &ValidatedIndexQuery) -> (String, Vec<Value>) {
SortDirection::Asc => "ASC",
SortDirection::Desc => "DESC",
};
+ let sort_selection = if include_sort { ", s.value" } else { "" };
let sql = format!(
- "WITH {} SELECT d.record_key FROM {matches} AS m JOIN effective_documents AS d ON d.source = m.source AND d.document_id = m.document_id JOIN {effective_sort} AS s ON s.source = m.source AND s.document_id = m.document_id ORDER BY s.value {sort_direction}, d.record_key {tie_direction} LIMIT ?",
+ "WITH {} SELECT d.record_key{sort_selection} FROM {matches} AS m JOIN effective_documents AS d ON d.source = m.source AND d.document_id = m.document_id JOIN {effective_sort} AS s ON s.source = m.source AND s.document_id = m.document_id ORDER BY s.value {sort_direction}, d.record_key {tie_direction} LIMIT ?",
compiler.ctes.join(", ")
);
(sql, compiler.parameters)
@@ -2422,12 +2527,24 @@ struct SqlPredicateCompiler {
}
impl SqlPredicateCompiler {
- fn new() -> Self {
+ fn new(excluded_optimistic: &[i64]) -> Self {
+ let exclusion = if excluded_optimistic.is_empty() {
+ String::new()
+ } else {
+ format!(
+ " AND o.id NOT IN ({})",
+ sql_placeholders(excluded_optimistic.len())
+ )
+ };
Self {
- ctes: vec![
- "effective_documents(source, document_id, record_key, profile, partition) AS (SELECT 0, d.id, d.record_key, d.profile, d.partition FROM index_documents AS d WHERE d.state = 0 AND NOT EXISTS (SELECT 1 FROM optimistic_index_documents AS o WHERE o.record_key = d.record_key) UNION ALL SELECT 1, o.id, o.record_key, o.profile, o.partition FROM optimistic_index_documents AS o WHERE o.state = 0)".to_owned(),
- ],
- parameters: Vec::new(),
+ ctes: vec![format!(
+ "effective_documents(source, document_id, record_key, profile, partition) AS (SELECT 0, d.id, d.record_key, d.profile, d.partition FROM index_documents AS d WHERE d.state = 0 AND NOT EXISTS (SELECT 1 FROM optimistic_index_documents AS o WHERE o.record_key = d.record_key) UNION ALL SELECT 1, o.id, o.record_key, o.profile, o.partition FROM optimistic_index_documents AS o WHERE o.state = 0{exclusion})"
+ )],
+ parameters: excluded_optimistic
+ .iter()
+ .copied()
+ .map(Value::from_i64)
+ .collect(),
next_id: 0,
}
}
diff --git a/crates/client/cache-turso/tests/predicate.rs b/crates/client/cache-turso/tests/predicate.rs
index e8ef437dcf..c427eb03ec 100644
--- a/crates/client/cache-turso/tests/predicate.rs
+++ b/crates/client/cache-turso/tests/predicate.rs
@@ -1,5 +1,8 @@
#![cfg(not(target_arch = "wasm32"))]
+#[path = "predicate/reconciliation.rs"]
+mod reconciliation;
+
use cache_core::{
engine::{BeginOptimisticWrite, Engine},
predicate::{
diff --git a/crates/client/cache-turso/tests/predicate/reconciliation.rs b/crates/client/cache-turso/tests/predicate/reconciliation.rs
new file mode 100644
index 0000000000..bdb92ff7e8
--- /dev/null
+++ b/crates/client/cache-turso/tests/predicate/reconciliation.rs
@@ -0,0 +1,227 @@
+use super::*;
+use cache_core::predicate::reconciliation::{PredicateBaselineEntry, PredicateReconciliation};
+
+fn baseline(key: &str, sort_value: i64) -> PredicateBaselineEntry {
+ PredicateBaselineEntry {
+ record_key: RecordKey::new(key).unwrap(),
+ sort_value,
+ }
+}
+
+#[test]
+fn hybrid_baseline_ignores_unrelated_stubs_and_preserves_rows_beyond_candidate_limit() {
+ pollster::block_on(async {
+ let mut storage = TursoStorage::open_in_memory("reconciliation").unwrap();
+ let a = document("GraphqlSoupDocument:a", "owner-1", 10, None);
+ let b = document("GraphqlSoupDocument:b", "owner-2", 15, None);
+ let c = document("GraphqlSoupDocument:c", "owner-1", 25, None);
+ let stub = RecordKey::new("GraphqlSoupDocument:stub").unwrap();
+ let records = [&a.record_key, &b.record_key, &c.record_key, &stub]
+ .into_iter()
+ .map(|key| (EntityKey(key.as_str().to_owned().into()), Record::default()))
+ .collect();
+ storage
+ .put_batch_with_projections(
+ records,
+ vec![
+ ProjectionMutation::Replace(a.clone()),
+ ProjectionMutation::Replace(b.clone()),
+ ProjectionMutation::Replace(c.clone()),
+ ProjectionMutation::MarkIncomplete {
+ record_key: stub.clone(),
+ profile: profile(),
+ partition: token("document"),
+ kind: ProjectionIncompleteKind::Missing,
+ },
+ ],
+ )
+ .await
+ .unwrap();
+ let mut descriptor = query().as_query().clone();
+ descriptor.limit = 1;
+ let query = ValidatedIndexQuery::new(descriptor).unwrap();
+ assert_eq!(
+ storage.query_predicate_index(&query).await.unwrap(),
+ PredicateQueryResult::Incomplete
+ );
+ let baseline = vec![
+ baseline(a.record_key.as_str(), 1),
+ baseline(b.record_key.as_str(), 40),
+ baseline(stub.as_str(), 20),
+ baseline("GraphqlSoupDocument:deleted", 100),
+ ];
+ let result = storage
+ .reconcile_predicate_index(&query, &baseline)
+ .await
+ .unwrap();
+ assert_eq!(
+ result,
+ PredicateReconciliation {
+ keys: vec![c.record_key, stub.clone(), a.record_key],
+ retained_keys: vec![stub],
+ optimistic: false
+ }
+ );
+ });
+}
+
+#[test]
+fn unknown_shadow_does_not_block_candidates_or_reveal_stale_authority() {
+ pollster::block_on(async {
+ let storage = TursoStorage::open_in_memory("reconciliation-shadow").unwrap();
+ let mut engine = Engine::new(storage);
+ let a = document("GraphqlSoupDocument:a", "owner-1", 10, None);
+ let b = document("GraphqlSoupDocument:b", "owner-1", 25, None);
+ engine
+ .put_records_with_projections(
+ None,
+ [&a, &b]
+ .into_iter()
+ .map(|doc| {
+ (
+ EntityKey(doc.record_key.as_str().to_owned().into()),
+ Record::default(),
+ )
+ })
+ .collect(),
+ vec![
+ ProjectionMutation::Replace(a.clone()),
+ ProjectionMutation::Replace(b.clone()),
+ ],
+ )
+ .await
+ .unwrap();
+ begin_optimistic_projection(
+ &mut engine,
+ vec![OptimisticProjectionMutation::Unknown {
+ record_key: b.record_key.clone(),
+ profile: profile(),
+ partition: token("document"),
+ affected_attributes: vec![token("owner")],
+ }],
+ )
+ .await;
+ assert_eq!(
+ engine.query_predicate_index(&query()).await.unwrap(),
+ PredicateQueryResult::Incomplete
+ );
+ let without_baseline = engine
+ .reconcile_predicate_index(&query(), &[])
+ .await
+ .unwrap();
+ assert_eq!(without_baseline.value.keys, vec![a.record_key.clone()]);
+ let with_baseline = engine
+ .reconcile_predicate_index(&query(), &[baseline(b.record_key.as_str(), 5)])
+ .await
+ .unwrap();
+ assert_eq!(
+ with_baseline.value.keys,
+ vec![a.record_key.clone(), b.record_key.clone()]
+ );
+ assert_eq!(
+ with_baseline.value.retained_keys,
+ vec![b.record_key.clone()]
+ );
+ begin_optimistic_projection(
+ &mut engine,
+ vec![OptimisticProjectionMutation::Delete {
+ record_key: b.record_key.clone(),
+ profile: profile(),
+ partition: token("document"),
+ }],
+ )
+ .await;
+ assert_eq!(
+ engine
+ .reconcile_predicate_index(&query(), &[baseline(b.record_key.as_str(), 5)])
+ .await
+ .unwrap()
+ .value
+ .keys,
+ vec![a.record_key]
+ );
+ });
+}
+
+#[test]
+fn all_except_uncertainty_and_irrelevant_fields_do_not_hide_known_matches() {
+ pollster::block_on(async {
+ let storage = TursoStorage::open_in_memory("reconciliation-certain-fields").unwrap();
+ let mut engine = Engine::new(storage);
+ let a = document("GraphqlSoupDocument:a", "owner-1", 10, None);
+ engine
+ .put_records_with_projections(
+ None,
+ vec![(
+ EntityKey(a.record_key.as_str().to_owned().into()),
+ Record::default(),
+ )],
+ vec![ProjectionMutation::Replace(a.clone())],
+ )
+ .await
+ .unwrap();
+ begin_optimistic_projection(
+ &mut engine,
+ vec![OptimisticProjectionMutation::Unknown {
+ record_key: a.record_key.clone(),
+ profile: profile(),
+ partition: token("document"),
+ affected_attributes: vec![token("name")],
+ }],
+ )
+ .await;
+ assert_eq!(
+ engine
+ .reconcile_predicate_index(&query(), &[])
+ .await
+ .unwrap()
+ .value
+ .keys,
+ vec![a.record_key.clone()]
+ );
+ begin_optimistic_projection(
+ &mut engine,
+ vec![
+ OptimisticProjectionMutation::Unknown {
+ record_key: a.record_key.clone(),
+ profile: profile(),
+ partition: token("document"),
+ affected_attributes: vec![],
+ },
+ OptimisticProjectionMutation::Patch {
+ record_key: a.record_key.clone(),
+ profile: profile(),
+ partition: token("document"),
+ exact: vec![
+ ExactAttributePatch {
+ attribute: token("owner"),
+ values: vec![ExactValue::utf8("owner-1").unwrap()],
+ },
+ ExactAttributePatch {
+ attribute: token("project-id"),
+ values: vec![],
+ },
+ ],
+ integers: vec![IntegerAttributePatch {
+ attribute: token("updated-at"),
+ values: vec![10],
+ }],
+ sorts: vec![IntegerFact {
+ attribute: token("updated-at"),
+ value: 10,
+ }],
+ },
+ ],
+ )
+ .await;
+ assert_eq!(
+ engine
+ .reconcile_predicate_index(&query(), &[])
+ .await
+ .unwrap()
+ .value
+ .keys,
+ vec![a.record_key]
+ );
+ });
+}
diff --git a/crates/client/cache-wasm/Cargo.toml b/crates/client/cache-wasm/Cargo.toml
index 6795099095..521d608bb6 100644
--- a/crates/client/cache-wasm/Cargo.toml
+++ b/crates/client/cache-wasm/Cargo.toml
@@ -2,7 +2,7 @@
edition = "2024"
name = "cache-wasm"
publish = false
-version = "0.6.4"
+version = "0.6.5"
[features]
default = []
diff --git a/crates/client/cache-wasm/src/shell.rs b/crates/client/cache-wasm/src/shell.rs
index 6178e10875..2ba3bdad8d 100644
--- a/crates/client/cache-wasm/src/shell.rs
+++ b/crates/client/cache-wasm/src/shell.rs
@@ -303,11 +303,26 @@ struct JsEntityFilterRequest {
sort_method: String,
sort_direction: String,
limit: u16,
+ baseline: Option<Vec<JsPredicateBaselineEntry>>,
+}
+
+#[derive(Deserialize)]
+#[serde(rename_all = "camelCase", deny_unknown_fields)]
+struct JsPredicateBaselineEntry {
+ key: String,
+ sort_timestamp: String,
}
#[derive(Serialize)]
#[serde(tag = "kind", rename_all = "kebab-case")]
enum JsEntityFilterResult {
+ Reconciled {
+ revision: String,
+ keys: Vec<String>,
+ #[serde(rename = "retainedKeys")]
+ retained_keys: Vec<String>,
+ optimistic: bool,
+ },
Complete {
revision: String,
keys: Vec<String>,
@@ -991,6 +1006,45 @@ impl CacheEngine {
.map_err(err_js)?;
let result = match outcome {
SoupFilterCompileOutcome::Unsupported => JsEntityFilterResult::Unsupported,
+ SoupFilterCompileOutcome::Supported(query) if request.baseline.is_some() => {
+ let baseline = request.baseline.unwrap();
+ if baseline.len()
+ > cache_core::predicate::reconciliation::MAX_RECONCILIATION_BASELINE
+ {
+ return Err(err_js("reconciliation baseline is too large"));
+ }
+ let baseline = baseline
+ .into_iter()
+ .map(|entry| {
+ soup_filter_cache_adapter::reconciliation_baseline_entry(
+ entry.key,
+ &entry.sort_timestamp,
+ )
+ })
+ .collect::<Result<Vec<_>, _>>()
+ .map_err(err_js)?;
+ let result = state
+ .engine_mut()?
+ .reconcile_predicate_index(&query, &baseline)
+ .await;
+ let result = state.engine_result(result)?;
+ JsEntityFilterResult::Reconciled {
+ revision: result.revision.to_string(),
+ keys: result
+ .value
+ .keys
+ .into_iter()
+ .map(|key| key.as_str().to_owned())
+ .collect(),
+ retained_keys: result
+ .value
+ .retained_keys
+ .into_iter()
+ .map(|key| key.as_str().to_owned())
+ .collect(),
+ optimistic: result.value.optimistic,
+ }
+ }
SoupFilterCompileOutcome::Supported(query) => {
let result = state.engine_mut()?.query_predicate_index(&query).await;
let result = state.engine_result(result)?;
diff --git a/crates/client/cache-wasm/src/shell/test.rs b/crates/client/cache-wasm/src/shell/test.rs
index 27b68e8381..31c257b487 100644
--- a/crates/client/cache-wasm/src/shell/test.rs
+++ b/crates/client/cache-wasm/src/shell/test.rs
@@ -184,6 +184,7 @@ async fn resolved(promise: js_sys::Promise) -> JsValue {
fn empty_js_write_result() -> JsWriteResult {
JsWriteResult {
revision: "0".to_string(),
+ revision_advanced: false,
changed: Vec::new(),
affected_ops: Vec::new(),
reset: false,
@@ -907,6 +908,59 @@ async fn soup_updated_v3_supplements_advance_revision_and_recompute_documents_pr
close_and_destroy(&engine, SCOPE).await;
}
+#[wasm_bindgen_test(async)]
+async fn server_baseline_reconciliation_survives_notification_stubs_and_deletions() {
+ const SCOPE: &str = "cache-wasm-server-baseline-reconciliation";
+ const A: &str = "00000000-0000-0000-0000-000000000001";
+ const B: &str = "00000000-0000-0000-0000-000000000002";
+ const STUB: &str = "00000000-0000-0000-0000-000000000003";
+ let key = |id| format!("GraphqlSoupDocument:{id}");
+ let engine = fresh_engine(SCOPE).await;
+ resolved(engine.write_query(
+ write_context(None),
+ QUERY.into(),
+ Some("Soup".into()),
+ js(serde_json::json!({"input": {"initial": {"limit": 100}}})),
+ js(
+ serde_json::json!({"user": {"id": "macro|[email protected]", "soup": {
+ "nextCursor": null, "items": [{"__typename": "GraphqlSoupDocument", "id": STUB}]
+ }}}),
+ ),
+ None,
+ ))
+ .await;
+ resolved(engine.write_query(
+ write_context(None), SOUP_UPDATES_WITH_PROJECTION_SUBSCRIPTION.into(),
+ Some("SoupUpdatesWithProjection".into()), js(serde_json::json!({})),
+ js(serde_json::json!({"soupUpdates": [
+ {"__typename": "SoupUpdated", "item": projected_document_item(A, "macro|[email protected]", false, None, 10)},
+ {"__typename": "SoupUpdated", "item": projected_document_item(B, "macro|[email protected]", false, None, 30)}
+ ]})), None,
+ )).await;
+ let mut request = documents_preset_filter(
+ serde_json::json!({"literal": {"owner": "macro|[email protected]"}}),
+ );
+ request["limit"] = serde_json::json!(1);
+ let exact: serde_json::Value =
+ from_js(resolved(engine.entity_filter(js(request.clone()))).await);
+ assert_eq!(exact["kind"], "incomplete");
+ request["baseline"] = serde_json::json!([
+ {"key": key(A), "sortTimestamp": "2025-01-01T00:00:00.000010Z"},
+ {"key": key(STUB), "sortTimestamp": "2025-01-01T00:00:00.000020Z"}
+ ]);
+ let result: serde_json::Value =
+ from_js(resolved(engine.entity_filter(js(request.clone()))).await);
+ assert_eq!(
+ result,
+ serde_json::json!({"kind": "reconciled", "revision": "2", "keys": [key(B), key(STUB), key(A)], "retainedKeys": [key(STUB)], "optimistic": false})
+ );
+ resolved(engine.delete_keys(vec![key(A), key(STUB)])).await;
+ let deleted: serde_json::Value = from_js(resolved(engine.entity_filter(js(request))).await);
+ assert_eq!(deleted["keys"], serde_json::json!([key(B)]));
+ assert_eq!(deleted["retainedKeys"], serde_json::json!([]));
+ close_and_destroy(&engine, SCOPE).await;
+}
+
#[wasm_bindgen_test(async)]
async fn soup_updated_v3_recomputes_my_tasks_importance_and_status_locally() {
const SCOPE: &str = "cache-wasm-realtime-my-tasks-v3";
diff --git a/crates/client/cache-wasm/src/shell/test_storage.rs b/crates/client/cache-wasm/src/shell/test_storage.rs
index 3209a54ba1..750d1775c4 100644
--- a/crates/client/cache-wasm/src/shell/test_storage.rs
+++ b/crates/client/cache-wasm/src/shell/test_storage.rs
@@ -215,6 +215,14 @@ impl Storage for BrowserStorage {
}
impl PredicateIndexStorage for BrowserStorage {
+ async fn reconcile_predicate_index(
+ &self,
+ query: &ValidatedIndexQuery,
+ baseline: &[cache_core::predicate::reconciliation::PredicateBaselineEntry],
+ ) -> Result<cache_core::predicate::reconciliation::PredicateReconciliation, Self::Error> {
+ self.inner.reconcile_predicate_index(query, baseline).await
+ }
+
async fn delete_batch_with_projections(
&mut self,
keys: &[EntityKey<'static>],
diff --git a/crates/soup_filter_cache_adapter/src/lib.rs b/crates/soup_filter_cache_adapter/src/lib.rs
index 397a6f36bf..8ab29d2337 100644
--- a/crates/soup_filter_cache_adapter/src/lib.rs
+++ b/crates/soup_filter_cache_adapter/src/lib.rs
@@ -76,6 +76,27 @@ pub fn compile_filter_request(
})
}
+/// Convert server-page sort evidence without losing sub-millisecond precision.
+/// The caller scopes the baseline to the same filter and sort as the request.
+pub fn reconciliation_baseline_entry(
+ record_key: String,
+ sort_timestamp: &str,
+) -> Result<
+ cache_core::predicate::reconciliation::PredicateBaselineEntry,
+ SoupFilterCacheAdapterError,
+> {
+ let record_key = RecordKey::new(record_key)
+ .map_err(|error| SoupFilterCacheAdapterError(error.to_string()))?;
+ let timestamp = chrono::DateTime::parse_from_rfc3339(sort_timestamp)
+ .map_err(|error| SoupFilterCacheAdapterError(error.to_string()))?;
+ Ok(
+ cache_core::predicate::reconciliation::PredicateBaselineEntry {
+ record_key,
+ sort_value: predicate_index::utc_timestamp_micros(timestamp.to_utc()),
+ },
+ )
+}
+
/// Derive authoritative generic projection mutations from a selected GraphQL response.
///
/// Document supplements are decoded only where `cacheProjection` is selected
diff --git a/docs/AGENT_GUIDE/surfaces.md b/docs/AGENT_GUIDE/surfaces.md
index 327ad4c862..e122cdf7fa 100644
--- a/docs/AGENT_GUIDE/surfaces.md
+++ b/docs/AGENT_GUIDE/surfaces.md
@@ -1,5 +1,20 @@
# Other Surfaces
+## Live updates in flat Soup lists
+
+With browser GraphQL caching enabled, locally supported flat lists reconcile their
+loaded server pages with matching cached entities. Complete matching updates can
+appear without a list refetch; confirmed non-matches and explicit deletions disappear.
+Rows whose current predicate facts are unknown retain their previous server membership
+and sort evidence until hydration or a network refresh resolves them. Unrelated
+notification-only cache records do not block other rows' updates.
+
+This is a best-effort display, not proof that every matching entity is cached. Loading
+more still follows the original server cursors and preserves already loaded pages.
+Changing filters or resetting the cache discards prior reconciliation evidence. Grouped
+lists, unsupported filters/sorts, and native/non-cache transports keep their existing
+network behavior.
+
## Inbox — `/app/component/inbox`
Unified triage list (emails, channel messages, task assignments, doc mentions, agent