Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.
TypeScript agentaiai-agentsall-in-onecrmcrm-systememaillinearmcpmessagingnotesnotionofficerustslackslack-alternativestartupstartup-toolstypescriptworkspace
Sean/cache local grouping (#8298)
Sean Ayed3ca22990b5f
diff --git a/apps/web/AGENTS.md b/apps/web/AGENTS.mdindex a44a48fa86..4a4ba6b469 100644--- a/apps/web/AGENTS.md+++ b/apps/web/AGENTS.md@@ -137,6 +137,15 @@ ownership. Cancel, logout, and the ten-minute timeout retire pending callbacks.- Preserve loaded block content on initial access confirmation when its loadersupplies no permissions; reload only on a later confirmed permission change.Apply shared access in place only after a successful permissionless load.+- Grouped Soup's `graphql/create-grouped-soup-live-query.ts` owns initial and+ continuation pages plus their local predicate projection. Group readers only+ delegate paging; pass the parent's live `groupedPaging` accessor from every+ caller, including Tasks and project-task lists. Do not introduce independent+ continuation owners. Keep server+ membership snapshots separate from normalized field/link updates; restored+ properties do not prove prior membership. Supply complete prior multi-value+ memberships when adjusting counts: a row loaded in one bin may already be an+ unloaded member of another.- Reactive GraphQL reads use `createLiveQuery` for a document and`useSoupAstItemsQuery` (backed by `createSoupLiveQuery`) for Soup lists; see[choosing a reactive read](docs/graphql-normalized-cache-plan.md#choosing-a-reactive-read).diff --git a/apps/web/docs/graphql-normalized-cache-plan.md b/apps/web/docs/graphql-normalized-cache-plan.mdindex e5a848cb32..c8985dde3f 100644--- a/apps/web/docs/graphql-normalized-cache-plan.md+++ b/apps/web/docs/graphql-normalized-cache-plan.md@@ -728,6 +728,74 @@ use `createSoupLiveQuery`: it adds Soup's local membership, ordering, paginationand Mail policy through its predicate adapter. That policy remains outsidegeneric cache-core.+Grouped Soup uses `graphql/create-grouped-soup-live-query.ts` to own the initial+request and every loaded per-bin continuation. With maintained cache queries and+a supported filter plus created/updated sort, their combined server membership+is the baseline for the same predicate adapter used by flat Soup. Selected+normalized rows then derive date, project and property memberships (including+multi-value and scoped property joins); entity-type bins retain their invariant+server keys. Missing grouping evidence retains the old membership. When local selection is+incomplete, display fields come from current merged server pages before falling+back to the original evidence; current page fields do not rebase membership or+count evidence. Unsupported+filters/sorts or hosts retain the merged live server pages. Cached Mail keeps its+separate date-grouping policy.++Server membership evidence is captured separately from normalized field/link+updates, and local evaluation waits for network persistence. Filter/account/cache+generation changes discard old local evidence and continuation owners. Initial+network refreshes restart continuation coverage. Group readers delegate paging+to this owner, so moving a continuation row cannot leave a second copy behind.+The initial cursor remains baseline metadata; paging follows the last loaded+server continuation, never a locally fabricated cursor.++The reconciliation core in `grouped/reconciliation.ts` deliberately ignores+candidates outside loaded pages: a bounded flat query cannot prove complete bin+coverage. Counts retain unseen server membership and apply proven deltas, not+loaded-row totals. Complete prior multi-value membership is necessary to prove+count increases: a row loaded in A can already belong to an unloaded part of B.+Restored cached pages cannot prove complete prior property membership: their+normalized values may be newer than their embedded bins. They use loaded+membership evidence only, without inferring count increases for existing bins.+Date rollover uses the original server bin as old membership. The+`graphql_cache.soup_grouped_local_evaluation` span records grouping field,+loaded baseline size and resulting bin count.++#### Grouped projection performance++Run `bun scripts/benchmark-grouped-soup.ts /tmp/grouped-soup.json` from+`apps/web` (set `PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH` for a system Chromium).+It bundles the production functions into an isolated Chromium page: 20 warmups+and 100 samples per case, with 100/1,000/5,000 unique loaded rows, property+(single and three-value), project and date groups. Projection reads real Solid+store proxies inside a tracked memo and indexes current page items for display+fallback; deterministic rows move groups or fail the+filter. Page merging uses ten-row continuations. Flat materialization is a+comparison, not equivalent grouped behavior.++The measurements isolate main-thread projection and page merging; they exclude+worker predicate execution, IPC, network, UI mapping and DOM rendering. A 4× CPU+throttle is a stress case, not a claim about any particular mobile device.++Measured on an Intel Xeon Platinum 8581C with Chromium 155.0.8059.79:++| Loaded rows, three-value properties | Group projection p95 | Flat materialization p95 | Page merge p95, before → after |+| --- | --- | --- | --- |+| 100 | 0.8 ms | 0.2 ms | 0.2 → 0.1 ms |+| 1,000 | 6.0 ms | 0.7 ms | 10.0 → 0.4 ms |+| 5,000 | 30.8 ms | 3.4 ms | 258.1 → 1.8 ms |+| 100, 4× throttle | 2.8 ms | 0.8 ms | 0.9 → 0.7 ms |+| 1,000, 4× throttle | 25.3 ms | 3.0 ms | 41.5 → 1.8 ms |+| 5,000, 4× throttle | 129.7 ms | 14.0 ms | 1,067.2 → 7.8 ms |++Page merging now accumulates each bin once instead of copying all earlier pages+for every continuation. The identity-read regression test guards linear work+without a machine-dependent timing assertion. Small loaded views have modest+projection cost; the 5,000-row reconciliation ceiling is not a frame-time budget.+Full property regrouping at that depth still exceeds a 16.7 ms frame, especially+under throttling, and needs incremental projection before claiming smooth bulk+updates at the ceiling.+Network responses must provide the schema's identity fields (usually `id`, and`__typename` for abstract types) for normalization. They are not automaticallyinjected. Queries against other schemas, incremental `@defer`/`@stream` deliverydiff --git a/apps/web/scripts/benchmark-grouped-soup.ts b/apps/web/scripts/benchmark-grouped-soup.tsnew file mode 100644index 0000000000..d0150b3115--- /dev/null+++ b/apps/web/scripts/benchmark-grouped-soup.ts@@ -0,0 +1,47 @@+// bun scripts/benchmark-grouped-soup.ts /tmp/grouped-soup-benchmark.json+// Set PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH when using a system Chromium.+import { cpus } from 'node:os';+import { fileURLToPath } from 'node:url';+import { chromium } from '@playwright/test';+import type { runGroupedSoupBenchmark } from '../src/lib/queries/soup/graphql/grouped-local-projection.benchmark';++console.error('Building the browser benchmark…');+const bundle = await Bun.build({+ entrypoints: [fileURLToPath(new URL('../src/lib/queries/soup/graphql/grouped-local-projection.benchmark.ts', import.meta.url))],+ target: 'browser',+ conditions: ['browser'],+ minify: true,+ define: { 'process.env.NODE_ENV': '"production"' },+});+if (!bundle.success) throw new AggregateError(bundle.logs, 'Benchmark build failed');+console.error('Launching Chromium…');+const browser = await chromium.launch({+ executablePath: process.env.PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH,+});+try {+ const page = await browser.newPage();+ page.on('console', (message) => console.error(message.text()));+ console.error('Loading the benchmark…');+ await page.addScriptTag({ content: await bundle.outputs[0].text(), type: 'module' });+ await page.waitForFunction(() => typeof window.runGroupedSoupBenchmark === 'function');+ const cdp = await page.context().newCDPSession(page);+ const runs: Array<{ cpuThrottlingRate: number; measurement: ReturnType<typeof runGroupedSoupBenchmark> }> = [];+ for (const cpuThrottlingRate of [1, 4]) {+ await cdp.send('Emulation.setCPUThrottlingRate', { rate: cpuThrottlingRate });+ console.error(`Measuring at ${cpuThrottlingRate}x CPU throttle…`);+ const measurement = await page.evaluate(() => window.runGroupedSoupBenchmark());+ runs.push({ cpuThrottlingRate, measurement });+ }+ const report = {+ timestamp: new Date().toISOString(),+ browser: browser.version(),+ cpu: cpus()[0]?.model,+ scope: 'Tracked Solid-store projection and loaded-page merge; excludes worker, IPC, network and DOM.',+ runs,+ };+ const json = JSON.stringify(report, null, 2);+ if (Bun.argv[2]) await Bun.write(Bun.argv[2], `${json}\n`);+ console.log(json);+} finally {+ await browser.close();+}diff --git a/apps/web/src/features/next-soup/soup-view/soup-view-context.tsx b/apps/web/src/features/next-soup/soup-view/soup-view-context.tsxindex 321d9f5d24..0789c2d1b1 100644--- a/apps/web/src/features/next-soup/soup-view/soup-view-context.tsx+++ b/apps/web/src/features/next-soup/soup-view/soup-view-context.tsx@@ -1139,6 +1139,7 @@ export const createSoupViewState = (props: SoupViewContextProviderProps) => {};const groupQueries = createGroupedSoupQueries({+ groupedPaging: () => itemsQuery.groupedPaging,initialPage: createMemo(() => {if (itemsSource().isPlaceholderData()) return;diff --git a/apps/web/src/features/tasks-view/queries/use-tasks-query.test.ts b/apps/web/src/features/tasks-view/queries/use-tasks-query.test.tsindex d1c5794abc..4f066e9b04 100644--- a/apps/web/src/features/tasks-view/queries/use-tasks-query.test.ts+++ b/apps/web/src/features/tasks-view/queries/use-tasks-query.test.ts@@ -1,10 +1,13 @@import type { TaskEntityWithProperties } from '@entity/types/entity';+import type { GroupedSoupPaging } from '@queries/soup/graphql/create-grouped-soup-live-query';+import type { CreateGroupedSoupQueriesArgs } from '@queries/soup/grouped/create-grouped-soup-queries';import { createRoot, createSignal } from 'solid-js';import { expect, it, vi } from 'vitest';const fixture = vi.hoisted(() => ({query: {} as Record<string, unknown>,groupQueries: new Map<string, unknown>(),+ groupArgs: undefined as CreateGroupedSoupQueriesArgs | undefined,calls: [] as { args: () => unknown; options?: () => unknown }[],searchState: vi.fn(),mapItem: vi.fn(),@@ -41,7 +44,10 @@ vi.mock('@queries/soup/items', () => ({},}));vi.mock('@queries/soup/grouped/create-grouped-soup-queries', () => ({- createGroupedSoupQueries: () => ({ map: () => fixture.groupQueries }),+ createGroupedSoupQueries: (args: CreateGroupedSoupQueriesArgs) => {+ fixture.groupArgs = args;+ return { map: () => fixture.groupQueries };+ },}));import { useTasksDataSource } from './use-tasks-query';@@ -134,6 +140,96 @@ it.each([false, true])(});+it('delegates grouped task pagination to the current Soup query owner', async () => {+ const [paging, setPaging] = createSignal<GroupedSoupPaging>();+ const [hasMore, setHasMore] = createSignal(true);+ const task: TaskEntityWithProperties = {+ type: 'document',+ fileType: 'md',+ id: 'task',+ name: 'Task',+ ownerId: 'viewer',+ subType: { type: 'task', is_completed: false },+ properties: [],+ };+ const fetchNextPage = vi.fn(async () => {+ setHasMore(false);+ });+ fixture.query = {+ isLoading: false,+ isPending: false,+ isPlaceholderData: false,+ get groupedPaging() {+ return paging();+ },+ data: {+ entities: [task],+ itemsById: {},+ groups: [+ {+ key: 'todo',+ label: 'To do',+ totalCount: 2,+ itemIds: [task.id],+ nextCursor: 'next',+ },+ ],+ },+ };+ // The group reader consumes the owner's live capability, not another cursor chain.+ fixture.groupQueries = new Map([+ [+ 'todo',+ {+ data: () => ({ entities: [] }),+ hasNextPage: () =>+ fixture.groupArgs?.groupedPaging?.()?.hasNextPage('todo') ?? false,+ isFetchingNextPage: () => false,+ fetchNextPage: () =>+ fixture.groupArgs?.groupedPaging?.()?.fetchNextPage('todo'),+ error: () => null,+ },+ ],+ ]);+ let cleanup = () => {};+ const source = createRoot((dispose) => {+ cleanup = dispose;+ return useTasksDataSource(+ {+ tab: 'team-tasks',+ search: '',+ facets: {},+ groupBy: 'status',+ sort: [],+ },+ {+ userId: () => 'viewer',+ tagSets: () => [],+ tagSetsReady: () => true,+ isGroupExpanded: () => true,+ }+ );+ });+ try {+ expect(source.items().some((row) => row.kind === 'load-more')).toBe(false);+ setPaging({+ hasNextPage: hasMore,+ isFetchingNextPage: () => false,+ error: () => null,+ fetchNextPage,+ resetToInitialPage: () => {},+ });+ expect(source.items().some((row) => row.kind === 'load-more')).toBe(true);+ await source.loadMoreGroup('todo');+ expect(fetchNextPage).toHaveBeenCalledExactlyOnceWith('todo');+ expect(source.items().some((row) => row.kind === 'load-more')).toBe(false);+ } finally {+ cleanup();+ fixture.groupQueries = new Map();+ fixture.groupArgs = undefined;+ }+});+it('retains referenced list rows but hides stale board groups and inaccessible tasks', () => {const [placeholder, setPlaceholder] = createSignal(false);const [enabled, setEnabled] = createSignal(true);diff --git a/apps/web/src/features/tasks-view/queries/use-tasks-query.ts b/apps/web/src/features/tasks-view/queries/use-tasks-query.tsindex f8ba622bcd..2b1dfda173 100644--- a/apps/web/src/features/tasks-view/queries/use-tasks-query.ts+++ b/apps/web/src/features/tasks-view/queries/use-tasks-query.ts@@ -235,6 +235,7 @@ export function useTasksDataSource(});const groupedQueries = createGroupedSoupQueries({+ groupedPaging: () => query.groupedPaging,initialPage: createMemo(() => {if (search.isSearching() || query.isLoading || query.isPlaceholderData) {return;diff --git a/apps/web/src/lib/queries/soup/graphql/create-grouped-soup-live-query.test.ts b/apps/web/src/lib/queries/soup/graphql/create-grouped-soup-live-query.test.tsnew file mode 100644index 0000000000..fc9995b5bd--- /dev/null+++ b/apps/web/src/lib/queries/soup/graphql/create-grouped-soup-live-query.test.ts@@ -0,0 +1,400 @@+import {+ type Client,+ type GraphQLRequest,+ makeOperation,+ type OperationContext,+ type OperationResult,+} from '@urql/core';+import { createRoot, createSignal } from 'solid-js';+import { afterEach, describe, expect, it, vi } from 'vitest';+import { makeSubject } from 'wonka';+import { createNoopCacheHost } from '../../../graphql-cache/host/noop-host';+import type {+ CacheChangeListener,+ CacheHost,+} from '../../../graphql-cache/host/types';+import {+ type EntityFilterCacheArgs,+ type EntityFilterCacheResult,+ type LiveQueryUpdate,+ parseCacheRevision,+} from '../../../graphql-cache/protocol';+import type {+ GroupedSoupInput,+ GroupSoupQuery,+ GroupSoupQueryVariables,+ SoupItemFieldsFragment,+} from '../../../service-clients/service-storage/graphql/generated/graphql';+import { createGroupedSoupLiveQuery } from './create-grouped-soup-live-query';+import { groupedDocument } from './grouped-test-fixtures';+import { soupItemKey } from './reconciliation';++const mocks = vi.hoisted(() => ({ host: vi.fn(), client: vi.fn() }));+vi.mock('../../../service-clients/service-storage/graphql-soup', () => ({+ getGraphqlSoupCacheHost: mocks.host,+ getGraphqlSoupClient: mocks.client,+}));+vi.mock('@macro-inc/observability', () => ({+ Telemetry: {+ span: () => ({ setAttr: vi.fn(), end: vi.fn() }),+ error: vi.fn(),+ },+}));++const cleanups: Array<() => void> = [];+afterEach(() => {+ cleanups.splice(0).forEach((dispose) => dispose());+});++function setup() {+ const executions: Array<{+ variables: GroupSoupQueryVariables;+ publish(+ data: GroupSoupQuery,+ persistence?: Promise<string | undefined>,+ source?:+ | 'live-network'+ | 'normalized-cache-hit'+ | 'normalized-cache-patch'+ ): void;+ }> = [];+ const client = {+ executeQuery(+ request: GraphQLRequest<GroupSoupQuery, GroupSoupQueryVariables>,+ context: Partial<OperationContext>+ ) {+ const subject =+ makeSubject<OperationResult<GroupSoupQuery, GroupSoupQueryVariables>>();+ const operation = makeOperation('query', request, {+ url: 'http://unused',+ requestPolicy: 'cache-first',+ ...context,+ });+ executions.push({+ variables: request.variables,+ publish: (data, persistence, source = 'live-network') =>+ subject.next({+ operation,+ data,+ hasNext: false,+ stale: false,+ extensions: {+ __macroNormalizedCache: { source, persistence },+ },+ }),+ });+ return subject.source;+ },+ } as Client;+ const requests: Array<{+ args: EntityFilterCacheArgs;+ resolve(result: EntityFilterCacheResult): void;+ promise: Promise<EntityFilterCacheResult>;+ }> = [];+ const listeners = new Set<CacheChangeListener>();+ const generations = new Set<+ Parameters<CacheHost['onCacheGenerationChanged']>[0]+ >();+ const host: CacheHost = {+ ...createNoopCacheHost(),+ liveQueries: true,+ onCacheChanged: (listener) => {+ listeners.add(listener);+ return () => {+ listeners.delete(listener);+ };+ },+ onCacheGenerationChanged: (listener) => {+ generations.add(listener);+ return () => {+ generations.delete(listener);+ };+ },+ entityFilter: (args) => {+ if (args.liveQuery?.release)+ return Promise.resolve({ kind: 'unsupported' });+ let resolve!: (result: EntityFilterCacheResult) => void;+ const promise = new Promise<EntityFilterCacheResult>((done) => {+ resolve = done;+ });+ requests.push({ args, resolve, promise });+ return promise;+ },+ };+ mocks.host.mockReturnValue(host);+ mocks.client.mockReturnValue(client);+ const state = createRoot((dispose) => {+ cleanups.push(dispose);+ const [input, setInput] = createSignal<GroupedSoupInput>({+ initial: {+ sortMethod: 'UPDATED_AT',+ groupBy: {+ field: 'PROPERTY',+ propertyDefinitionId: 'status-definition',+ },+ limit: 2,+ filters: {},+ },+ });+ const query = createGroupedSoupLiveQuery(+ input,+ () => ({ enabled: true, onInitialResult: () => {} }),+ () => new Date('2026-10-10T12:00:00Z')+ );+ return { query, setInput, dispose };+ });+ return { ...state, executions, requests, listeners, generations };+}++function page(+ items: SoupItemFieldsFragment[],+ cursor: string | null = 'next'+): GroupSoupQuery {+ return {+ user: {+ id: 'viewer',+ groupSoup: {+ bins: [{ key: 'todo', items, totalCount: 20, nextCursor: cursor }],+ },+ },+ };+}++function update(+ records: SoupItemFieldsFragment[],+ revision = '1'+): LiveQueryUpdate {+ return {+ kind: 'live-query',+ revision: parseCacheRevision(revision),+ reset: true,+ keys: records.map(soupItemKey),+ upserts: records.map((record) => ({+ recordKey: soupItemKey(record),+ record,+ identity: { pending: false, mutationUuid: null },+ })),+ patches: [],+ removed: [],+ retainedKeys: [],+ optimistic: false,+ };+}++const ids = (data: GroupSoupQuery | undefined) =>+ data?.user.groupSoup.bins.map((bin) => ({+ key: bin.key,+ ids: bin.items.map((item) => item.id),+ count: bin.totalCount,+ cursor: bin.nextCursor,+ }));++describe('grouped Soup live ownership', () => {+ it('moves initial and continuation rows through one maintained query and preserves server pagination', async () => {+ const state = setup();+ state.executions[0].publish(page([groupedDocument('a')]));+ expect(state.requests[0].args.baseline?.map((entry) => entry.key)).toEqual([+ 'GraphqlSoupDocument:a',+ ]);+ state.requests[0].resolve(update([groupedDocument('a')]));+ await state.requests[0].promise;+ const next = state.query.paging.fetchNextPage('todo');+ state.executions[1].publish(page([groupedDocument('b')], 'third'));+ await next;+ const request = state.requests.at(-1)!;+ expect(request.args.baseline?.map((entry) => entry.key)).toEqual([+ 'GraphqlSoupDocument:a',+ 'GraphqlSoupDocument:b',+ ]);+ request.resolve(+ update([groupedDocument('b', ['done']), groupedDocument('a')], '2')+ );+ await request.promise;+ expect(ids(state.query.data())).toEqual([+ { key: 'todo', ids: ['a'], count: 19, cursor: 'next' },+ { key: 'done', ids: ['b'], count: 1, cursor: null },+ ]);+ expect(state.query.paging.hasNextPage('todo')).toBe(true);+ const third = state.query.paging.fetchNextPage('todo');+ expect(state.executions[2].variables.input.continuation?.cursor).toBe(+ 'third'+ );+ state.executions[2].publish(page([groupedDocument('c')], null));+ await third;+ expect(state.query.paging.hasNextPage('todo')).toBe(false);+ state.query.paging.resetToInitialPage();+ expect(ids(state.query.data())?.flatMap((bin) => bin.ids)).not.toContain(+ 'b'+ );+ });++ it('recomputes normalized property changes without another network operation, including rollback', async () => {+ const state = setup();+ state.executions[0].publish(page([groupedDocument('a')]));+ state.requests[0].resolve(update([groupedDocument('a', ['done'])]));+ await state.requests[0].promise;+ expect(+ ids(state.query.data())?.find((bin) => bin.key === 'done')?.ids+ ).toEqual(['a']);+ state.listeners.forEach((listener) => listener(parseCacheRevision('2')));+ const request = state.requests.at(-1)!;+ request.resolve(update([groupedDocument('a')], '2'));+ await request.promise;+ expect(ids(state.query.data())).toEqual([+ { key: 'todo', ids: ['a'], count: 20, cursor: 'next' },+ ]);+ expect(state.executions).toHaveLength(1);+ });++ it.each([false, true])(+ 'keeps current initial and continuation fields when selection is incomplete (unknown membership: %s)',+ async (unknown) => {+ const state = setup();+ const initial = groupedDocument('a');+ const continued = groupedDocument('b');+ state.executions[0].publish(page([initial]));+ const next = state.query.paging.fetchNextPage('todo');+ state.executions[1].publish(page([continued], null));+ await next;+ const keys = [initial, continued].map(soupItemKey);+ const request = state.requests.at(-1)!;+ request.resolve({+ ...update([]),+ keys,+ retainedKeys: unknown ? keys : [],+ });+ await request.promise;++ state.executions[0].publish(+ page([+ { ...groupedDocument('a', ['done']), displayName: 'Renamed initial' },+ ]),+ undefined,+ 'normalized-cache-patch'+ );+ state.executions[1].publish(+ page(+ [+ {+ ...groupedDocument('b', ['done']),+ displayName: 'Renamed continuation',+ },+ ],+ null+ ),+ undefined,+ 'normalized-cache-patch'+ );+ expect(+ state.query+ .data()+ ?.user.groupSoup.bins[0].items.map((item) => item.displayName)+ ).toEqual(['Renamed initial', 'Renamed continuation']);+ // Display updates are not new evidence for moving rows or changing counts.+ expect(ids(state.query.data())).toEqual([+ { key: 'todo', ids: ['a', 'b'], count: 20, cursor: 'next' },+ ]);+ expect(state.query.paging.hasNextPage('todo')).toBe(false);+ expect(state.requests.at(-1)).toBe(request);+ expect(state.executions).toHaveLength(2);+ }+ );++ it('ignores an obsolete evaluation after filters change', async () => {+ const state = setup();+ state.executions[0].publish(page([groupedDocument('old')]));+ const old = state.requests[0];+ state.setInput({+ initial: {+ sortMethod: 'UPDATED_AT',+ groupBy: { field: 'PROJECT' },+ filters: {},+ },+ });+ state.executions[1].publish(page([groupedDocument('new')]));+ old.resolve(update([groupedDocument('old', ['done'])]));+ await old.promise;+ expect(ids(state.query.data())?.flatMap((bin) => bin.ids)).toEqual(['new']);+ });++ it('keeps server membership until persistence and never rebases old count evidence on a normalized patch', async () => {+ const state = setup();+ let acknowledge!: (revision: string) => void;+ const persistence = new Promise<string>((resolve) => {+ acknowledge = resolve;+ });+ state.executions[0].publish(page([groupedDocument('a')]), persistence);+ expect(state.requests).toHaveLength(0);+ expect(ids(state.query.data())?.[0].ids).toEqual(['a']);+ acknowledge('1');+ await persistence;+ expect(state.requests).toHaveLength(1);+ state.requests[0].resolve(update([groupedDocument('a', ['done'])]));+ await state.requests[0].promise;+ expect(ids(state.query.data())?.[0].count).toBe(19);+ state.executions[0].publish(+ page([groupedDocument('a', ['done'])]),+ undefined,+ 'normalized-cache-patch'+ );+ expect(state.requests).toHaveLength(1);+ expect(ids(state.query.data())).toEqual([+ { key: 'todo', ids: [], count: 19, cursor: 'next' },+ { key: 'done', ids: ['a'], count: 1, cursor: null },+ ]);+ });++ it('does not treat restored normalized properties as the original multi-value membership', async () => {+ const state = setup();+ const cached = page([groupedDocument('a', ['done'])]);+ cached.user.groupSoup.bins.push({+ key: 'done',+ items: [],+ totalCount: 12,+ nextCursor: 'done-next',+ });+ state.executions[0].publish(cached, undefined, 'normalized-cache-hit');+ state.requests[0].resolve(update([groupedDocument('a', ['done'])]));+ await state.requests[0].promise;+ expect(ids(state.query.data())).toEqual([+ { key: 'todo', ids: [], count: 19, cursor: 'next' },+ { key: 'done', ids: ['a'], count: 12, cursor: 'done-next' },+ ]);+ });++ it('keeps network data when persistence is unavailable and ignores late acknowledgements after reset', async () => {+ const state = setup();+ let acknowledge!: (revision: string | undefined) => void;+ const persistence = new Promise<string | undefined>((resolve) => {+ acknowledge = resolve;+ });+ state.executions[0].publish(page([groupedDocument('a')]), persistence);+ acknowledge(undefined);+ await persistence;+ expect(state.requests).toHaveLength(0);+ expect(ids(state.query.data())?.[0].ids).toEqual(['a']);+ let lateAcknowledge!: (revision: string) => void;+ const late = new Promise<string>((resolve) => {+ lateAcknowledge = resolve;+ });+ state.executions[0].publish(page([groupedDocument('b')]), late);+ state.generations.forEach((listener) => listener({ storage: 'reset' }));+ lateAcknowledge('2');+ await late;+ expect(state.requests).toHaveLength(0);+ });++ it('drops local evidence and loaded continuations on a cache generation reset', async () => {+ const state = setup();+ state.executions[0].publish(page([groupedDocument('a')]));+ const next = state.query.paging.fetchNextPage('todo');+ state.executions[1].publish(page([groupedDocument('b')], null));+ await next;+ const old = state.requests.at(-1)!;+ state.generations.forEach((listener) => listener({ storage: 'reset' }));+ old.resolve(update([groupedDocument('b', ['done'])]));+ await old.promise;+ expect(ids(state.query.data())?.flatMap((bin) => bin.ids)).toEqual(['a']);+ expect(state.query.paging.hasNextPage('todo')).toBe(true);+ });+});diff --git a/apps/web/src/lib/queries/soup/graphql/create-grouped-soup-live-query.ts b/apps/web/src/lib/queries/soup/graphql/create-grouped-soup-live-query.tsnew file mode 100644index 0000000000..ca27594d05--- /dev/null+++ b/apps/web/src/lib/queries/soup/graphql/create-grouped-soup-live-query.ts@@ -0,0 +1,373 @@+import { createUrqlInfiniteQuery, createUrqlQuery } from '@app/lib/urql-solid';+import { Telemetry } from '@macro-inc/observability';+import type { OperationResult } from '@urql/core';+import {+ type Accessor,+ createComputed,+ createEffect,+ createMemo,+ createRoot,+ createSignal,+ on,+ onCleanup,+} from 'solid-js';+import { normalizedCacheResultMetadata } from '../../../graphql-cache/exchange/normalized-cache-exchange';+import { selectRecords } from '../../../graphql-cache/exchange/record-selection';+import {+ type GroupedSoupInput,+ GroupSoupDocument,+ type GroupSoupQuery,+ type GroupSoupQueryVariables,+ type SoupItemFieldsFragment,+ SoupItemFieldsFragmentDoc,+} from '../../../service-clients/service-storage/graphql/generated/graphql';+import {+ getGraphqlSoupCacheHost,+ getGraphqlSoupClient,+} from '../../../service-clients/service-storage/graphql-soup';+import { registerGroupedSoupContinuation } from '../grouped/graphql-operation-registry';+import { registerGraphqlSoupRevalidations } from './active-queries';+import { createPredicateQuery } from './create-predicate-query';+import {+ mergeGroupedSoupBins,+ projectLocalGroupedSoup,+} from './grouped-local-projection';+import { createGroupedPageEvidence } from './grouped-page-evidence';+import { soupItemKey, soupReconciliationBaseline } from './reconciliation';++/** Paging capabilities; rows for all loaded groups are owned by the parent source. */+export type GroupedSoupPaging = {+ hasNextPage(key: string): boolean;+ isFetchingNextPage(key: string): boolean;+ error(key: string): Error | null;+ fetchNextPage(key: string): Promise<void>;+ resetToInitialPage(): void;+};++type Options = {+ enabled: boolean;+ networkPaused?: boolean;+ keepPreviousData?: boolean;+ onInitialResult(+ result: OperationResult<GroupSoupQuery, GroupSoupQueryVariables>+ ): void;+};++/** One owner for grouped server pages and their maintained local membership. */+export function createGroupedSoupLiveQuery(+ input: Accessor<GroupedSoupInput | undefined>,+ options: Accessor<Options>,+ now: Accessor<Date>+) {+ const [generation, setGeneration] = createSignal(0);+ let previousHost = getGraphqlSoupCacheHost();+ createEffect(+ on(getGraphqlSoupCacheHost, (host) => {+ if (host !== previousHost) setGeneration((value) => value + 1);+ previousHost = host;+ if (host)+ onCleanup(+ host.onCacheGenerationChanged(() =>+ setGeneration((value) => value + 1)+ )+ );+ })+ );+ const scope = () => JSON.stringify([input(), generation()]);+ const initialEvidence = createGroupedPageEvidence(scope);+ let resetContinuations = () => {};+ const query = createUrqlQuery<+ GroupSoupQuery,+ GroupSoupQueryVariables,+ { inputKey: string; data: GroupSoupQuery }+ >(() => {+ const queryInput = input();+ const requestScope = scope();+ const queryOptions = options();+ const common = {+ query: GroupSoupDocument,+ client: getGraphqlSoupClient(),+ requestPolicy: queryOptions.networkPaused+ ? ('cache-only' as const)+ : ('cache-and-network' as const),+ keepPreviousData: queryOptions.keepPreviousData ?? false,+ onResult: (+ result: OperationResult<GroupSoupQuery, GroupSoupQueryVariables>+ ) => {+ if (scope() !== requestScope) return;+ if (+ result.data &&+ !result.error &&+ !result.hasNext &&+ normalizedCacheResultMetadata(result)?.source === 'live-network'+ )+ resetContinuations();+ initialEvidence.accept(result, 0, requestScope);+ queryOptions.onInitialResult(result);+ },+ select: (data: GroupSoupQuery) => ({+ inputKey: JSON.stringify(queryInput),+ data,+ }),+ };+ return queryOptions.enabled && queryInput+ ? { ...common, variables: { input: queryInput }, enabled: true }+ : { ...common, enabled: false as const };+ });+ const currentInitial = () =>+ query.data?.inputKey === JSON.stringify(input())+ ? query.data.data+ : undefined;+ const continuationScope = () =>+ JSON.stringify([scope(), currentInitial()?.user.id]);++ const createContinuation = (+ key: string,+ cursor: string,+ initial: GroupedSoupInput+ ) =>+ createRoot((dispose) => {+ const requestScope = continuationScope();+ const evidence = createGroupedPageEvidence(continuationScope);+ const [activated, setActivated] = createSignal(false);+ const continuationInput = (cursor: string): GroupedSoupInput => ({+ continuation: {+ cursor,+ groupKey: key,+ groupBy: (initial.initial ?? initial.continuation).groupBy,+ },+ });+ let settle = () => {};+ const settled = new Promise<void>((resolve) => {+ settle = resolve;+ });+ const continued = createUrqlInfiniteQuery<+ GroupSoupQuery,+ GroupSoupQueryVariables,+ string,+ { pages: Accessor<GroupSoupQuery[]>; pageParams: readonly string[] }+ >(() => ({+ query: GroupSoupDocument,+ client: getGraphqlSoupClient(),+ initialPageParam: cursor,+ variables: (cursor) => {+ const pageInput = continuationInput(cursor);+ registerGroupedSoupContinuation(initial, pageInput);+ return { input: pageInput };+ },+ enabled:+ activated() &&+ options().enabled &&+ continuationScope() === requestScope,+ requestPolicy: options().networkPaused ? 'cache-only' : 'cache-first',+ keepPreviousData: false,+ getNextPageParam: (page) =>+ page.user.groupSoup.bins.find((bin) => bin.key === key)?.nextCursor ??+ undefined,+ onResult: (result, page) =>+ evidence.accept(result, page.pageIndex, requestScope),+ select: ({ pages, pageParams }) => ({ pages: () => pages, pageParams }),+ }));+ onCleanup(+ registerGraphqlSoupRevalidations(() =>+ continued.isEnabled+ ? [...new Set([cursor, ...(continued.data?.pageParams ?? [])])].map(+ (cursor) => ({+ document: GroupSoupDocument,+ variables: { input: continuationInput(cursor) },+ })+ )+ : []+ )+ );+ createComputed(() => {+ if (+ activated() &&+ (!continued.isEnabled ||+ (continued.isFetched && !continued.isFetching))+ )+ settle();+ });+ onCleanup(settle);+ return {+ query: continued,+ evidence,+ dispose,+ activate: async () => {+ setActivated(true);+ await settled;+ },+ };+ });+ const [continuations, setContinuations] = createSignal<+ ReadonlyMap<string, ReturnType<typeof createContinuation>>+ >(new Map());+ resetContinuations = () => {+ for (const continuation of continuations().values()) continuation.dispose();+ setContinuations(new Map());+ };+ createComputed(on(continuationScope, () => resetContinuations()));+ createComputed(+ on(+ () => currentInitial()?.user.groupSoup.bins.map((bin) => bin.key) ?? [],+ (keys) => {+ const active = new Set(keys);+ const remaining = new Map(continuations());+ for (const [key, continuation] of remaining) {+ if (active.has(key)) continue;+ continuation.dispose();+ remaining.delete(key);+ }+ if (remaining.size !== continuations().size)+ setContinuations(remaining);+ }+ )+ );+ onCleanup(() => {+ for (const continuation of continuations().values()) continuation.dispose();+ });++ const serverPages = createMemo(() => {+ const initial = currentInitial();+ if (!initial) return [];+ return [+ initial,+ ...[...continuations().values()].flatMap(+ (continuation) => continuation.query.data?.pages() ?? []+ ),+ ].filter((page) => page.user.id === initial.user.id);+ });+ const baseline = createMemo(() => {+ const initial = initialEvidence.data();+ if (+ !initial ||+ !currentInitial() ||+ initial[0].user.id !== currentInitial()?.user.id+ )+ return undefined;+ const pages = [...initial];+ const untrustedKeys = initialEvidence.untrustedKeys();+ for (const continuation of continuations().values()) {+ const loaded = continuation.query.data?.pages();+ if (!loaded?.length) continue;+ const evidence = continuation.evidence.data();+ if (!evidence || evidence.length !== loaded.length) return undefined;+ pages.push(...evidence);+ for (const key of continuation.evidence.untrustedKeys())+ untrustedKeys.add(key);+ }+ return {+ bins: mergeGroupedSoupBins(+ pages.filter((page) => page.user.id === initial[0].user.id)+ ),+ untrustedKeys,+ };+ });+ const selection = selectRecords(SoupItemFieldsFragmentDoc);+ const live = createPredicateQuery<SoupItemFieldsFragment>({+ host: getGraphqlSoupCacheHost,+ query: () => {+ const initial = input()?.initial;+ const bins = baseline();+ if (+ !options().enabled ||+ !getGraphqlSoupCacheHost()?.liveQueries ||+ !initial ||+ !bins ||+ (initial.sortMethod !== 'CREATED_AT' &&+ initial.sortMethod !== 'UPDATED_AT')+ )+ return undefined;+ const membership = soupReconciliationBaseline(+ bins.bins.flatMap((bin) => bin.items),+ initial.sortMethod+ );+ if (!membership) return undefined;+ return {+ source: {+ filters: initial.filters ?? {},+ sortMethod: initial.sortMethod,+ sortDirection: 'DESC',+ limit: initial.limit ?? 20,+ baseline: membership,+ },+ select: selection,+ };+ },+ });+ const bins = createMemo(() => {+ const initial = input()?.initial;+ const evidence = baseline();+ const snapshot = live.data();+ if (!initial || !evidence || !snapshot || live.status() !== 'ready')+ return mergeGroupedSoupBins(serverPages());+ const span = Telemetry.span('graphql_cache.soup_grouped_local_evaluation');+ try {+ const currentItems = new Map<string, SoupItemFieldsFragment>();+ for (const page of serverPages())+ for (const bin of page.user.groupSoup.bins)+ for (const item of bin.items)+ currentItems.set(soupItemKey(item), item);+ const result = projectLocalGroupedSoup(+ evidence.bins,+ currentItems,+ snapshot,+ initial,+ now(),+ evidence.untrustedKeys+ );+ span.setAttr('grouping.field', initial.groupBy.field);+ span.setAttr('grouping.bin_count', result.length);+ span.setAttr(+ 'grouping.baseline_count',+ evidence.bins.reduce((count, bin) => count + bin.items.length, 0)+ );+ return result;+ } finally {+ span.end();+ }+ });+ const paging: GroupedSoupPaging = {+ hasNextPage: (key) => {+ const continuation = continuations().get(key);+ return continuation?.query.data+ ? continuation.query.hasNextPage+ : Boolean(bins().find((bin) => bin.key === key)?.nextCursor);+ },+ isFetchingNextPage: (key) =>+ continuations().get(key)?.query.isFetching ?? false,+ error: (key) => continuations().get(key)?.query.error ?? null,+ fetchNextPage: async (key) => {+ const initial = input();+ if (!initial || !options().enabled) return;+ const current = continuations().get(key);+ if (current) {+ if (current.query.isFetching) return;+ if (!current.query.data) await current.query.refetch();+ else await current.query.fetchNextPage();+ return;+ }+ const cursor = bins().find((bin) => bin.key === key)?.nextCursor;+ if (!cursor) return;+ const continuation = createContinuation(key, cursor, initial);+ setContinuations((previous) => new Map(previous).set(key, continuation));+ await continuation.activate();+ },+ resetToInitialPage: () => resetContinuations(),+ };+ return {+ query,+ paging,+ data: createMemo((): GroupSoupQuery | undefined => {+ const initial = currentInitial();+ if (!initial) return query.data?.data;+ return {+ ...initial,+ user: {+ ...initial.user,+ groupSoup: { ...initial.user.groupSoup, bins: bins() },+ },+ };+ }),+ };+}diff --git a/apps/web/src/lib/queries/soup/graphql/grouped-items-offline.test.ts b/apps/web/src/lib/queries/soup/graphql/grouped-items-offline.test.tsindex 395ce081e8..2038f1d436 100644--- a/apps/web/src/lib/queries/soup/graphql/grouped-items-offline.test.ts+++ b/apps/web/src/lib/queries/soup/graphql/grouped-items-offline.test.ts@@ -25,6 +25,7 @@ vi.mock('@queries/client', async () => {return { queryClient: new QueryClient() };});vi.mock('@service-storage/graphql-soup', () => ({+ getGraphqlSoupCacheHost: () => undefined,getGraphqlSoupClient: mocks.client,mapGraphqlSoupItem: (item: unknown) => item,}));diff --git a/apps/web/src/lib/queries/soup/graphql/grouped-items.test.ts b/apps/web/src/lib/queries/soup/graphql/grouped-items.test.tsindex b6690447fb..4e5446f0ad 100644--- a/apps/web/src/lib/queries/soup/graphql/grouped-items.test.ts+++ b/apps/web/src/lib/queries/soup/graphql/grouped-items.test.ts@@ -17,6 +17,7 @@ vi.mock('@queries/storage/instructions-md', () => ({}));vi.mock('@service-storage/graphql-soup', () => ({getGraphqlSoupClient: () => ({}),+ getGraphqlSoupCacheHost: () => undefined,mapGraphqlGroupedSoupPage: vi.fn(),}));vi.mock('./grouped-projection', () => ({@@ -72,7 +73,10 @@ function fixture(emailView: 'inbox' | 'drafts' | 'all' = 'drafts') {) => {onResult = options().onResult;return {- data: options().select({ user: { id: 'viewer' }, data: network }),+ data: options().select({+ user: { id: 'viewer', groupSoup: { bins: [] } },+ data: network,+ }),error: undefined,};}@@ -96,7 +100,7 @@ function fixture(emailView: 'inbox' | 'drafts' | 'all' = 'drafts') {setRevision,publish(persistence: Promise<string | undefined>) {onResult({- data: {},+ data: { user: { id: 'viewer', groupSoup: { bins: [] } } },metadata: { source: 'live-network', persistence },} as unknown as OperationResult);},diff --git a/apps/web/src/lib/queries/soup/graphql/grouped-items.ts b/apps/web/src/lib/queries/soup/graphql/grouped-items.tsindex 00e8ecfb04..4256833da2 100644--- a/apps/web/src/lib/queries/soup/graphql/grouped-items.ts+++ b/apps/web/src/lib/queries/soup/graphql/grouped-items.ts@@ -8,19 +8,13 @@ import {type CacheRevision,normalizedCacheResultMetadata,} from '@app/lib/graphql-cache';-import { createUrqlQuery } from '@app/lib/urql-solid';import { createBrowserOfflineSignal } from '@core/util/connectivity';import { fireAndForget } from '@core/util/fireAndForget';import { isTransientRequestError } from '@core/util/request-error';import { Telemetry } from '@macro-inc/observability';import type { GroupByField } from '@queries/soup/grouped/types';import { useInstructionsMdIdQuery } from '@queries/storage/instructions-md';-import {- GroupSoupDocument,- type GroupSoupQuery,- type GroupSoupQueryVariables,-} from '@service-storage/graphql/generated/graphql';-import { getGraphqlSoupClient } from '@service-storage/graphql-soup';+import { GroupSoupDocument } from '@service-storage/graphql/generated/graphql';import type { CombinedError } from '@urql/core';import {type Accessor,@@ -35,6 +29,7 @@ import { groupCachedMailByDate } from '../grouped/mail-date-groups';import type { SoupAstBody, SoupAstParams } from '../items';import { registerGraphqlSoupRevalidations } from './active-queries';import { makeGraphqlGroupedSoupInput } from './ast';+import { createGroupedSoupLiveQuery } from './create-grouped-soup-live-query';import { createGraphqlSoupDoneProjection } from './done-projection';import { createGraphqlGroupedSoupProjection } from './grouped-projection';import {@@ -122,62 +117,35 @@ export function createGraphqlGroupedSoupAstItemsQuery(const isSupported = () => input() !== undefined;const inputKey = createMemo(() => JSON.stringify(input()));- const query = createUrqlQuery<- GroupSoupQuery,- GroupSoupQueryVariables,- { inputKey: string; data: GroupSoupQuery }- >(() => {- const queryOptions = options();- const groupBy = args().groupBy;- const queryInput = input();-- const common = {- query: GroupSoupDocument,- client: getGraphqlSoupClient(),- requestPolicy: queryOptions.networkPaused- ? ('cache-only' as const)- : ('cache-and-network' as const),- keepPreviousData: queryOptions.keepPreviousData ?? false,- onResult: (- result: import('@urql/core').OperationResult<- GroupSoupQuery,- GroupSoupQueryVariables- >- ) => {- const metadata = normalizedCacheResultMetadata(result);- if (metadata?.source !== 'live-network') return;- const version = ++networkResultVersion;- setNetworkRevision(undefined);- if (result.error || result.data == null || result.hasNext) return;- fireAndForget(- acknowledgeNetworkResult(- metadata.persistence ?? Promise.resolve(metadata.revision),- queryInput,- version- ),- 'soup-grouped-items'- );- },- // Associate cached data with its filters without remapping live rows.- select: (data: GroupSoupQuery) => ({- inputKey: JSON.stringify(queryInput),- data,- }),- };-- if (!queryOptions.enabled || !groupBy || queryInput === undefined) {- return { ...common, enabled: false as const };- }-- return {- ...common,- variables: { input: queryInput },- enabled: true,- };- });+ const grouped = createGroupedSoupLiveQuery(+ input,+ () => {+ const queryInput = input();+ return {+ ...options(),+ onInitialResult: (result) => {+ const metadata = normalizedCacheResultMetadata(result);+ if (metadata?.source !== 'live-network') return;+ const version = ++networkResultVersion;+ setNetworkRevision(undefined);+ if (result.error || result.data == null || result.hasNext) return;+ fireAndForget(+ acknowledgeNetworkResult(+ metadata.persistence ?? Promise.resolve(metadata.revision),+ queryInput,+ version+ ),+ 'soup-grouped-items'+ );+ },+ };+ },+ now+ );+ const query = grouped.query;const projected = createGraphqlGroupedSoupProjection(- () => query.data?.data,+ grouped.data,() => args().groupBy,() => ({instructionsIdQuery,@@ -246,6 +214,7 @@ export function createGraphqlGroupedSoupAstItemsQuery();return {+ groupedPaging: grouped.paging,data: createMemo(() =>projectDone(JSON.stringify([input(), viewerId()]),@@ -267,7 +236,7 @@ export function createGraphqlGroupedSoupAstItemsQuery(fetchNextPage: async () => {if (cachedMail()) await local.fetchNextPage();},- resetToInitialPage: () => undefined,+ resetToInitialPage: grouped.paging.resetToInitialPage,refresh: async () => {if (input() === undefined) return;await query.refetch({diff --git a/apps/web/src/lib/queries/soup/graphql/grouped-local-projection.benchmark.ts b/apps/web/src/lib/queries/soup/graphql/grouped-local-projection.benchmark.tsnew file mode 100644index 0000000000..f17c852bae--- /dev/null+++ b/apps/web/src/lib/queries/soup/graphql/grouped-local-projection.benchmark.ts@@ -0,0 +1,204 @@+import { createMemo, createRoot, createSignal } from 'solid-js';+import { createStore } from 'solid-js/store';+import { INITIAL_CACHE_REVISION } from '../../../graphql-cache/protocol';+import type {+ GroupedSoupInitialInput,+ GroupSoupQuery,+ SoupItemFieldsFragment,+} from '../../../service-clients/service-storage/graphql/generated/graphql';+import {+ localSoupGroupKeys,+ mergeGroupedSoupBins,+ projectLocalGroupedSoup,+} from './grouped-local-projection';+import { groupedDocument } from './grouped-test-fixtures';+import { materializeReconciledSoup, soupItemKey } from './reconciliation';++const now = new Date('2026-10-10T12:00:00Z');+const warmup = 20;+const samples = 100;++function measure(run: () => number) {+ let checksum = 0;+ for (let index = 0; index < warmup; index++) checksum += run();+ const durations: number[] = [];+ for (let index = 0; index < samples; index++) {+ const started = performance.now();+ checksum += run();+ durations.push(performance.now() - started);+ }+ durations.sort((a, b) => a - b);+ return {+ medianMs: durations[Math.floor(samples / 2)],+ p95Ms: durations[Math.floor(samples * 0.95)],+ maxMs: durations.at(-1),+ checksum,+ };+}++function scenario(+ size: number,+ field: GroupedSoupInitialInput['groupBy']['field'],+ multi: boolean+) {+ const input: GroupedSoupInitialInput = {+ groupBy: { field, propertyDefinitionId: 'status-definition' },+ sortMethod: 'UPDATED_AT',+ limit: 100,+ };+ const before = Array.from({ length: size }, (_, index) => ({+ ...groupedDocument(+ String(index),+ Array.from(+ { length: multi ? 3 : 1 },+ (_, offset) => `bin-${(index + offset) % 5}`+ )+ ),+ projectId: `project-${index % 20}`,+ updatedAt: new Date(+ now.getTime() - (index % 90) * 86_400_000+ ).toISOString(),+ }));+ const initialBins = new Map<string, typeof before>();+ for (const item of before) {+ for (const key of localSoupGroupKeys(item, input, now) ?? []) {+ const items = initialBins.get(key) ?? [];+ items.push(item);+ initialBins.set(key, items);+ }+ }+ const pages: GroupSoupQuery[] = [];+ const initial: GroupSoupQuery = {+ user: { id: 'benchmark', groupSoup: { bins: [] } },+ };+ pages.push(initial);+ for (const [key, items] of initialBins) {+ for (let offset = 0; offset < items.length; offset += 10) {+ const bin = {+ key,+ items: items.slice(offset, offset + 10),+ totalCount: items.length + 100,+ nextCursor: `${key}:${offset + 10}`,+ };+ if (offset === 0) initial.user.groupSoup.bins.push(bin);+ else+ pages.push({ user: { id: 'benchmark', groupSoup: { bins: [bin] } } });+ }+ }+ const baseline = mergeGroupedSoupBins(pages);+ // Deterministic moves and proven filter exclusions, including multi-value bins.+ const after = before+ .filter((_, index) => index % 23 !== 0)+ .map((item, index) =>+ index % 20 === 0+ ? {+ ...item,+ projectId: 'project-new',+ updatedAt: now.toISOString(),+ properties: groupedDocument(item.id, ['destination']).properties,+ }+ : item+ );+ const records = after.map((record) => ({+ recordKey: soupItemKey(record),+ record,+ }));+ const [selected] = createStore({ records });+ const snapshot = {+ revision: INITIAL_CACHE_REVISION,+ optimistic: true,+ retainedKeys: [],+ keys: records.map((row) => row.recordKey),+ records: selected.records,+ };+ return { input, before, baseline, pages, snapshot };+}++/** Incremental main-thread cost only: excludes worker filtering, IPC, network and DOM layout. */+export function runGroupedSoupBenchmark() {+ const results = [];+ for (const size of [100, 1_000, 5_000]) {+ for (const [field, multi] of [+ ['PROPERTY', false],+ ['PROPERTY', true],+ ['PROJECT', false],+ ['DATE', false],+ ] as const) {+ console.info(`Starting ${size} rows, ${field}, multi=${multi}`);+ const data = scenario(size, field, multi);+ const result = createRoot((dispose) => {+ try {+ const [tick, setTick] = createSignal(0);+ // Measure tracked reads of actual Solid store proxies, not just plain JSON.+ const grouped = createMemo(() => {+ tick();+ const currentItems = new Map<string, SoupItemFieldsFragment>();+ for (const page of data.pages)+ for (const bin of page.user.groupSoup.bins)+ for (const item of bin.items)+ currentItems.set(soupItemKey(item), item);+ return projectLocalGroupedSoup(+ data.baseline,+ currentItems,+ data.snapshot,+ data.input,+ now+ );+ });+ const groupedProjection = measure(() => {+ setTick((value) => value + 1);+ return grouped().reduce(+ (count, bin) => count + bin.items.length,+ 0+ );+ });+ return { groupedProjection };+ } finally {+ dispose();+ }+ });+ const flatProjection = createRoot((dispose) => {+ try {+ const [tick, setTick] = createSignal(0);+ const flat = createMemo(() => {+ tick();+ return materializeReconciledSoup(+ data.snapshot.keys,+ data.snapshot.records,+ data.before+ );+ });+ return measure(() => {+ setTick((value) => value + 1);+ return flat().length;+ });+ } finally {+ dispose();+ }+ });+ const pageMerge = measure(() => mergeGroupedSoupBins(data.pages).length);+ results.push({+ size,+ field,+ multi,+ bins: data.baseline.length,+ memberships: data.baseline.reduce(+ (count, bin) => count + bin.items.length,+ 0+ ),+ pages: data.pages.length,+ ...result,+ flatProjection,+ pageMerge,+ });+ }+ }+ return { warmup, samples, results };+}++declare global {+ interface Window {+ runGroupedSoupBenchmark: typeof runGroupedSoupBenchmark;+ }+}+window.runGroupedSoupBenchmark = runGroupedSoupBenchmark;diff --git a/apps/web/src/lib/queries/soup/graphql/grouped-local-projection.test.ts b/apps/web/src/lib/queries/soup/graphql/grouped-local-projection.test.tsnew file mode 100644index 0000000000..c365c61bdb--- /dev/null+++ b/apps/web/src/lib/queries/soup/graphql/grouped-local-projection.test.ts@@ -0,0 +1,286 @@+import { describe, expect, it } from 'vitest';+import { parseCacheRevision } from '../../../graphql-cache/protocol';+import type {+ GroupedSoupInitialInput,+ GroupSoupQuery,+ SoupItemFieldsFragment,+} from '../../../service-clients/service-storage/graphql/generated/graphql';+import {+ localSoupGroupKeys,+ mergeGroupedSoupBins,+ projectLocalGroupedSoup,+} from './grouped-local-projection';+import { groupedDocument } from './grouped-test-fixtures';+import { soupItemKey } from './reconciliation';++const input: GroupedSoupInitialInput = {+ groupBy: { field: 'PROPERTY', propertyDefinitionId: 'status-definition' },+ sortMethod: 'UPDATED_AT',+};+const now = new Date('2026-10-10T12:00:00Z');++function snapshot(+ records: SoupItemFieldsFragment[],+ retainedKeys: string[] = []+) {+ return {+ revision: parseCacheRevision('1'),+ optimistic: false,+ keys: [...records.map(soupItemKey), ...retainedKeys],+ retainedKeys,+ records: records.map((record) => ({+ recordKey: soupItemKey(record),+ record,+ })),+ };+}++describe('loaded grouped pages', () => {+ it('merges overlapping pages without rescanning earlier item identities', () => {+ let identityReads = 0;+ const items = Array.from({ length: 200 }, (_, index) => ({+ ...groupedDocument(String(index)),+ get id() {+ identityReads += 1;+ return String(index);+ },+ }));+ const pages: GroupSoupQuery[] = Array.from({ length: 20 }, (_, index) => ({+ user: {+ id: 'viewer',+ groupSoup: {+ bins: [+ {+ key: 'todo',+ totalCount: 300 - index * 10,+ nextCursor: `page-${index + 1}`,+ items: items.slice(index * 10, index * 10 + 10),+ },+ ],+ },+ },+ }));+ pages.push({+ user: {+ id: 'viewer',+ groupSoup: {+ bins: [+ {+ key: 'todo',+ totalCount: 0,+ nextCursor: null,+ items: [groupedDocument('0', ['done'])],+ },+ ],+ },+ },+ });+ const merged = mergeGroupedSoupBins(pages);+ expect(identityReads).toBeLessThanOrEqual(items.length * 2);+ expect(merged).toHaveLength(1);+ expect(merged[0].items).toHaveLength(200);+ expect(merged[0].items[0]).toBe(items[0]);+ expect(merged[0].items[199]).toBe(items[199]);+ expect(merged[0].totalCount).toBe(300);+ expect(merged[0].nextCursor).toBe('page-1');+ expect(pages[0].user.groupSoup.bins[0].items).toHaveLength(10);+ });+});++describe('local grouped Soup projection', () => {+ it('moves normalized rows and removes filter nonmatches without losing unknown rows', () => {+ const a = groupedDocument('a');+ const b = groupedDocument('b');+ const unknown = groupedDocument('unknown');+ const baseline = [+ {+ key: 'todo',+ items: [a, b, unknown],+ totalCount: 30,+ nextCursor: 'more',+ },+ ];+ const next = groupedDocument('a', ['done']);+ const result = projectLocalGroupedSoup(+ baseline,+ new Map(),+ snapshot([next], [soupItemKey(unknown)]),+ input,+ now+ );+ expect(result).toEqual([+ { key: 'todo', items: [unknown], totalCount: 28, nextCursor: 'more' },+ { key: 'done', items: [next], totalCount: 1, nextCursor: null },+ ]);+ expect(result[1].items[0]).toBe(next);+ expect(baseline[0].items).toEqual([a, b, unknown]);+ });++ it('prefers selected then current display fields without changing baseline membership', () => {+ const selectedBefore = groupedDocument('selected');+ const retained = groupedDocument('retained');+ const missing = groupedDocument('missing');+ const excluded = groupedDocument('excluded');+ const selected = { ...selectedBefore, displayName: 'Selected name' };+ const current = {+ ...groupedDocument('retained', ['done']),+ displayName: 'Current name',+ };+ const currentItems = new Map([+ [+ soupItemKey(selectedBefore),+ { ...selectedBefore, displayName: 'Older name' },+ ],+ [soupItemKey(retained), current],+ [soupItemKey(excluded), excluded],+ ]);+ expect(+ projectLocalGroupedSoup(+ [+ {+ key: 'todo',+ items: [selectedBefore, retained, missing, excluded],+ totalCount: 30,+ nextCursor: 'more',+ },+ ],+ currentItems,+ snapshot([selected], [soupItemKey(retained), soupItemKey(missing)]),+ input,+ now+ )+ ).toEqual([+ {+ key: 'todo',+ items: [selected, current, missing],+ totalCount: 29,+ nextCursor: 'more',+ },+ ]);+ });++ it('uses complete old multi-value membership rather than just loaded bins for count deltas', () => {+ const before = groupedDocument('a', ['todo', 'done']);+ const after = groupedDocument('a', ['done']);+ expect(+ projectLocalGroupedSoup(+ [+ {+ key: 'todo',+ items: [before],+ totalCount: 30,+ nextCursor: 'todo-more',+ },+ { key: 'done', items: [], totalCount: 20, nextCursor: 'done-more' },+ ],+ new Map(),+ snapshot([after]),+ input,+ now+ )+ ).toEqual([+ { key: 'todo', items: [], totalCount: 29, nextCursor: 'todo-more' },+ { key: 'done', items: [after], totalCount: 20, nextCursor: 'done-more' },+ ]);+ });++ it('ignores bounded flat candidates outside the baseline', () => {+ expect(+ projectLocalGroupedSoup(+ [],+ new Map(),+ snapshot([groupedDocument('new')]),+ input,+ now+ )+ ).toEqual([]);+ });++ it('moves date buckets at midnight using the actual previous bin, not a recomputed old bucket', () => {+ const row = { ...groupedDocument('a'), updatedAt: '2026-10-09T23:30:00Z' };+ expect(+ projectLocalGroupedSoup(+ [{ key: 'today', items: [row], totalCount: 8, nextCursor: 'more' }],+ new Map(),+ snapshot([row]),+ { ...input, groupBy: { field: 'DATE' } },+ now+ )+ ).toEqual([+ { key: 'today', items: [], totalCount: 7, nextCursor: 'more' },+ { key: 'yesterday', items: [row], totalCount: 1, nextCursor: null },+ ]);+ });++ it.each([+ ['2026-10-11T00:00:00Z', 'this_week'],+ ['2026-10-10T00:00:00Z', 'today'],+ ['2026-10-09T00:00:00Z', 'yesterday'],+ ['2026-10-04T00:00:00Z', 'this_week'],+ ['2026-10-03T00:00:00Z', 'last_week'],+ ['2026-09-27T00:00:00Z', 'last_week'],+ ['2026-09-26T00:00:00Z', 'this_month'],+ ['2026-09-10T00:00:00Z', 'this_month'],+ ['2026-09-09T00:00:00Z', 'last_month'],+ ['2026-08-11T00:00:00Z', 'last_month'],+ ['2026-08-10T00:00:00Z', 'older'],+ ])('matches grouped SQL date boundaries for %s', (updatedAt, key) => {+ expect(+ localSoupGroupKeys(+ { ...groupedDocument('a'), updatedAt },+ { ...input, groupBy: { field: 'DATE' } },+ now+ )+ ).toEqual([key]);+ });++ it('matches canonical property entity scopes and preserves unsupported viewed-date grouping', () => {+ expect(+ localSoupGroupKeys(+ groupedDocument('a'),+ { ...input, groupBy: { ...input.groupBy, entityType: 'TASK' } },+ now+ )+ ).toEqual(['todo']);+ expect(+ localSoupGroupKeys(+ groupedDocument('a'),+ { ...input, groupBy: { ...input.groupBy, entityType: 'DOCUMENT' } },+ now+ )+ ).toEqual(['']);+ expect(+ localSoupGroupKeys(+ groupedDocument('a'),+ { groupBy: { field: 'DATE' }, sortMethod: 'VIEWED_UPDATED' },+ now+ )+ ).toBeUndefined();+ });++ it('distinguishes known empty property values from unknown grouping evidence', () => {+ expect(localSoupGroupKeys(groupedDocument('a', []), input, now)).toEqual([+ '',+ ]);+ expect(+ localSoupGroupKeys(+ { ...groupedDocument('a'), projectId: 'project-2' },+ { ...input, groupBy: { field: 'PROJECT' } },+ now+ )+ ).toEqual(['project-2']);+ const linked = groupedDocument('a');+ linked.properties[0].value = {+ __typename: 'GraphqlLinkPropertyValue',+ urls: ['https://example.com', 'https://example.com'],+ };+ expect(localSoupGroupKeys(linked, input, now)).toEqual([+ 'https://example.com',+ ]);+ linked.properties[0].value = {+ __typename: 'GraphqlNumberPropertyValue',+ numberValue: 3,+ };+ expect(localSoupGroupKeys(linked, input, now)).toEqual(['']);+ });+});diff --git a/apps/web/src/lib/queries/soup/graphql/grouped-local-projection.ts b/apps/web/src/lib/queries/soup/graphql/grouped-local-projection.tsnew file mode 100644index 0000000000..074f33fcc1--- /dev/null+++ b/apps/web/src/lib/queries/soup/graphql/grouped-local-projection.ts@@ -0,0 +1,205 @@+import { match } from 'ts-pattern';+import type {+ GroupedSoupInitialInput,+ GroupSoupQuery,+ SoupItemFieldsFragment,+} from '../../../service-clients/service-storage/graphql/generated/graphql';+import { reconcileGroupedSoup } from '../grouped/reconciliation';+import type { LiveQuerySnapshot } from './create-predicate-query';+import { soupItemKey } from './reconciliation';++export type GroupedSoupBin =+ GroupSoupQuery['user']['groupSoup']['bins'][number];++/** Only complete selected grouping evidence can replace server membership. */+export function localSoupGroupKeys(+ item: SoupItemFieldsFragment,+ input: GroupedSoupInitialInput,+ now: Date+): string[] | undefined {+ return (+ match(input.groupBy.field)+ // Entity type cannot change for a normalized identity. Retain its server key.+ .with('ENTITY_TYPE', () => undefined)+ .with('PROJECT', () => {+ if ('projectId' in item) return [item.projectId ?? ''];+ if (item.__typename === 'GraphqlSoupProject')+ return [item.parentId ?? ''];+ return undefined;+ })+ .with('PROPERTY', () => {+ if (!('properties' in item)) return undefined;+ const entityType = match(item)+ .with({ __typename: 'GraphqlSoupDocument' }, (item) =>+ item.subType?.__typename === 'GraphqlTaskSubType'+ ? 'TASK'+ : 'DOCUMENT'+ )+ .with({ __typename: 'GraphqlSoupEmailThread' }, () => 'THREAD')+ .with({ __typename: 'GraphqlSoupCrmCompany' }, () => 'COMPANY')+ .with({ __typename: 'GraphqlSoupCall' }, () => 'CALL_RECORD')+ .otherwise((item) => item.entityType);+ if (input.groupBy.entityType && input.groupBy.entityType !== entityType)+ return [''];+ const property = item.properties.find(+ (property) =>+ property.propertyDefinitionId === input.groupBy.propertyDefinitionId+ );+ const value = property?.value;+ if (!value) return [''];+ const keys = match(value)+ .with(+ { __typename: 'GraphqlSelectOptionPropertyValue' },+ (value) => value.optionIds+ )+ .with(+ { __typename: 'GraphqlEntityReferencePropertyValue' },+ (value) => value.references.map((reference) => reference.entityId)+ )+ .with(+ { __typename: 'GraphqlLinkPropertyValue' },+ (value) => value.urls+ )+ // The server's lateral join puts scalar property values in Not Set.+ .otherwise(() => []);+ return keys.length ? [...new Set(keys)] : [''];+ })+ .with('DATE', () => {+ if (+ input.sortMethod !== 'CREATED_AT' &&+ input.sortMethod !== 'UPDATED_AT'+ )+ return undefined;+ const value =+ input.sortMethod === 'CREATED_AT'+ ? 'createdAt' in item+ ? item.createdAt+ : undefined+ : 'updatedAt' in item+ ? item.updatedAt+ : undefined;+ if (!value) return undefined;+ const timestamp = new Date(value);+ if (!Number.isFinite(timestamp.getTime())) return undefined;+ const day = (date: Date) =>+ Date.UTC(+ date.getUTCFullYear(),+ date.getUTCMonth(),+ date.getUTCDate()+ );+ const age = (day(now) - day(timestamp)) / 86_400_000;+ // Matches the grouped SQL CASE, including future timestamps (this_week).+ return [+ age === 0+ ? 'today'+ : age === 1+ ? 'yesterday'+ : age <= 6+ ? 'this_week'+ : age <= 13+ ? 'last_week'+ : age <= 30+ ? 'this_month'+ : age <= 60+ ? 'last_month'+ : 'older',+ ];+ })+ .exhaustive()+ );+}++/** Merge page evidence once per normalized identity; initial metadata wins. */+export function mergeGroupedSoupBins(+ pages: readonly GroupSoupQuery[]+): GroupedSoupBin[] {+ const bins = new Map<+ string,+ { metadata: GroupedSoupBin; items: Map<string, SoupItemFieldsFragment> }+ >();+ for (const page of pages) {+ for (const bin of page.user.groupSoup.bins) {+ let merged = bins.get(bin.key);+ if (!merged) {+ merged = { metadata: bin, items: new Map() };+ bins.set(bin.key, merged);+ }+ for (const item of bin.items) {+ const key = soupItemKey(item);+ if (!merged.items.has(key)) merged.items.set(key, item);+ }+ }+ }+ return [...bins.values()].map(({ metadata, items }) => ({+ ...metadata,+ items: [...items.values()],+ }));+}++/** Apply the maintained filter result without treating bounded candidates as complete bins. */+export function projectLocalGroupedSoup(+ baseline: readonly GroupedSoupBin[],+ currentItems: ReadonlyMap<string, SoupItemFieldsFragment>,+ snapshot: LiveQuerySnapshot<SoupItemFieldsFragment>,+ input: GroupedSoupInitialInput,+ now: Date,+ untrustedKeys: ReadonlySet<string> = new Set()+): GroupedSoupBin[] {+ const original = new Map(+ baseline.flatMap((bin) =>+ bin.items.map((item) => [soupItemKey(item), item] as const)+ )+ );+ const oldKeys = new Map<string, string[]>();+ for (const bin of baseline)+ for (const item of bin.items) {+ const key = soupItemKey(item);+ oldKeys.set(key, [...(oldKeys.get(key) ?? []), bin.key]);+ }+ const previousKeys = (key: string, item: SoupItemFieldsFragment) =>+ input.groupBy.field === 'PROPERTY'+ ? untrustedKeys.has(key)+ ? undefined+ : localSoupGroupKeys(item, input, now)+ : oldKeys.get(key);+ const selected = new Map(+ snapshot.records.map((row) => [row.recordKey, row.record])+ );+ const surviving = new Set(snapshot.keys);+ const evidence = snapshot.keys.flatMap((key) => {+ const previous = original.get(key);+ if (!previous) return [];+ const current = selected.get(key);+ return [+ {+ id: key,+ previousGroupKeys: previousKeys(key, previous),+ groupKeys: current+ ? localSoupGroupKeys(current, input, now)+ : undefined,+ },+ ];+ });+ for (const [key, item] of original) {+ if (!surviving.has(key))+ evidence.push({+ id: key,+ previousGroupKeys: previousKeys(key, item),+ groupKeys: [],+ });+ }+ const bins = reconcileGroupedSoup(+ baseline.map((bin) => ({ ...bin, itemIds: bin.items.map(soupItemKey) })),+ evidence+ );+ return bins.map((bin) => ({+ key: bin.key,+ totalCount: bin.totalCount,+ nextCursor: bin.nextCursor,+ items: bin.itemIds.flatMap((key) => {+ const item =+ selected.get(key) ?? currentItems.get(key) ?? original.get(key);+ return item ? [item] : [];+ }),+ }));+}diff --git a/apps/web/src/lib/queries/soup/graphql/grouped-page-evidence.ts b/apps/web/src/lib/queries/soup/graphql/grouped-page-evidence.tsnew file mode 100644index 0000000000..3396493f6e--- /dev/null+++ b/apps/web/src/lib/queries/soup/graphql/grouped-page-evidence.ts@@ -0,0 +1,111 @@+import { fireAndForget } from '@core/util/fireAndForget';+import type { OperationResult } from '@urql/core';+import { type Accessor, createSignal, onCleanup } from 'solid-js';+import { normalizedCacheResultMetadata } from '../../../graphql-cache/exchange/normalized-cache-exchange';+import type {+ GroupSoupQuery,+ GroupSoupQueryVariables,+} from '../../../service-clients/service-storage/graphql/generated/graphql';++import { soupItemKey } from './reconciliation';++type Evidence = {+ scope: string;+ data: GroupSoupQuery;+ persisted: boolean;+ network: boolean;+};++/** Preserve server membership separately from later normalized field/link writes. */+export function createGroupedPageEvidence(scope: Accessor<string>) {+ const [pages, setPages] = createSignal<ReadonlyMap<number, Evidence>>(+ new Map()+ );+ let disposed = false;+ onCleanup(() => {+ disposed = true;+ });++ async function acknowledge(+ index: number,+ page: Evidence,+ persistence: Promise<unknown>+ ) {+ try {+ const revision = await persistence;+ if (+ revision === undefined ||+ disposed ||+ scope() !== page.scope ||+ pages().get(index) !== page+ )+ return;+ setPages((previous) =>+ new Map(previous).set(index, { ...page, persisted: true })+ );+ } catch {+ // A failed cache write cannot supersede the usable server response.+ }+ }++ return {+ accept(+ result: OperationResult<GroupSoupQuery, GroupSoupQueryVariables>,+ index: number,+ requestScope: string+ ) {+ if (+ disposed ||+ requestScope !== scope() ||+ !result.data ||+ result.hasNext ||+ result.error+ )+ return;+ const metadata = normalizedCacheResultMetadata(result);+ const previous = pages().get(index);+ if (+ metadata?.source !== 'live-network' &&+ previous?.scope === requestScope &&+ previous.data.user.id === result.data.user.id+ )+ return;+ const persistence =+ metadata?.source === 'live-network' ? metadata.persistence : undefined;+ const page = {+ scope: requestScope,+ data: result.data,+ persisted: !persistence,+ network: metadata?.source === 'live-network',+ };+ setPages((previous) => new Map(previous).set(index, page));+ if (persistence)+ fireAndForget(+ acknowledge(index, page, persistence),+ 'soup-grouped-evidence'+ );+ },+ data: () => {+ const current = [...pages().entries()].filter(+ ([, page]) => page.scope === scope()+ );+ if (!current.length || current.some(([, page]) => !page.persisted))+ return undefined;+ return current+ .sort(([left], [right]) => left - right)+ .map(([, page]) => page.data);+ },+ // A restored page can combine old bin membership with newer normalized+ // properties. Only raw network rows prove complete *previous* membership.+ untrustedKeys: () =>+ new Set(+ [...pages().values()]+ .filter((page) => page.scope === scope() && !page.network)+ .flatMap((page) =>+ page.data.user.groupSoup.bins.flatMap((bin) =>+ bin.items.map(soupItemKey)+ )+ )+ ),+ };+}diff --git a/apps/web/src/lib/queries/soup/graphql/grouped-test-fixtures.ts b/apps/web/src/lib/queries/soup/graphql/grouped-test-fixtures.tsnew file mode 100644index 0000000000..abf454b263--- /dev/null+++ b/apps/web/src/lib/queries/soup/graphql/grouped-test-fixtures.ts@@ -0,0 +1,39 @@+import type { SoupItemFieldsFragment } from '../../../service-clients/service-storage/graphql/generated/graphql';++export function groupedDocument(+ id: string,+ optionIds = ['todo']+): Extract<SoupItemFieldsFragment, { __typename: 'GraphqlSoupDocument' }> {+ return {+ __typename: 'GraphqlSoupDocument',+ id,+ entityType: 'DOCUMENT',+ displayName: id,+ documentName: id,+ ownerId: 'viewer',+ cacheProjection: null,+ frecencyScore: null,+ fileType: 'md',+ projectId: null,+ createdAt: '2026-10-01T00:00:00Z',+ updatedAt: '2026-10-01T00:00:00Z',+ viewedAt: null,+ deletedAt: null,+ subType: { __typename: 'GraphqlTaskSubType', isCompleted: false },+ isFavorited: false,+ notifications: [],+ properties: [+ {+ id: `property-${id}`,+ propertyDefinitionId: 'status-definition',+ displayName: 'Status',+ dataType: 'SELECT_STRING',+ isMultiSelect: true,+ specificEntityType: null,+ isSystem: false,+ isMetadata: false,+ value: { __typename: 'GraphqlSelectOptionPropertyValue', optionIds },+ },+ ],+ };+}diff --git a/apps/web/src/lib/queries/soup/graphql/items.ts b/apps/web/src/lib/queries/soup/graphql/items.tsindex 53a84d39c1..f2e3a57511 100644--- a/apps/web/src/lib/queries/soup/graphql/items.ts+++ b/apps/web/src/lib/queries/soup/graphql/items.ts@@ -16,6 +16,7 @@ import {mapSoupPageToEntityList,} from '../transform-utils';import { makeGraphqlSoupInput } from './ast';+import type { GroupedSoupPaging } from './create-grouped-soup-live-query';import {createSoupLiveQuery,type SoupLiveQueryOptions,@@ -36,6 +37,7 @@ export type GraphqlSoupAstItemsQueryOptions = SoupLiveQueryOptions & {};export type GraphqlSoupAstItemsQuery = {+ groupedPaging?: GroupedSoupPaging;localRevision?: Accessor<CacheRevision | undefined>;localOptimistic?: Accessor<boolean>;data: Accessor<SoupAstItemsData | undefined>;diff --git a/apps/web/src/lib/queries/soup/grouped/create-grouped-soup-queries.ts b/apps/web/src/lib/queries/soup/grouped/create-grouped-soup-queries.tsindex 3ddbaa3619..77c314d5dc 100644--- a/apps/web/src/lib/queries/soup/grouped/create-grouped-soup-queries.ts+++ b/apps/web/src/lib/queries/soup/grouped/create-grouped-soup-queries.ts@@ -53,6 +53,10 @@ export type GroupQueryData = {};export type CreateGroupedSoupQueriesArgs = {+ groupedPaging?: Accessor<+ | import('../graphql/create-grouped-soup-live-query').GroupedSoupPaging+ | undefined+ >;initialPage: Accessor<InitialGroupPage | undefined>;groupByField: Accessor<GroupByField | undefined>;soupParams: Accessor<SoupParams>;@@ -214,10 +218,8 @@ export function createGroupedSoupQueries(args: CreateGroupedSoupQueriesArgs) {GroupQueryData | undefined>(configs);const graphqlQueries = createGraphqlGroupedSoupQueries({+ groupedPaging: args.groupedPaging,initialPage: args.initialPage,- groupByField: args.groupByField,- soupParams: args.soupParams,- soupBody: args.soupBody,enabled: () => usesGraphql() && args.queryOptions().enabled === true,itemFilter: () => args.queryOptions().meta?.itemFilter,});diff --git a/apps/web/src/lib/queries/soup/grouped/graphql/create-graphql-grouped-soup-queries.test.ts b/apps/web/src/lib/queries/soup/grouped/graphql/create-graphql-grouped-soup-queries.test.tsindex 07e1a6eedf..a58fc950a6 100644--- a/apps/web/src/lib/queries/soup/grouped/graphql/create-graphql-grouped-soup-queries.test.ts+++ b/apps/web/src/lib/queries/soup/grouped/graphql/create-graphql-grouped-soup-queries.test.ts@@ -195,13 +195,8 @@ function setup() {});grouped = createGraphqlGroupedSoupQueries({+ groupedPaging: () => parent.groupedPaging,initialPage,- groupByField: () => ({- type: 'property',- propertyDefinitionId: 'status-definition',- }),- soupParams: () => ({ limit: 2, sort_method: 'updated_at' }),- soupBody: () => ({}),enabled: () => true,itemFilter: () => undefined,});diff --git a/apps/web/src/lib/queries/soup/grouped/graphql/create-graphql-grouped-soup-queries.ts b/apps/web/src/lib/queries/soup/grouped/graphql/create-graphql-grouped-soup-queries.tsindex 95a4599bd8..1a1a99050f 100644--- a/apps/web/src/lib/queries/soup/grouped/graphql/create-graphql-grouped-soup-queries.ts+++ b/apps/web/src/lib/queries/soup/grouped/graphql/create-graphql-grouped-soup-queries.ts@@ -1,73 +1,22 @@-import { createUrqlInfiniteQuery } from '@app/lib/urql-solid';import type { EntityData } from '@entity';-import { createKeyedProjection } from '@queries/soup/create-keyed-projection';+import { keyArray } from '@solid-primitives/keyed';+import { type Accessor, createMemo } from 'solid-js';+import { useInstructionsMdIdQuery } from '../../../storage/instructions-md';+import { createKeyedProjection } from '../../create-keyed-projection';+import type { GroupedSoupPaging } from '../../graphql/create-grouped-soup-live-query';import {- makeGraphqlGroupedSoupContinuationInput,- makeGraphqlGroupedSoupInput,-} from '@queries/soup/graphql/ast';-import { registerGroupedSoupContinuation } from '@queries/soup/grouped/graphql-operation-registry';-import type { GroupByField, GroupMeta } from '@queries/soup/grouped/types';-import type {- SoupApiItemFilter,- SoupAstBody,- SoupAstItemsGroupedPage,- SoupParams,-} from '@queries/soup/items';+ usePendingGraphqlSoupDeleteIds,+ withoutPendingSoupEntities,+} from '../../graphql/optimistic-deletions';+import type { SoupApiItemFilter, SoupAstItemsGroupedPage } from '../../items';import {isDisplayableSoupItem,isInstructionsMdDoc,mapApiSoupItemToEntity,-} from '@queries/soup/transform-utils';-import { useInstructionsMdIdQuery } from '@queries/storage/instructions-md';-import {- type GroupedSoupInput,- GroupSoupDocument,- type GroupSoupQuery,- type GroupSoupQueryVariables,-} from '@service-storage/graphql/generated/graphql';-import {- getGraphqlSoupClient,- mapGraphqlGroupedSoupPage,-} from '@service-storage/graphql-soup';-import {- type Accessor,- createComputed,- createMemo,- createRoot,- createSignal,- on,- onCleanup,-} from 'solid-js';--import { registerGraphqlSoupRevalidations } from '../../graphql/active-queries';-import {- usePendingGraphqlSoupDeleteIds,- withoutPendingSoupEntities,-} from '../../graphql/optimistic-deletions';--export type GroupQueryData = {- entities: EntityData[];-};--type GraphqlGroupConfig = {- key: string;- group: GroupMeta;- field: GroupByField;- initialInput: GroupedSoupInput;- enabled: boolean;- itemFilter: SoupApiItemFilter | undefined;-};+} from '../../transform-utils';+import type { GroupMeta } from '../types';-type GraphqlGroupQuery = {- key: string;- data: Accessor<GroupQueryData | undefined>;- hasNextPage: Accessor<boolean>;- isFetchingNextPage: Accessor<boolean>;- error: Accessor<Error | null>;- fetchNextPage: () => Promise<void>;- resetToInitialPage: () => void;- dispose: () => void;-};+export type GroupQueryData = { entities: EntityData[] };type GraphqlGroupedInitialPage = {cachedMail?: boolean;@@ -77,395 +26,65 @@ type GraphqlGroupedInitialPage = {type CreateGraphqlGroupedSoupQueriesArgs = {initialPage: Accessor<GraphqlGroupedInitialPage | undefined>;- groupByField: Accessor<GroupByField | undefined>;- soupParams: Accessor<SoupParams>;- soupBody: Accessor<SoupAstBody>;+ groupedPaging?: Accessor<GroupedSoupPaging | undefined>;enabled: Accessor<boolean>;itemFilter: Accessor<SoupApiItemFilter | undefined>;};-function groupPage(- data: GroupSoupQuery,- group: GroupMeta-):- | {- items: ReturnType<typeof mapGraphqlGroupedSoupPage>['items'];- group: GroupMeta;- }- | undefined {- const page = mapGraphqlGroupedSoupPage(data);- const bin = page.groups.find((candidate) => candidate.key === group.key);- if (!bin) return;- return {- items: page.items,- group: {- ...group,- totalCount: bin.totalCount,- itemIds: bin.itemIds,- nextCursor: bin.nextCursor,- },- };-}--/** Derives initial bins from the live parent and observes loaded continuations. */+/** Group readers project the parent's combined pages; they never own a second cursor chain. */export function createGraphqlGroupedSoupQueries(args: CreateGraphqlGroupedSoupQueriesArgs-): {- list: Accessor<GraphqlGroupQuery[]>;- map: Accessor<Map<string, GraphqlGroupQuery>>;- resetToInitialPage: () => void;-} {- const instructionsIdQuery = useInstructionsMdIdQuery();+) {+ const instructions = useInstructionsMdIdQuery();const pendingDeleteIds = usePendingGraphqlSoupDeleteIds();-- const mapItems = (- items: SoupAstItemsGroupedPage['items'],- itemIds: readonly string[],- itemFilter: SoupApiItemFilter | undefined- ): EntityData[] =>- itemIds.flatMap((id) => {- const item = items[id];- if (!item || !isDisplayableSoupItem(item)) return [];- if (itemFilter && !itemFilter(item)) return [];- if (isInstructionsMdDoc(item, instructionsIdQuery)) return [];- return [mapApiSoupItemToEntity(item)];- });-- const initialItems = createKeyedProjection(+ const rows = createKeyedProjection(() => Object.entries(args.initialPage()?.items ?? {}),([id]) => id,- ([id, item]) => ({- id,- entity: mapItems({ [id]: item }, [id], args.itemFilter())[0],- })- );- const initialEntities = createMemo(- () => new Map(initialItems().map(({ id, entity }) => [id, entity]))- );-- const configs = createMemo<GraphqlGroupConfig[]>(() => {- const field = args.groupByField();- const initialPage = args.initialPage();- if (!field || !initialPage) return [];-- let initialInput: GroupedSoupInput;- try {- initialInput = makeGraphqlGroupedSoupInput({- params: args.soupParams(),- body: args.soupBody(),- groupBy: field,- });- } catch {- return [];+ ([id, item]) => {+ const filter = args.itemFilter();+ const entity =+ isDisplayableSoupItem(item) &&+ (!filter || filter(item)) &&+ !isInstructionsMdDoc(item, instructions)+ ? mapApiSoupItemToEntity(item)+ : undefined;+ return { id, entity };}-- const enabled = args.enabled();- const itemFilter = args.itemFilter();- return initialPage.groups.map((group) => ({- key: group.key,- group,- field,- initialInput,- enabled,- itemFilter,- }));- });-- const queryByKey = new Map<string, GraphqlGroupQuery>();- const [revision, setRevision] = createSignal(0);-- const createGroupQuery = (- key: string,- initialConfig: GraphqlGroupConfig- ): GraphqlGroupQuery => {- let dispose: (() => void) | undefined;-- const queryResult = createRoot((rootDispose) => {- dispose = rootDispose;- const getConfig = createMemo(- () => configs().find((config) => config.key === key) ?? initialConfig- );- const [continuationRevision, setContinuationRevision] = createSignal(0);- const [isFetchingFirstPage, setIsFetchingFirstPage] = createSignal(false);-- const createContinuationQuery = (firstCursor: string) => {- let continuationDispose: (() => void) | undefined;-- return createRoot((continuationRootDispose) => {- continuationDispose = continuationRootDispose;- const [activated, setActivated] = createSignal(false);- let initialPageDidSettle = false;- let settleInitialPage = () => undefined;- const initialPageSettled = new Promise<void>((resolve) => {- settleInitialPage = () => {- if (initialPageDidSettle) return;- initialPageDidSettle = true;- resolve();- };- });-- const query = createUrqlInfiniteQuery<- GroupSoupQuery,- GroupSoupQueryVariables,- string,- GroupQueryData & { pageParams: readonly string[] }- >(() => {- const config = getConfig();- return {- query: GroupSoupDocument,- client: getGraphqlSoupClient(),- initialPageParam: firstCursor,- variables: (cursor) => {- const input = makeGraphqlGroupedSoupContinuationInput({- groupBy: config.field,- groupKey: config.key,- cursor,- });- registerGroupedSoupContinuation(config.initialInput, input);- return { input };- },- getNextPageParam: (lastPage) =>- groupPage(lastPage, config.group)?.group.nextCursor,- select: ({ pages, pageParams }) => ({- pageParams,- entities: pages.flatMap((page) => {- const selected = groupPage(page, config.group);- return selected- ? mapItems(- selected.items,- selected.group.itemIds,- config.itemFilter- )- : [];- }),- }),- enabled: activated() && config.enabled,- requestPolicy: 'cache-first',- keepPreviousData: false,- };- });-- onCleanup(- registerGraphqlSoupRevalidations(() => {- if (!query.isEnabled) return [];- const config = getConfig();- const cursors = new Set([- firstCursor,- ...(query.data?.pageParams ?? []),- ]);- return [...cursors].map((cursor) => ({- document: GroupSoupDocument,- variables: {- input: makeGraphqlGroupedSoupContinuationInput({- groupBy: config.field,- groupKey: config.key,- cursor,- }),- },- }));- })- );-- createComputed(() => {- if (!activated()) return;- if (- !getConfig().enabled ||- (query.isFetched && !query.isFetching)- ) {- settleInitialPage();- }- });- onCleanup(settleInitialPage);-- return {- firstCursor,- query,- activate: () => {- setActivated(true);- return initialPageSettled;- },- dispose: () => continuationDispose?.(),- };- });- };-- let continuation: ReturnType<typeof createContinuationQuery> | undefined;- let firstPagePromise: Promise<void> | undefined;-- const getContinuation = () => {- continuationRevision();- return continuation;- };-- const disposeContinuation = () => {- continuation?.dispose();- continuation = undefined;- firstPagePromise = undefined;- setIsFetchingFirstPage(false);- setContinuationRevision((value) => value + 1);- };-- const cachedMail = createMemo(- () => args.initialPage()?.cachedMail === true- );- const initialData = createMemo<GroupQueryData | undefined>(() => {- const config = getConfig();- const initialPage = args.initialPage();- if (!initialPage) return;- const group = initialPage.groups.find(- (candidate) => candidate.key === config.key- );- if (!group) return;- return {- entities: group.itemIds.flatMap((id) => {- const entity = initialEntities().get(id);- return entity ? [entity] : [];- }),- };- });-- const data = createMemo<GroupQueryData | undefined>(() => {- const initial = initialData();- if (!initial) return;- const continued = cachedMail()- ? undefined- : getContinuation()?.query.data;- const combined = continued- ? { entities: [...initial.entities, ...continued.entities] }- : initial;- const entities = withoutPendingSoupEntities(- combined.entities,- pendingDeleteIds()- );- return entities === combined.entities ? combined : { entities };- });-- const trackFirstPage = (action: Promise<unknown>): Promise<void> => {- setIsFetchingFirstPage(true);- const tracked = action- .then(() => undefined)- .finally(() => {- if (firstPagePromise !== tracked) return;- firstPagePromise = undefined;- setIsFetchingFirstPage(false);- });- firstPagePromise = tracked;- return tracked;- };-- const startContinuation = async (cursor: string): Promise<void> => {- continuation = createContinuationQuery(cursor);- setContinuationRevision((value) => value + 1);- await trackFirstPage(continuation.activate());- };-- const fetchNextPage = async (): Promise<void> => {- const config = getConfig();- if (!config.enabled || cachedMail()) return;- if (firstPagePromise) return firstPagePromise;-- let current = getContinuation();- if (!current) {- const cursor = config.group.nextCursor;- if (cursor === null) return;- await startContinuation(cursor);- return;- }-- if (current.query.data === undefined) {- const cursor = getConfig().group.nextCursor;- if (cursor === null) return;- if (cursor !== current.firstCursor) {- disposeContinuation();- await startContinuation(cursor);- return;- }- await trackFirstPage(current.query.refetch());- return;- }-- await current.query.fetchNextPage();- };-- createComputed(on(cachedMail, () => disposeContinuation()));- onCleanup(disposeContinuation);-- return {- key,- data,- error: () => getContinuation()?.query.error ?? null,- hasNextPage: () => {- if (cachedMail()) return false;- const current = getContinuation();- return current?.query.data === undefined- ? getConfig().group.nextCursor !== null- : current.query.hasNextPage;- },- isFetchingNextPage: () =>- isFetchingFirstPage() ||- (getContinuation()?.query.isFetchingNextPage ?? false),- fetchNextPage,- resetToInitialPage: disposeContinuation,- };- });-- return {- ...queryResult,- dispose: () => dispose?.(),- };- };-- createComputed(- on(- () => configs().map((config) => config.key),- (keys) => {- const activeKeys = new Set(keys);- let changed = false;-- for (const [key, query] of queryByKey) {- if (activeKeys.has(key)) continue;- query.dispose();- queryByKey.delete(key);- changed = true;- }-- for (const key of keys) {- if (queryByKey.has(key)) continue;- const config = configs().find((candidate) => candidate.key === key);- if (!config) continue;- queryByKey.set(key, createGroupQuery(key, config));- changed = true;- }-- if (changed) setRevision((value) => value + 1);- }- ));-- onCleanup(() => {- for (const query of queryByKey.values()) query.dispose();- queryByKey.clear();- });-- const list = createMemo(() => {- revision();- return configs().flatMap((config) => {- const query = queryByKey.get(config.key);- return query ? [query] : [];- });- });- const map = createMemo(() =>- list().reduce((queries, query) => {- queries.set(query.key, query);- return queries;- }, new Map<string, GraphqlGroupQuery>())+ const byId = createMemo(+ () => new Map(rows().map((row) => [row.id, row.entity]))+ );+ const paging = () =>+ args.initialPage()?.cachedMail ? undefined : args.groupedPaging?.();+ const list = keyArray(+ () => args.initialPage()?.groups ?? [],+ (group) => group.key,+ (group) => ({+ key: group().key,+ data: createMemo(+ (): GroupQueryData => ({+ entities: withoutPendingSoupEntities(+ group().itemIds.flatMap((id) => {+ const entity = byId().get(id);+ return entity ? [entity] : [];+ }),+ pendingDeleteIds()+ ),+ })+ ),+ hasNextPage: () => paging()?.hasNextPage(group().key) ?? false,+ isFetchingNextPage: () =>+ paging()?.isFetchingNextPage(group().key) ?? false,+ error: () => paging()?.error(group().key) ?? null,+ fetchNextPage: async () => {+ if (args.enabled()) await paging()?.fetchNextPage(group().key);+ },+ resetToInitialPage: () => paging()?.resetToInitialPage(),+ }));-return {list,- map,- resetToInitialPage: () => {- for (const query of list()) query.resetToInitialPage();- },+ map: createMemo(() => new Map(list().map((query) => [query.key, query]))),+ resetToInitialPage: () => paging()?.resetToInitialPage(),};}diff --git a/apps/web/src/lib/queries/soup/grouped/reconciliation.test.ts b/apps/web/src/lib/queries/soup/grouped/reconciliation.test.tsnew file mode 100644index 0000000000..59b3df24ff--- /dev/null+++ b/apps/web/src/lib/queries/soup/grouped/reconciliation.test.ts@@ -0,0 +1,218 @@+import { describe, expect, it } from 'vitest';+import { reconcileGroupedSoup } from './reconciliation';++const baseline = [+ {+ key: 'todo',+ totalCount: 30,+ itemIds: ['a', 'b'],+ nextCursor: 'todo-page-2',+ },+ { key: 'done', totalCount: 10, itemIds: ['c'], nextCursor: 'done-page-2' },+];++describe('baseline-backed grouped Soup reconciliation', () => {+ it('moves loaded rows without replacing server totals with loaded counts', () => {+ expect(+ reconcileGroupedSoup(baseline, [+ { id: 'a', previousGroupKeys: ['todo'], groupKeys: ['done'] },+ { id: 'b', previousGroupKeys: ['todo'], groupKeys: ['todo'] },+ { id: 'c', previousGroupKeys: ['done'], groupKeys: ['done'] },+ ])+ ).toEqual([+ {+ key: 'todo',+ totalCount: 29,+ itemIds: ['b'],+ nextCursor: 'todo-page-2',+ },+ {+ key: 'done',+ totalCount: 11,+ itemIds: ['a', 'c'],+ nextCursor: 'done-page-2',+ },+ ]);+ });++ it('deduplicates overlapping pages and changes a count only once', () => {+ expect(+ reconcileGroupedSoup(+ [+ ...baseline,+ {+ key: 'todo',+ totalCount: 30,+ itemIds: ['b', 'd'],+ nextCursor: 'todo-page-3',+ },+ ],+ [+ { id: 'b', previousGroupKeys: ['todo'], groupKeys: ['done'] },+ { id: 'b', previousGroupKeys: ['todo'], groupKeys: ['done'] },+ { id: 'd', previousGroupKeys: ['todo'], groupKeys: ['done'] },+ ]+ )+ ).toEqual([+ {+ key: 'todo',+ totalCount: 28,+ itemIds: ['a'],+ nextCursor: 'todo-page-2',+ },+ {+ key: 'done',+ totalCount: 12,+ itemIds: ['b', 'd', 'c'],+ nextCursor: 'done-page-2',+ },+ ]);+ });++ it('preserves unknown membership and rows not covered by an evaluation', () => {+ expect(+ reconcileGroupedSoup(baseline, [+ { id: 'a', previousGroupKeys: ['todo', 'done'] },+ ])+ ).toEqual(baseline);+ });++ it('removes proven nonmatches from all known memberships', () => {+ expect(+ reconcileGroupedSoup(baseline, [+ { id: 'a', previousGroupKeys: ['todo', 'done'], groupKeys: [] },+ ])+ ).toEqual([+ {+ key: 'todo',+ totalCount: 29,+ itemIds: ['b'],+ nextCursor: 'todo-page-2',+ },+ { key: 'done', totalCount: 9, itemIds: ['c'], nextCursor: 'done-page-2' },+ ]);+ });++ it('does not count an already-existing but unloaded multi-value membership twice', () => {+ expect(+ reconcileGroupedSoup(baseline, [+ {+ id: 'a',+ previousGroupKeys: ['todo', 'done'],+ groupKeys: ['todo', 'done', 'done'],+ },+ ])+ ).toEqual([+ {+ key: 'todo',+ totalCount: 30,+ itemIds: ['a', 'b'],+ nextCursor: 'todo-page-2',+ },+ {+ key: 'done',+ totalCount: 10,+ itemIds: ['a', 'c'],+ nextCursor: 'done-page-2',+ },+ ]);+ });++ it('does not infer an increase from incomplete previous membership', () => {+ expect(+ reconcileGroupedSoup(baseline, [{ id: 'a', groupKeys: ['done'] }])+ ).toEqual([+ {+ key: 'todo',+ totalCount: 29,+ itemIds: ['b'],+ nextCursor: 'todo-page-2',+ },+ {+ key: 'done',+ totalCount: 10,+ itemIds: ['a', 'c'],+ nextCursor: 'done-page-2',+ },+ ]);+ });++ it('creates a destination bin, including Not Set, without a fabricated cursor', () => {+ expect(+ reconcileGroupedSoup(baseline, [+ { id: 'a', previousGroupKeys: ['todo'], groupKeys: [''] },+ ])+ ).toEqual([+ {+ key: 'todo',+ totalCount: 29,+ itemIds: ['b'],+ nextCursor: 'todo-page-2',+ },+ {+ key: 'done',+ totalCount: 10,+ itemIds: ['c'],+ nextCursor: 'done-page-2',+ },+ { key: '', totalCount: 1, itemIds: ['a'], nextCursor: null },+ ]);+ });++ it('does not claim global candidate coverage from a bounded flat query', () => {+ expect(+ reconcileGroupedSoup(baseline, [+ { id: 'not-loaded', previousGroupKeys: [], groupKeys: ['todo'] },+ ])+ ).toEqual(baseline);+ expect(+ reconcileGroupedSoup([], [{ id: 'not-loaded', groupKeys: ['todo'] }])+ ).toEqual([]);+ });++ it('keeps empty bins and their server cursors when unseen rows remain', () => {+ expect(+ reconcileGroupedSoup(baseline, [+ { id: 'a', groupKeys: [] },+ { id: 'b', groupKeys: [] },+ ])+ ).toEqual([+ { key: 'todo', totalCount: 28, itemIds: [], nextCursor: 'todo-page-2' },+ {+ key: 'done',+ totalCount: 10,+ itemIds: ['c'],+ nextCursor: 'done-page-2',+ },+ ]);+ });++ it('uses evaluated row ordering without a global limit or mutating the baseline', () => {+ const original = structuredClone(baseline);+ expect(+ reconcileGroupedSoup(baseline, [+ { id: 'b', groupKeys: ['todo'] },+ { id: 'a', groupKeys: ['todo'] },+ ])[0].itemIds+ ).toEqual(['b', 'a']);+ expect(baseline).toEqual(original);+ });++ it('never counts fewer than the visible unique rows', () => {+ expect(+ reconcileGroupedSoup(+ [+ {+ key: 'todo',+ totalCount: 0,+ itemIds: ['a', 'b', 'b'],+ nextCursor: null,+ },+ ],+ [{ id: 'a', groupKeys: [] }]+ )+ ).toEqual([+ { key: 'todo', totalCount: 1, itemIds: ['b'], nextCursor: null },+ ]);+ });+});diff --git a/apps/web/src/lib/queries/soup/grouped/reconciliation.ts b/apps/web/src/lib/queries/soup/grouped/reconciliation.tsnew file mode 100644index 0000000000..eb7ae8d310--- /dev/null+++ b/apps/web/src/lib/queries/soup/grouped/reconciliation.ts@@ -0,0 +1,88 @@+import type { GroupMeta } from './types';++/** Membership metadata only; presentation resolves labels after reconciliation. */+export type ReconciledSoupBin = Pick<+ GroupMeta,+ 'key' | 'totalCount' | 'itemIds' | 'nextCursor'+>;++/** One loaded row, in locally evaluated sort order. */+export type GroupedSoupMembership = {+ id: string;+ /** Complete membership at the server baseline, when known. Not just loaded bins. */+ previousGroupKeys?: readonly string[];+ /** Undefined is unknown; [] proves exclusion from this view. */+ groupKeys?: readonly string[];+};++function collectGroupedBaseline(baseline: readonly ReconciledSoupBin[]) {+ const bins = new Map<string, ReconciledSoupBin>();+ const memberships = new Map<string, Set<string>>();+ for (const bin of baseline) {+ if (!bins.has(bin.key)) bins.set(bin.key, { ...bin, itemIds: [] });+ for (const id of bin.itemIds) {+ const keys = memberships.get(id) ?? new Set<string>();+ keys.add(bin.key);+ memberships.set(id, keys);+ }+ }+ return { bins, memberships };+}++/**+ * Reconcile loaded server membership, not an exhaustive local collection.+ * Pass initial bins before continuation bins; repeated bins contribute rows,+ * not counts or cursors. The owner retains each continuation's paging cursor.+ *+ * Rows absent from the evaluation retain their server membership (for example,+ * a page loaded while evaluation was in flight). Candidates absent from every+ * baseline bin are ignored: their contribution to server totals is unproven.+ */+export function reconcileGroupedSoup(+ baseline: readonly ReconciledSoupBin[],+ evaluated: readonly GroupedSoupMembership[]+): ReconciledSoupBin[] {+ const { bins, memberships } = collectGroupedBaseline(baseline);+ const visited = new Set<string>();+ for (const row of evaluated) {+ const loadedKeys = memberships.get(row.id);+ if (!loadedKeys || visited.has(row.id)) continue;+ visited.add(row.id);+ const nextKeys = new Set(row.groupKeys ?? loadedKeys);+ const previousKeys = new Set(row.previousGroupKeys ?? loadedKeys);+ // Unknown current membership cannot justify any count changes, even when+ // the previous complete membership includes unloaded bins.+ if (row.groupKeys !== undefined) {+ for (const key of previousKeys) {+ const bin = bins.get(key);+ if (bin && !nextKeys.has(key)) bin.totalCount -= 1;+ }+ }+ for (const key of nextKeys) {+ let bin = bins.get(key);+ if (!bin) {+ bin = { key, totalCount: 0, itemIds: [], nextCursor: null };+ bins.set(key, bin);+ }+ // A row loaded in A might already be an unloaded member of B. Only a+ // complete old membership can prove that entering B increases its count.+ if (+ row.groupKeys !== undefined &&+ row.previousGroupKeys !== undefined &&+ !previousKeys.has(key)+ )+ bin.totalCount += 1;+ bin.itemIds.push(row.id);+ }+ }+ for (const [id, keys] of memberships) {+ if (visited.has(id)) continue;+ for (const key of keys) bins.get(key)?.itemIds.push(id);+ }+ return [...bins.values()].map((bin) => ({+ ...bin,+ // Partial evidence never permits a negative count, or less than the rows+ // actually visible. Retain empty bins and their cursors for unloaded rows.+ totalCount: Math.max(bin.totalCount, bin.itemIds.length),+ }));+}diff --git a/apps/web/src/lib/queries/soup/items.ts b/apps/web/src/lib/queries/soup/items.tsindex b259df3378..cbf4130459 100644--- a/apps/web/src/lib/queries/soup/items.ts+++ b/apps/web/src/lib/queries/soup/items.ts@@ -367,6 +367,7 @@ export type SoupAstItemsQueryTransport = 'rest' | 'graphql';/** Stable query surface shared by the REST and urql-solid implementations. */export type SoupAstItemsQuery = {+ readonly groupedPaging?: import('./graphql/create-grouped-soup-live-query').GroupedSoupPaging;readonly data: SoupAstItemsData | undefined;readonly error: Error | null;/** Includes paused initial requests, whose data resource is not readable yet. */@@ -482,6 +483,9 @@ export function useSoupAstItemsQuery(};return {+ get groupedPaging() {+ return usesGraphql() ? activeGraphqlQuery().groupedPaging : undefined;+ },get data() {return usesGraphql() ? activeGraphqlQuery().data() : restQuery.data;},diff --git a/docs/agent-guide/soup-live-updates.md b/docs/agent-guide/soup-live-updates.mdindex fb61a8fd48..4ae1b8ae79 100644--- a/docs/agent-guide/soup-live-updates.md+++ b/docs/agent-guide/soup-live-updates.md@@ -33,10 +33,32 @@ and preserves already loaded pages.Newly loaded server rows join the retained display immediately, without duplicates orwaiting for local recomputation to succeed. Removing pages from the server baselineinvalidates overlays built from those pages.-Changing filters or resetting the cache discards prior reconciliation evidence. Grouped-lists, unsupported filters/sorts, and non-cache transports keep their existing+Changing filters or resetting the cache discards prior reconciliation evidence.+Unsupported filters/sorts and non-cache transports keep their existingnetwork behavior.+### Grouped views++With maintained GraphQL cache queries, supported filters and a created/updated+sort, loaded grouped rows also reconcile locally. Initial and per-group “load+more” pages share one owner. Property edits (including multiple assignees),+project changes and UTC date rollover can move loaded rows between bins; proven+filter nonmatches leave the view. Unknown evidence keeps the previous membership.+If local selection is incomplete, normalized display-field updates from initial+and continuation pages remain visible without changing counts or server cursors.+This is not a complete offline grouping query: uncached/unloaded candidates are+not added, and counts preserve unseen server rows rather than counting only the+loaded page. Unsupported sorts/filters retain server membership.++To verify, load a second page in a property group, then edit a property on a row+from that page. It should appear in its destination once, disappear from its old+group, and reverse on rollback without a grouped-query network round trip. Check+multi-value membership and group counts, then load another page: the request must+use the server continuation cursor. Switch filters or reset the cache during a+pending local read; its late result must not restore old rows. A server refresh+restarts continuation coverage. The local phase emits+`graphql_cache.soup_grouped_local_evaluation`.+For Documents (including Tasks), Projects, Chats, and participating Channels,exact `UNSEEN`/`SEEN` notification filters also reconcile locally withcreated/updated timestamp sorts.diff --git a/docs/agent-guide/tasks.md b/docs/agent-guide/tasks.mdindex e83b51db42..4fa3145c28 100644--- a/docs/agent-guide/tasks.md+++ b/docs/agent-guide/tasks.md@@ -41,6 +41,10 @@ editor initializes; the project title, properties, and discussion stay visible.Project task lists automatically fetch all matching pages in every group, withoutper-group `Load More` rows. A failed continuation stops automatic requests and shows`Try again` above the list; already fetched group rows remain available.+With GraphQL Soup enabled, verify a group with more than one page in both the+regular Tasks list (`Load More`) and a project task list (automatic pagination).+Both must use the parent Soup query's continuation owner; reaching the last page+removes the group's paging control without hiding already loaded tasks.The sidebar has no Tags section: filter by tag from the `Tags` group of the `Filter` menu,and switching tabs clears it like any other filter. On mobile, the tabsare pills and the leading sliders button opens one drawer containing Sort, Group, and