Skip to content

armgit / macro

Public Imported from GitHub
0

Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.

perf(search): build Soup's local search pool only when search has text (#9424)

Wolf Mermelstein5844f2b77912
diff --git a/apps/web/AGENTS.md b/apps/web/AGENTS.md
index ca8224a445..68e9e9208b 100644
--- a/apps/web/AGENTS.md
+++ b/apps/web/AGENTS.md
@@ -170,6 +170,9 @@ boosts (Cmd+K's channel and DM boosts) order rows within those groups and never
lift a row past a better name match. The one exception: Cmd+K then keeps Macro
users and DMs ahead of CRM contacts across the people positions, whatever their
match. Tune name relevance there for every menu.
+Soup's local search pool (`createSearchState`) stays lazy: it builds only when
+local search has text. Measure Quick Access ranking at scale with the
+[scale fixture](src/features/command/browser-test/README.md).
Cmd+K and mobile name search widen the first group: a query equal to the first
word of a DM or person name counts as exact, so a person's first name reaches
their full-name DM. `commaSeparatedChannelMatch` (Cmd+K, mobile name search,
diff --git a/apps/web/src/features/command/browser-test/README.md b/apps/web/src/features/command/browser-test/README.md
new file mode 100644
index 0000000000..4a7f6ff6b3
--- /dev/null
+++ b/apps/web/src/features/command/browser-test/README.md
@@ -0,0 +1,42 @@
+# Quick Access scale measurement
+
+Run the existing standalone Vite frontend, then from `apps/web`:
+
+```sh
+URL=https://localhost:3009 OUT=/tmp/command-scale \
+PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH=/path/to/google-chrome-stable \
+node src/features/command/browser-test/measure.mjs
+```
+
+No login or backend is required. Each size uses a new browser context and a
+random cache namespace, with 1k/5k/20k synthetic notes written through the real
+worker host. Closing Chrome releases the contexts. Seed fields are complete for
+the history projection; missing materialized rows fail the run. Page errors also
+fail the run. Freeze source and restart the owned frontend between revisions;
+do not combine samples interrupted by dependency reloads.
+
+At 4× CDP CPU throttle the runner records:
+
+- Synthetic candidate construction and cache seed time separately.
+- Cold search plus selected-record materialization, then 54 warm searches.
+- 126 local filter + final Cmd+K ranking calls over 18 short prefixes.
+- A separate CPU profile of those prefixes, outside timed samples.
+- Main-thread heap before/after forced GC and outgoing worker/port message kinds.
+
+JSON and CPU profiles go to `OUT`. Heap deltas include the fixture's candidate
+objects and main-thread host, not worker/WASM memory or isolated app-index memory.
+Construction time is not QuickAccessSource activation. Message counts cover web
+worker transport, not desktop native IPC. JS CPU timings are not input-to-paint.
+The profile and synthetic candidate stress test intentionally exercise a full
+local corpus: the cache-enabled app normally ranks a bounded selection in JS.
+
+Compare revisions with identical corpora, throttle, and queries. For causal scorer
+comparisons, counterbalance revision order and hold wall-clock timestamps fixed;
+compare complete result ordering outside the timed section. Mixed entities,
+aliases, comma queries, dynamic labels, people priority and boosts are covered by
+`freshSort.test.ts` and `rank-command-search-items.test.ts`.
+
+The engine's `quick-access-v1` policy and JS `NAME_MATCH_RANKING` are different.
+These timings do not establish ranking parity or justify replacing final contextual
+ranking with the engine's existing order. The app already searches the complete
+cache there, materializes up to 500 hits, then combines local commands and people.
diff --git a/apps/web/src/features/command/browser-test/measure.mjs b/apps/web/src/features/command/browser-test/measure.mjs
new file mode 100644
index 0000000000..224216cd3f
--- /dev/null
+++ b/apps/web/src/features/command/browser-test/measure.mjs
@@ -0,0 +1,147 @@
+import { chromium } from 'playwright';
+import fs from 'node:fs';
+
+const out = process.env.OUT;
+if (!out)
+ throw new Error('Set OUT to an evidence directory outside the checkout');
+const url = new URL(
+ '/src/features/command/browser-test/scale.html',
+ process.env.URL ?? 'https://localhost:3009'
+);
+fs.mkdirSync(out, { recursive: true });
+(async () => {
+ const browser = await chromium.launch({
+ executablePath: process.env.PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH,
+ args: ['--ignore-certificate-errors', '--enable-precise-memory-info'],
+ });
+ try {
+ for (const size of [1000, 5000, 20000]) {
+ const ctx = await browser.newContext({ ignoreHTTPSErrors: true });
+ const page = await ctx.newPage(),
+ errors = [];
+ await ctx.addInitScript(() => {
+ window.__messages = {};
+ for (const [label, proto] of [
+ ['port', MessagePort.prototype],
+ ['worker', Worker.prototype],
+ ]) {
+ const orig = proto.postMessage;
+ proto.postMessage = function (m, ...args) {
+ const payload = Array.isArray(m) ? m[1] : m;
+ const kind = payload?.request?.kind ?? payload?.kind;
+ if (kind) {
+ const key = label + ':' + kind;
+ window.__messages[key] = (window.__messages[key] ?? 0) + 1;
+ }
+ return orig.call(this, m, ...args);
+ };
+ }
+ });
+ page.on('pageerror', (e) => errors.push(e.message));
+ await page.goto(url.href, {
+ waitUntil: 'domcontentloaded',
+ timeout: 90000,
+ });
+ await page.waitForFunction(
+ () => window.commandScale,
+ {},
+ { timeout: 90000 }
+ );
+ await page.waitForTimeout(3000);
+ await page.reload({ waitUntil: 'domcontentloaded' });
+ await page.waitForFunction(
+ () => window.commandScale,
+ {},
+ { timeout: 90000 }
+ );
+ await page.waitForTimeout(1000);
+ const cdp = await ctx.newCDPSession(page);
+ await cdp.send('Emulation.setCPUThrottlingRate', { rate: 4 });
+ await cdp.send('HeapProfiler.collectGarbage');
+ const before = await cdp.send('Runtime.getHeapUsage');
+ const seed = await page.evaluate(
+ (n) => window.commandScale.seed(n),
+ size
+ );
+ await cdp.send('HeapProfiler.collectGarbage');
+ const after = await cdp.send('Runtime.getHeapUsage');
+ const cold = await page.evaluate(() =>
+ window.commandScale.engine('project')
+ );
+ const data = await page.evaluate(async () => {
+ const a = window.commandScale,
+ rows = [];
+ for (let r = 0; r < 7; r++)
+ for (const q of a.samples) {
+ rows.push({
+ q,
+ js: a.rank(q),
+ ...(r < 3 ? { engine: await a.engine(q) } : {}),
+ });
+ }
+ return rows;
+ });
+ await cdp.send('Profiler.enable');
+ await cdp.send('Profiler.setSamplingInterval', { interval: 1000 });
+ await cdp.send('Profiler.start');
+ await page.evaluate(() => {
+ for (const q of window.commandScale.samples)
+ window.commandScale.rank(q);
+ });
+ const { profile } = await cdp.send('Profiler.stop');
+ fs.writeFileSync(
+ out + '/' + size + '.cpuprofile',
+ JSON.stringify(profile)
+ );
+ if (errors.length) throw Error(errors.join('\n'));
+ if (
+ (await page.evaluate(() => window.commandScale.instance)) !==
+ seed.instance
+ )
+ throw Error('Fixture reloaded during measurement');
+ const messages = await page.evaluate(() => window.__messages);
+ const result = {
+ size,
+ seed,
+ cold,
+ heapBefore: before,
+ heapAfter: after,
+ messages,
+ data,
+ };
+ fs.writeFileSync(out + '/' + size + '.json', JSON.stringify(result));
+ const q = (a, p) => a.sort((x, y) => x - y)[Math.ceil(a.length * p) - 1];
+ console.log(
+ JSON.stringify({
+ size,
+ seed,
+ cold,
+ jsP50: q(
+ data.map((x) => x.js.totalMs),
+ 0.5
+ ),
+ jsP95: q(
+ data.map((x) => x.js.totalMs),
+ 0.95
+ ),
+ engineP50: q(
+ data.filter((x) => x.engine).map((x) => x.engine.searchMs),
+ 0.5
+ ),
+ engineP95: q(
+ data.filter((x) => x.engine).map((x) => x.engine.searchMs),
+ 0.95
+ ),
+ heapDelta: after.usedSize - before.usedSize,
+ messages,
+ })
+ );
+ await ctx.close();
+ }
+ } finally {
+ await browser.close();
+ }
+})().catch((e) => {
+ console.error(e);
+ process.exitCode = 1;
+});
diff --git a/apps/web/src/features/command/browser-test/scale.html b/apps/web/src/features/command/browser-test/scale.html
new file mode 100644
index 0000000000..daa6066ca7
--- /dev/null
+++ b/apps/web/src/features/command/browser-test/scale.html
@@ -0,0 +1 @@
+<!doctype html><html lang="en"><head><meta charset="utf-8"><title>Quick Access scale measurement</title></head><body><pre id="status">loading</pre><script type="module" src="./scale.ts"></script></body></html>
diff --git a/apps/web/src/features/command/browser-test/scale.ts b/apps/web/src/features/command/browser-test/scale.ts
new file mode 100644
index 0000000000..c514fb6c5f
--- /dev/null
+++ b/apps/web/src/features/command/browser-test/scale.ts
@@ -0,0 +1,142 @@
+import { searchQuickAccessItems } from '@core/context/quickAccess/entity-search';
+import type { EntityItem } from '@core/context/quickAccess/types';
+import { createWorkerCacheHost } from '@graphql-cache/host/worker-host';
+import { materializeCachedGraphqlHistoryItems } from '@queries/history/graphql';
+import { rankCommandSearchItems } from '../rank-command-search-items';
+
+const query = `query Seed($input: SoupInput!) { user { id soup(input: $input) { items { __typename id ... on GraphqlSoupDocument { name fileType ownerId createdAt updatedAt viewedAt deletedAt subType { __typename } } } } } }`;
+const now = Date.now();
+const words = [
+ 'Project roadmap',
+ 'Release planning',
+ 'Team design notes',
+ 'Weekly status',
+ 'Product research',
+ 'Launch checklist',
+ 'Meeting notes',
+ 'Budget report',
+];
+let items: EntityItem[] = [];
+let host: ReturnType<typeof createWorkerCacheHost> | undefined;
+const status = document.querySelector('#status')!;
+const samples = [
+ 'p',
+ 'pr',
+ 'pro',
+ 'proj',
+ 'project',
+ 'project r',
+ 'r',
+ 're',
+ 'rel',
+ 'release',
+ 'tea',
+ 'team',
+ 'notes',
+ 'launch',
+ 'design',
+ 'research',
+ 'budget',
+ 'zzzz',
+];
+
+const api = {
+ instance: crypto.randomUUID(),
+ candidates: () => items,
+ async seed(size: number) {
+ host?.dispose();
+ host = createWorkerCacheHost({
+ scope: `command-scale-${crypto.randomUUID()}`,
+ });
+ items = [];
+ const start = performance.now();
+ for (let i = 0; i < size; i++) {
+ const id = `scale-${String(i).padStart(6, '0')}`,
+ name = `${words[i % words.length]} ${i}`;
+ const viewedAt = new Date(now - i * 60000).toISOString();
+ items.push({
+ kind: 'entity',
+ id,
+ bucket: 'note',
+ searchText: name,
+ sortTimestamp: now - i * 60000,
+ timestamps: { viewedAt, updatedAt: viewedAt },
+ data: {
+ id,
+ type: 'document',
+ name,
+ fileType: 'md',
+ ownerId: 'fixture-owner',
+ },
+ });
+ }
+ const buildMs = performance.now() - start;
+ const seedStart = performance.now();
+ for (let offset = 0; offset < size; offset += 500) {
+ const data = items.slice(offset, offset + 500).map((i) => ({
+ __typename: 'GraphqlSoupDocument',
+ id: i.id,
+ name: i.searchText,
+ fileType: 'md',
+ ownerId: 'fixture-owner',
+ createdAt: i.timestamps.viewedAt,
+ updatedAt: i.timestamps.viewedAt,
+ viewedAt: i.timestamps.viewedAt,
+ deletedAt: null,
+ subType: null,
+ }));
+ await host.writeQuery({
+ query,
+ variables: { input: { initial: { limit: offset + 1 } } },
+ data: { user: { id: 'fixture-owner', soup: { items: data } } },
+ });
+ }
+ return {
+ instance: api.instance,
+ size,
+ buildMs,
+ seedMs: performance.now() - seedStart,
+ };
+ },
+ rank(query: string) {
+ const start = performance.now();
+ const candidates = searchQuickAccessItems(items, query);
+ const filtered = performance.now();
+ const ranked = rankCommandSearchItems(candidates, query, {
+ preserveAdditionalEntityMatches: true,
+ });
+ return {
+ filterMs: filtered - start,
+ rankMs: performance.now() - filtered,
+ totalMs: performance.now() - start,
+ count: ranked.length,
+ ids: ranked.slice(0, 20).map((i) => i.id),
+ };
+ },
+ async engine(query: string) {
+ if (!host) throw Error('seed first');
+ const start = performance.now();
+ const page = await host.search({
+ profile: 'quick-access-v1',
+ buckets: ['note'],
+ query,
+ limit: 500,
+ });
+ const searched = performance.now();
+ const materialized = await materializeCachedGraphqlHistoryItems(
+ host,
+ page.documents
+ );
+ if (materialized.length !== page.documents.length)
+ throw Error('Incomplete cache projection');
+ return {
+ searchMs: searched - start,
+ materializeMs: performance.now() - searched,
+ count: materialized.length,
+ };
+ },
+ samples,
+};
+Object.assign(window, { commandScale: api });
+status.textContent = 'ready';
+window.addEventListener('pagehide', () => host?.dispose(), { once: true });
diff --git a/apps/web/src/features/command/rank-command-search-items.test.ts b/apps/web/src/features/command/rank-command-search-items.test.ts
index 2a47958852..9e1962b335 100644
--- a/apps/web/src/features/command/rank-command-search-items.test.ts
+++ b/apps/web/src/features/command/rank-command-search-items.test.ts
@@ -142,6 +142,29 @@ describe('command menu search ranking', () => {
}
);
+ it('reranks dynamic command descriptions without replacing rows', () => {
+ let description = 'Create';
+ const items: CommandMenuItem[] = [
+ {
+ id: 'dynamic-command',
+ kind: 'command',
+ bucket: 'command',
+ searchText: 'create make',
+ sortTimestamp: NOW,
+ timestamps: { viewedAt: new Date(NOW) },
+ data: {
+ description: () => description,
+ scopeId: 'command-menu',
+ runWithInputFocused: true,
+ },
+ },
+ note('plan', 'Create plan'),
+ ];
+ expect(ids(items, 'create')).toEqual(['dynamic-command', 'plan']);
+ description = 'Make';
+ expect(ids(items, 'create')).toEqual(['plan', 'dynamic-command']);
+ });
+
it('keeps commands and other entities ordered by recency', () => {
expect(
ids(
diff --git a/apps/web/src/features/next-soup/soup-view/create-search-state.test.ts b/apps/web/src/features/next-soup/soup-view/create-search-state.test.ts
new file mode 100644
index 0000000000..e1d22f5bc8
--- /dev/null
+++ b/apps/web/src/features/next-soup/soup-view/create-search-state.test.ts
@@ -0,0 +1,57 @@
+import { createRoot, createSignal } from 'solid-js';
+import { afterEach, expect, it, vi } from 'vitest';
+
+const pool = vi.hoisted(() => vi.fn(() => []));
+vi.mock('@app/lib/analytics', () => ({ analytics: { track: vi.fn() } }));
+vi.mock('@app/features/soup/search', async (importOriginal) => ({
+ ...(await importOriginal<typeof import('@app/features/soup/search')>()),
+ useSearchContext: () => ({ entityPool: pool }),
+}));
+vi.mock('@components/app/GlobalAppState', () => ({
+ useGlobalNotificationSource: () => ({}),
+}));
+vi.mock('@core/context/user', () => ({ useUserId: () => () => 'viewer' }));
+vi.mock('@queries/soup/search', () => ({
+ useSearchSoupQuery: () => ({ isSuccess: false }),
+ validateSearchServiceText: (text: string) => text.length >= 3,
+}));
+vi.mock('@app/features/next-soup/filters/configs/', () => ({
+ SOUP_FILTERS: [],
+}));
+vi.mock('@app/features/next-soup/soup-view/sort-options', () => ({
+ SORT_CONFIGS: {
+ updated_at: { id: 'updated_at', fn: () => 0 },
+ },
+}));
+vi.mock('@core/mobile/inputModality', () => ({ isModality: () => false }));
+vi.mock('@core/component/EntityIcon', () => ({ getIconConfig: vi.fn() }));
+
+import { createSoupState } from '../create-soup-state';
+import { createSearchState } from './create-search-state';
+
+afterEach(() => vi.useRealTimers());
+it('does not demand the entity pool until local search has text', () => {
+ vi.useFakeTimers();
+ const [text, setText] = createSignal('');
+ const { search, dispose } = createRoot((dispose) => ({
+ dispose,
+ search: createSearchState({
+ surface: () => 'home',
+ soup: createSoupState(),
+ filters: () => ({ include: {}, exclude: {} }),
+ assignees: () => [],
+ searchText: text,
+ setSearchText: setText,
+ }),
+ }));
+ expect(search.localFuzzyResults()).toEqual([]);
+ expect(pool).not.toHaveBeenCalled();
+ setText('budget');
+ vi.advanceTimersByTime(20);
+ expect(search.localFuzzyResults()).toEqual([]);
+ expect(pool).toHaveBeenCalledOnce();
+ setText('');
+ vi.advanceTimersByTime(20);
+ expect(pool).toHaveBeenCalledOnce();
+ dispose();
+});
diff --git a/apps/web/src/features/next-soup/soup-view/create-search-state.ts b/apps/web/src/features/next-soup/soup-view/create-search-state.ts
index e2a1fe4bbc..dac41fff5c 100644
--- a/apps/web/src/features/next-soup/soup-view/create-search-state.ts
+++ b/apps/web/src/features/next-soup/soup-view/create-search-state.ts
@@ -8,7 +8,8 @@ import {
} from '@app/features/soup/search';
import { useGlobalNotificationSource } from '@components/app/GlobalAppState';
import { useUserId } from '@core/context/user';
-import { type Accessor, createMemo, type Setter } from 'solid-js';
+import { createLazyMemo } from '@solid-primitives/memo';
+import type { Accessor, Setter } from 'solid-js';
import { buildSoupSearchRequest } from '../queries/search-request';
interface CreateSearchStateArgs {
@@ -47,7 +48,7 @@ export const createSearchState = ({
});
const { entityPool, routinePool } = useSearchContext();
- const localPool = createMemo(() => {
+ const localPool = createLazyMemo(() => {
const pool =
surface() === 'search'
? [...entityPool(), ...routinePool()]