Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

feat(search): test ranking exact-match settings with command k results (#123865)

Shy Alterd9d4b0d443ac
diff --git a/frontend/src/lib/components/CommandKSearch/COMMAND_K_SEARCH_UX.md b/frontend/src/lib/components/CommandKSearch/COMMAND_K_SEARCH_UX.md
index 3bad6666e68f..47b7a7d5af96 100644
--- a/frontend/src/lib/components/CommandKSearch/COMMAND_K_SEARCH_UX.md
+++ b/frontend/src/lib/components/CommandKSearch/COMMAND_K_SEARCH_UX.md
@@ -94,7 +94,7 @@ The input is always in exactly one mode, decided from the token under the cursor
- A search with words goes to the ranked search endpoint (`file_system/search/`). A search made only of filters goes to the list endpoint, because there are no words to rank.
- Sections, in this fixed order, covering everything the current palette finds:
1. **Results**: file system objects (insights, dashboards, feature flags, experiments, surveys, notebooks, cohorts, actions, early access features, replay playlists, data pipelines), and products. For a search with words, see [Ranked results](#ranked-results).
- 2. In-memory lists, filtered on every keystroke: **Data management**, **People**, **Health**, **Misc**, **Settings** (with a Dark mode / Light mode row for "dark", "light", "theme", "appearance"), **Create new**. For a search with words they keep only items that match every word with no typo. Products rank with the results instead.
+ 2. In-memory lists, filtered on every keystroke: **Data management**, **People**, **Health**, **Misc**, **Settings** (with a Dark mode / Light mode row for "dark", "light", "theme", "appearance"), **Create new**. For a search with words they keep only items that match every word with no typo. Products rank with the results instead. In the test group of the `command-k-settings-in-results` experiment, settings that match every word with no typo also rank with the results, unless a product has the same name, and the **Settings** section lists only the ones that did not.
3. Remote lists that the file system does not hold: **Events**, **Properties**, **Workflows** (one unified search request), **Accounts** (behind the customer analytics flag), **Support tickets** (3+ characters or a ticket number).
4. **Ask PostHog AI**.
5. **Persons**, **Groups**. Person and group searches can take seconds, so they usually come last. The person search starts with the file system search, after 50 ms. A search that contains "@" waits for the person search and ranks the persons it finds with **Results** / **Best matches**. A person who matches every word comes first. Persons that do not rank stay in **Persons** below, with no duplicates. Other searches do not wait for persons or groups.
diff --git a/frontend/src/lib/components/CommandKSearch/CommandKSearchRankedItem.tsx b/frontend/src/lib/components/CommandKSearch/CommandKSearchRankedItem.tsx
index 4f2ca0c7108f..2a3dc0c73da6 100644
--- a/frontend/src/lib/components/CommandKSearch/CommandKSearchRankedItem.tsx
+++ b/frontend/src/lib/components/CommandKSearch/CommandKSearchRankedItem.tsx
@@ -37,7 +37,9 @@ export function CommandKSearchRankedItem({ row }: { row: Extract<CommandKRow, {
? getItemTypeDisplayName(item.itemType)
: item.category === 'persons'
? 'Person'
- : 'Product'
+ : item.category === 'settings'
+ ? 'Setting'
+ : 'Product'
const context = rankedContext(item, !!row.showCreator)
return (
<>
diff --git a/frontend/src/lib/components/CommandKSearch/commandKRankedResults.ts b/frontend/src/lib/components/CommandKSearch/commandKRankedResults.ts
index d21d40d992c0..11c552946bf9 100644
--- a/frontend/src/lib/components/CommandKSearch/commandKRankedResults.ts
+++ b/frontend/src/lib/components/CommandKSearch/commandKRankedResults.ts
@@ -9,7 +9,7 @@ import type { CommandKRow, CommandKSection } from './commandKSections'
const BEST_MATCHES_LIMIT = 3
const GROUP_LIMIT = 3
const FLAT_LIST_LIMIT = 8
-const LOCAL_GROUPS = ['products', 'persons']
+const LOCAL_GROUPS = ['products', 'persons', 'settings']
interface RankedItem {
item: SearchItem
@@ -24,6 +24,9 @@ function groupLabel(groupKey: string): string {
if (groupKey === 'products') {
return 'Products'
}
+ if (groupKey === 'settings') {
+ return 'Settings'
+ }
return capitalizeFirstLetter(pluralTypeName(getItemTypeDisplayName(groupKey) ?? groupKey.replace(/_/g, ' ')))
}
@@ -80,7 +83,10 @@ const itemRow = (
kind: 'item',
key: `${section}/${item.id}`,
item,
- section: item.category === 'results' ? 'results' : item.category === 'persons' ? 'persons' : 'products',
+ section:
+ item.category === 'results' || item.category === 'persons' || item.category === 'settings'
+ ? item.category
+ : 'products',
highlightWords,
ranked: true,
showType,
@@ -159,6 +165,7 @@ interface RankedResultsInput {
results: SearchItem[]
products: SearchItem[]
persons?: SearchItem[]
+ settings?: SearchItem[]
state: CommandKSection['state']
scoped: boolean
}
@@ -169,6 +176,7 @@ export function buildRankedResults({
results,
products,
persons = [],
+ settings = [],
state,
scoped,
}: RankedResultsInput): CommandKSection[] {
@@ -176,6 +184,7 @@ export function buildRankedResults({
...results.map((item) => ({ item, rank: item.rankKey ?? [], groupKey: groupKeyFor(item) })),
...rankLocalItems(products, query, 'products'),
...rankLocalItems(persons, query, 'persons'),
+ ...rankLocalItems(settings, query, 'settings'),
].sort((a, b) => compareRanks(a.rank, b.rank))
if (!candidates.length) {
return []
diff --git a/frontend/src/lib/components/CommandKSearch/commandKSearchLogic.tsx b/frontend/src/lib/components/CommandKSearch/commandKSearchLogic.tsx
index 96ac1238aa7d..c787f25da482 100644
--- a/frontend/src/lib/components/CommandKSearch/commandKSearchLogic.tsx
+++ b/frontend/src/lib/components/CommandKSearch/commandKSearchLogic.tsx
@@ -75,6 +75,7 @@ import {
emptyRemoteSlots,
isSelectable,
pickDisplayedQuery,
+ rankableSettings,
resultsSettled,
themeSearchItem,
} from './commandKSections'
@@ -347,7 +348,8 @@ export interface commandKSearchLogicMeta {
displayedQuery: {
query: SectionsQuery
stale: boolean
- }
+ },
+ featureFlags: FeatureFlagsSet
) => CommandKSection[]
selectableRows: (sections: CommandKSection[]) => CommandKRow[]
highlightedRow: (selectableRows: CommandKRow[], highlightedKey: string | null) => CommandKRow | null
@@ -697,14 +699,23 @@ export const commandKSearchLogic = kea<commandKSearchLogicType>([
): { query: SectionsQuery; stale: boolean } => pickDisplayedQuery(current, shown, remote),
],
sections: [
- (s) => [s.mode, s.cursorContext, s.suggestionsSection, s.localLists, s.remote, s.displayedQuery],
+ (s) => [
+ s.mode,
+ s.cursorContext,
+ s.suggestionsSection,
+ s.localLists,
+ s.remote,
+ s.displayedQuery,
+ s.featureFlags,
+ ],
(
mode: CommandKMode,
context: CursorContext,
suggestions: CommandKSection | null,
local: LocalLists,
remote: RemoteSlots,
- displayed: { query: SectionsQuery; stale: boolean }
+ displayed: { query: SectionsQuery; stale: boolean },
+ featureFlags: FeatureFlagsSet
): CommandKSection[] =>
buildSections({
mode,
@@ -715,6 +726,9 @@ export const commandKSearchLogic = kea<commandKSearchLogicType>([
...displayed.query,
stale: displayed.stale,
personsFirst: !!displayed.query.remoteQueries.persons?.includes('@'),
+ settingsInResults:
+ rankableSettings(local, displayed.query.localQuery).length > 0 &&
+ featureFlags[FEATURE_FLAGS.COMMAND_K_SETTINGS_IN_RESULTS] === 'test',
}),
],
selectableRows: [
diff --git a/frontend/src/lib/components/CommandKSearch/commandKSections.test.ts b/frontend/src/lib/components/CommandKSearch/commandKSections.test.ts
index 8c6d9c0a339e..cdfd8f839e49 100644
--- a/frontend/src/lib/components/CommandKSearch/commandKSections.test.ts
+++ b/frontend/src/lib/components/CommandKSearch/commandKSections.test.ts
@@ -52,6 +52,7 @@ const input = (mode: CommandKMode, overrides: Partial<SectionsInput>): SectionsI
viewAll: null,
stale: false,
personsFirst: false,
+ settingsInResults: false,
...overrides,
})
@@ -261,6 +262,40 @@ describe('buildSections', () => {
expect(sections[0].rows).toMatchObject([{ kind: 'item', section: 'persons', item: { id: 'person-1' } }])
})
+ test.each([
+ ['off', false, 'Personal API keys', ['best-matches', 'settings'], 'Ada board'],
+ ['on', true, 'Personal API keys', ['best-matches'], 'Personal API keys'],
+ ['on, but a product has its name', true, 'API keys', ['best-matches', 'settings'], 'API keys'],
+ ])('ranks an exact-match setting with the results: flag %s', (_, settingsInResults, settingName, keys, first) => {
+ const setting: SearchItem = {
+ id: 'settings-api-keys',
+ name: `User: ${settingName} (api keys)`,
+ displayName: settingName,
+ category: 'settings',
+ }
+ const products: SearchItem[] =
+ settingName === 'API keys' ? [{ id: 'product-api-keys', name: 'API keys', category: 'products' }] : []
+ const remote = emptyRemoteSlots()
+ remote.results = {
+ cache: {
+ 'api keys': [{ ...dashboard('a', '1'), name: 'Ada board', rankKey: [-1, 0, 0, 0, -1, 0, 1, 0, 0] }],
+ },
+ }
+ const sections = buildSections(
+ input('search', {
+ local: { ...emptyLocal(), settings: [setting], products },
+ remote,
+ remoteQueries: { results: 'api keys' },
+ localQuery: 'api keys',
+ settingsInResults,
+ })
+ )
+
+ expect(sections.map((section) => section.key)).toEqual(keys)
+ const firstItem = sections[0].rows[0]
+ expect(firstItem.kind === 'item' ? String(firstItem.item.displayName || firstItem.item.name) : null).toBe(first)
+ })
+
it('says which words were dropped when a filter is active', () => {
const remote = emptyRemoteSlots()
remote.results = {
diff --git a/frontend/src/lib/components/CommandKSearch/commandKSections.ts b/frontend/src/lib/components/CommandKSearch/commandKSections.ts
index a4362e29fe24..7b4fc09b0821 100644
--- a/frontend/src/lib/components/CommandKSearch/commandKSections.ts
+++ b/frontend/src/lib/components/CommandKSearch/commandKSections.ts
@@ -224,6 +224,7 @@ export interface SectionsInput extends SectionsQuery {
remote: RemoteSlots
stale: boolean
personsFirst: boolean
+ settingsInResults: boolean
}
const isRanked = (query: SectionsQuery): boolean =>
@@ -243,6 +244,9 @@ export function pickDisplayedQuery(
return { query: shown, stale: true }
}
+const exactSettingsMatches = (settings: SearchItem[], localQuery: string): SearchItem[] =>
+ localQuery ? matchLocal('settings', settings, localQuery, true) : []
+
export const resultsSettled = (query: SectionsQuery, remote: RemoteSlots): boolean =>
isSettled(remote.results, query.remoteQueries.results)
@@ -282,17 +286,30 @@ function withViewAll(results: CommandKSection | null, viewAll: ViewAll | null):
}
}
+const plainName = (item: SearchItem): string =>
+ String(item.displayName || item.name)
+ .toLowerCase()
+ .replace(/\s*\([^)]*\)\s*$/, '')
+ .trim()
+
+export function rankableSettings(local: LocalLists, localQuery: string): SearchItem[] {
+ const productNames = new Set(local.products.map(plainName))
+ return exactSettingsMatches(local.settings, localQuery).filter((item) => !productNames.has(plainName(item)))
+}
+
function rankedResultsSections(
results: SearchItem[],
query: string,
local: LocalLists,
localQuery: string,
state: CommandKSection['state'],
- persons: SearchItem[]
+ persons: SearchItem[],
+ settingsInResults: boolean
): CommandKSection[] {
const words = freeWords(query)
return buildRankedResults({
persons,
+ settings: settingsInResults ? rankableSettings(local, localQuery) : [],
query: words.join(' '),
highlightWords: words,
results,
@@ -338,7 +355,7 @@ function lowerSections(
if (ranked && source === 'products') {
continue
}
- const filtered = matchLocal(source, local[source], localQuery, ranked)
+ const filtered = matchLocal(source, local[source], localQuery, ranked).filter((item) => !shown.has(item.id))
const themeItem = source === 'settings' ? input.themeItem : null
const rows = themeItem
? itemRows(source, [themeItem, ...filtered], SECONDARY_LIMIT + 1)
@@ -404,7 +421,15 @@ export function buildSections(input: SectionsInput): CommandKSection[] {
sections.push(remoteSection('persons', leadingPersons, state))
}
if (ranked && results && resultsQuery) {
- const rankedSections = rankedResultsSections(results, resultsQuery, local, localQuery, state, leadingPersons)
+ const rankedSections = rankedResultsSections(
+ results,
+ resultsQuery,
+ local,
+ localQuery,
+ state,
+ leadingPersons,
+ input.settingsInResults
+ )
sections.push(
...(rankedSections.some((section) => section.key === 'results')
? rankedSections.map((section) =>
diff --git a/frontend/src/lib/constants.tsx b/frontend/src/lib/constants.tsx
index ee48a06c7ea2..d7faf9a6af30 100644
--- a/frontend/src/lib/constants.tsx
+++ b/frontend/src/lib/constants.tsx
@@ -284,6 +284,7 @@ export const FEATURE_FLAGS = {
CODE_BADGE_SELF_DRIVING_LABEL: 'code-badge-self-driving-label', // owner: #team-self-driving, experiment: label the Code badge "Self-driving" and compare setup activations
COHORT_INLINE_CALCULATION: 'inline-cohort-calculation', // owner: #team-analytics-platform, inlines fast dynamic cohort queries instead of using precomputed cohortpeople table
COHORTS_TAXONOMIC_BASIC_LIST: 'cohorts-taxonomic-basic-list', // owner: @adamleith, picker sends ?basic=true to the cohorts list endpoint (trimmed payload: no query/groups/last_error_message/experiment_set; filters is kept)
+ COMMAND_K_SETTINGS_IN_RESULTS: 'command-k-settings-in-results', // owner: @puemos #team-platform-ux multivariate=control,test, experiment: rank exact-match settings with the Command K results
CONDENSED_FILTER_BAR: 'condensed_filter_bar', // owner: @jordanm-posthog #team-web-analytics
CREATE_FORM_TOOL: 'phai-create-form-tool', // owner: @kappa90 #team-posthog-ai
CRM_ITERATION_ONE: 'crm-iteration-one', // owner: @arthurdedeus #team-customer-analytics