Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

feat(insights): expose the insight data checks as an mcp tool (#116836)

Sam Penningtoncbff03843268
diff --git a/services/mcp/schema/tool-definitions-all.json b/services/mcp/schema/tool-definitions-all.json
index e211bbcd8401..93219c889d24 100644
--- a/services/mcp/schema/tool-definitions-all.json
+++ b/services/mcp/schema/tool-definitions-all.json
@@ -7600,6 +7600,21 @@
"readOnlyHint": true
}
},
+ "insight-trust-check": {
+ "description": "Check whether the numbers of a product analytics insight can be trusted. It looks at the events behind the insight for four common problems that inflate its numbers: events from internal and test users, events from local development, bot and crawler traffic, and events sent twice. Pass the `id` or 8-character `short_id` of a saved insight, or the `query` of one that is not saved yet, such as a query you are about to pass to `insight-create`. Use it after you create or change a trends, funnel, stickiness, or lifecycle insight, and when the user asks whether an insight's numbers are accurate or can be trusted.\n\nOnly event series are checked. The `status` is `unsupported` for a SQL insight, for a series built on an action or a data warehouse table, for an insight that still stores legacy `filters`, and when a property of the project is hidden from you. It is `no_events` when no event matches the insight.\n\nEach check has a `level` (`passed`, `warning`, `problem` or `skipped`), the `events` it found with their `share`, and the `top_events` it found most often. When `action` is `filter_internal_users`, the fix is to set `filterTestAccounts` to true on the query, with `insight-update` for a saved insight. When it is `open_filter_settings`, the project's internal and test users filter misses the team, and the user must change it in the project settings. The checks read the newest days of the date range, up to 7, and `coverage` says which days and how many events. Each check's `method` says whether it counted every event on those days or read a sample, and `scaled` is true when its `events` is an estimate from a sample. The internal users check matches the organization's member emails only with the `organization_member:read` scope, and its verified domains only with `organization:read`. Checking a `query` needs the `query:read` scope as well.",
+ "category": "Product analytics",
+ "feature": "product_analytics",
+ "summary": "Check whether an insight's numbers can be trusted.",
+ "title": "Check whether an insight's numbers can be trusted",
+ "required_scopes": ["insight:read", "query:read"],
+ "annotations": {
+ "destructiveHint": false,
+ "idempotentHint": true,
+ "openWorldHint": true,
+ "readOnlyHint": true
+ },
+ "feature_flag": "insight-data-checks"
+ },
"insight-update": {
"description": "Update a saved insight by numeric `id` or `short_id`. Can update name, description, query, tags, favorited status, and dashboards. Returns insight metadata only — after updating the query, call the insight-query tool with the same identifier if you want to see the recomputed results. Query uses the same format as insight-create. Replacing query replaces the entire definition; include all filters and visualization settings you want to retain.",
"category": "Product analytics",
diff --git a/services/mcp/schema/tool-definitions.json b/services/mcp/schema/tool-definitions.json
index bbab925c7d24..db0fe0f86f61 100644
--- a/services/mcp/schema/tool-definitions.json
+++ b/services/mcp/schema/tool-definitions.json
@@ -270,6 +270,21 @@
"readOnlyHint": true
}
},
+ "insight-trust-check": {
+ "description": "Check whether the numbers of a product analytics insight can be trusted. It looks at the events behind the insight for four common problems that inflate its numbers: events from internal and test users, events from local development, bot and crawler traffic, and events sent twice. Pass the `id` or 8-character `short_id` of a saved insight, or the `query` of one that is not saved yet, such as a query you are about to pass to `insight-create`. Use it after you create or change a trends, funnel, stickiness, or lifecycle insight, and when the user asks whether an insight's numbers are accurate or can be trusted.\n\nOnly event series are checked. The `status` is `unsupported` for a SQL insight, for a series built on an action or a data warehouse table, for an insight that still stores legacy `filters`, and when a property of the project is hidden from you. It is `no_events` when no event matches the insight.\n\nEach check has a `level` (`passed`, `warning`, `problem` or `skipped`), the `events` it found with their `share`, and the `top_events` it found most often. When `action` is `filter_internal_users`, the fix is to set `filterTestAccounts` to true on the query, with `insight-update` for a saved insight. When it is `open_filter_settings`, the project's internal and test users filter misses the team, and the user must change it in the project settings. The checks read the newest days of the date range, up to 7, and `coverage` says which days and how many events. Each check's `method` says whether it counted every event on those days or read a sample, and `scaled` is true when its `events` is an estimate from a sample. The internal users check matches the organization's member emails only with the `organization_member:read` scope, and its verified domains only with `organization:read`. Checking a `query` needs the `query:read` scope as well.",
+ "category": "Product analytics",
+ "feature": "product_analytics",
+ "summary": "Check whether an insight's numbers can be trusted.",
+ "title": "Check whether an insight's numbers can be trusted",
+ "required_scopes": ["insight:read", "query:read"],
+ "annotations": {
+ "destructiveHint": false,
+ "idempotentHint": true,
+ "openWorldHint": true,
+ "readOnlyHint": true
+ },
+ "feature_flag": "insight-data-checks"
+ },
"notebook-edit": {
"description": "Edit part of a notebook's content by short_id.\n\nFor markdown notebooks, use `execute-sql` to read the current markdown from `system.notebooks.markdown`, then edit with `old_markdown` / `new_markdown`. These are plain markdown strings; do not copy or construct the nested `ph-markdown-notebook` JSON wrapper.\n\nFor legacy rich-text notebooks, inspect the JSON content from `system.notebooks.content`, then edit with `old_value` / `new_value` JSON subtree replacement. Matching is by content: the tool finds `old_value` anywhere in the notebook and swaps it for `new_value` in place.\n\nOn success, returns the updated notebook with the new content and bumped version. On a 409 or 410 error, the notebook has changed since you loaded it; reload the latest markdown or JSON content and re-apply your edit.",
"category": "Notebooks",
diff --git a/services/mcp/src/tools/index.ts b/services/mcp/src/tools/index.ts
index bf3fe7aaddbd..e8d64509b1ae 100644
--- a/services/mcp/src/tools/index.ts
+++ b/services/mcp/src/tools/index.ts
@@ -18,6 +18,7 @@ import submitFeedback from './feedback/submit'
import { GENERATED_TOOL_MAP } from './generated'
// Insights
import queryInsight from './insights/query'
+import insightTrustCheck from './insights/trustCheck'
// Links (utility — builds canonical app URLs from the frontend's route table)
import generateAppUrl from './links/generate-app-url'
import { mergeToolFactories } from './mergeToolFactories'
@@ -98,6 +99,7 @@ export const TOOL_MAP: Record<string, () => ToolBase<ZodObjectAny>> = {
// Insights
'insight-query': queryInsight,
+ 'insight-trust-check': insightTrustCheck,
// Links (utility — canonical app URLs so the model never hand-builds/mis-slugs entity links)
'generate-app-url': generateAppUrl,
diff --git a/services/mcp/src/tools/insights/trustCheck.ts b/services/mcp/src/tools/insights/trustCheck.ts
new file mode 100644
index 000000000000..9d477e4ac81b
--- /dev/null
+++ b/services/mcp/src/tools/insights/trustCheck.ts
@@ -0,0 +1,61 @@
+import { z } from 'zod'
+
+import type { Schemas } from '@/api/generated'
+import { normalizeParamAliases } from '@/tools/cast-helpers'
+import { withAgentNote, type WithAgentNote } from '@/tools/tool-utils'
+import type { Context, ToolBase } from '@/tools/types'
+
+const schema = z.preprocess(
+ normalizeParamAliases({ id: ['insightId', 'insight_id', 'short_id', 'shortId'] }),
+ z
+ .object({
+ id: z
+ .union([z.number(), z.string()])
+ .optional()
+ .describe('Numeric id or 8-character `short_id` of a saved insight. Pass this or `query`, not both.'),
+ query: z
+ .object({ kind: z.string() })
+ .catchall(z.unknown())
+ .optional()
+ .describe(
+ 'An insight query that is not saved yet, in the shape `insight-create` takes, for example ' +
+ '`{ "kind": "InsightVizNode", "source": { "kind": "TrendsQuery", "series": [{ "kind": "EventsNode", ' +
+ '"event": "$pageview" }], "dateRange": { "date_from": "-30d" } } }`. Pass this or `id`, not both.'
+ ),
+ })
+ .refine((params) => (params.id === undefined) !== (params.query === undefined), {
+ message: 'Pass either `id` for a saved insight or `query` for a query that is not saved yet, not both.',
+ })
+)
+
+type Params = z.infer<typeof schema>
+
+type Result = WithAgentNote<Schemas.InsightDataChecks>
+
+const AGENT_NOTE =
+ 'When a check has the level `warning` or `problem`, tell the user what it found in one sentence, with its share ' +
+ "of the events, and offer the fix. Do not change the insight, the query or the project's filter settings before " +
+ 'the user agrees. When every check passed, say so in a few words. Do not list the checks that passed.'
+
+const insightTrustCheck = (): ToolBase<typeof schema, Result> => ({
+ name: 'insight-trust-check',
+ schema,
+ handler: async (context: Context, params: Params) => {
+ const { id, query } = params
+ const projectId = encodeURIComponent(await context.stateManager.getProjectId())
+ const result =
+ id !== undefined
+ ? await context.api.request<Schemas.InsightDataChecks>({
+ method: 'GET',
+ path: `/api/projects/${projectId}/insights/${encodeURIComponent(String(id))}/data_checks/`,
+ })
+ : await context.api.request<Schemas.InsightDataChecks>({
+ method: 'POST',
+ path: `/api/projects/${projectId}/insights/data_checks/`,
+ body: { query },
+ })
+ return withAgentNote(result, AGENT_NOTE)
+ },
+})
+
+export default insightTrustCheck
diff --git a/services/mcp/tests/unit/__snapshots__/tool-schemas/insight-trust-check.json b/services/mcp/tests/unit/__snapshots__/tool-schemas/insight-trust-check.json
new file mode 100644
index 000000000000..f2cc5b420397
--- /dev/null
+++ b/services/mcp/tests/unit/__snapshots__/tool-schemas/insight-trust-check.json
@@ -0,0 +1,28 @@
+{
+ "$schema": "https://json-schema.org/draft/2020-12/schema",
+ "properties": {
+ "id": {
+ "anyOf": [
+ {
+ "type": "number"
+ },
+ {
+ "type": "string"
+ }
+ ],
+ "description": "Numeric id or 8-character `short_id` of a saved insight. Pass this or `query`, not both."
+ },
+ "query": {
+ "additionalProperties": {},
+ "description": "An insight query that is not saved yet, in the shape `insight-create` takes, for example `{ \"kind\": \"InsightVizNode\", \"source\": { \"kind\": \"TrendsQuery\", \"series\": [{ \"kind\": \"EventsNode\", \"event\": \"$pageview\" }], \"dateRange\": { \"date_from\": \"-30d\" } } }`. Pass this or `id`, not both.",
+ "properties": {
+ "kind": {
+ "type": "string"
+ }
+ },
+ "required": ["kind"],
+ "type": "object"
+ }
+ },
+ "type": "object"
+}
diff --git a/services/mcp/tests/unit/insight-id-aliases.test.ts b/services/mcp/tests/unit/insight-id-aliases.test.ts
index df0900b43396..d45089daec1f 100644
--- a/services/mcp/tests/unit/insight-id-aliases.test.ts
+++ b/services/mcp/tests/unit/insight-id-aliases.test.ts
@@ -1,6 +1,7 @@
import { describe, expect, it } from 'vitest'
import { InsightQueryInputSchema } from '@/schema/tool-inputs'
+import { TOOL_MAP } from '@/tools'
import { GENERATED_TOOLS } from '@/tools/generated/product_analytics'
// Agents interchange `id` / `insightId` / `insight_id` / `short_id` / `shortId`
@@ -11,10 +12,10 @@ import { GENERATED_TOOLS } from '@/tools/generated/product_analytics'
describe('insight id aliases', () => {
const ALIAS_KEYS = ['insightId', 'insight_id', 'short_id', 'shortId'] as const
- describe.each([['insight-get'], ['insight-update'], ['insight-delete']])(
+ describe.each([['insight-get'], ['insight-update'], ['insight-delete'], ['insight-trust-check']])(
'%s normalizes aliases to `id`',
(toolName) => {
- const schema = GENERATED_TOOLS[toolName]!().schema
+ const schema = (GENERATED_TOOLS[toolName] ?? TOOL_MAP[toolName])!().schema
it.each([
['id (numeric)', { id: 123 }, 123],
diff --git a/services/mcp/tests/unit/tool-filtering.test.ts b/services/mcp/tests/unit/tool-filtering.test.ts
index 9352dcdb3e89..6e789fdfc5c7 100644
--- a/services/mcp/tests/unit/tool-filtering.test.ts
+++ b/services/mcp/tests/unit/tool-filtering.test.ts
@@ -1083,6 +1083,7 @@ describe('Tool Filtering - Feature Flags', () => {
'metrics-dashboard-import',
'managed-warehouse-trino-query',
'llm-analytics-tags',
+ 'insight-data-checks',
]
expect(allFlags).toEqual(expect.arrayContaining(branchFlags))
// The flags branches add are asserted on the line above and held out of the list and
diff --git a/services/mcp/tests/unit/tool-schema-snapshots.test.ts b/services/mcp/tests/unit/tool-schema-snapshots.test.ts
index 06cbdaeaa817..86a1ea36fba8 100644
--- a/services/mcp/tests/unit/tool-schema-snapshots.test.ts
+++ b/services/mcp/tests/unit/tool-schema-snapshots.test.ts
@@ -106,6 +106,7 @@ async function loadSnapshotTools(): Promise<Tool<ZodObjectAny>[]> {
'subscription-source-summaries': true,
'workflows-data-suggestions': true,
'workflows-mcp-onboarding': true,
+ 'insight-data-checks': true,
}
return [...(await getToolsFromContext(createMockContext(), { featureFlags }))].sort((a, b) =>
a.name.localeCompare(b.name)