:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
# MCP tool definitions. Tools are opt-in: an operation is exposed only when it has an entry here.
# List operations without an entry: pnpm --filter=@posthog/mcp run scaffold-yaml -- --candidates --product engineering_analytics
# Add one: pnpm --filter=@posthog/mcp run scaffold-yaml -- --add <operationId> --product engineering_analytics
# To keep an operation off on purpose, set enabled: false and give a disabled_reason.
category: Engineering analytics
feature: engineering_analytics
url_prefix: /engineering-analytics
ui_apps: {}
tools:
engineering-analytics-broken-tests:
operation: engineering_analytics_broken_tests
enabled: true
scopes:
- engineering_analytics:read
annotations:
readOnly: true
destructive: false
idempotent: true
title: Broken tests triage
description: >
Live CI failures over the last 2 days, grouped into distinct failures (by failing test id + a normalized
error signature) and classified by how each is behaving right now, most urgent first. Reach for this to
answer "what CI failures should I care about right now": it is the triage counterpart to the test-health
queue (which ranks week-long unreliability). Each row's state is the verdict: breaking_master (failing on
the default branch and that job's latest run is still red, so trunk is broken now), blocking_merge_queue
(failing only on merge-queue gate branches, so it stopped a merge on a commit that had already passed the
PR's own CI: the conflict-with-trunk class the queue exists to catch), novel_burst (first seen within a day
and already spreading across branches, not on trunk yet), potentially_resolved (hit trunk but that job is
green again), flaky (sporadic across two or more branches over more than a day), or pr_only (confined to one
branch: one PR's own problem). Also returns breaking_master_jobs, the default-branch jobs whose latest run
is red. Pass a row's latest_run_id and latest_ci_engine to run-failure-logs to read the failing lines.
Caveats: fingerprinting is pytest-only for now (jest/playwright/cargo failures aren't grouped yet), and the
breaking/resolved distinction needs the job-level source synced: without it those failures fall through to
flaky/pr_only rather than being misreported. Counts are absolute, never rates (passing runs aren't in this
data). The list stops at 200 rows, so to ask about one test or one PR pass test (a test id substring) and/or
branch (the PR's head branch): both apply before the cap, and a branch-scoped row still counts every branch,
so its branches and master_hits show whether the failure also happens elsewhere.
hooks: engineeringAnalytics/redactFailureLogsHooks
feature_flag: engineering-analytics
engineering-analytics-ci-failure-logs:
operation: engineering_analytics_ci_failure_logs
enabled: true
scopes:
- engineering_analytics:read
annotations:
readOnly: true
destructive: false
idempotent: true
title: CI failure logs for a PR
description: >
The thinned CI failure logs for a pull request, grouped by failed job: reach for this to answer "why did
this PR's CI fail". Each failed job returns its failure region (the error lines plus surrounding context,
with "... N lines omitted ..." markers) as ordered lines, and every line carries its original 1-based line
number in the full pre-thinning log so you can locate it in the raw GitHub log. The PR is resolved to ALL
its workflow runs across every push (via the pull_requests association, not the latest commit), so earlier
pushes' failures are included too. logs_available is false when CI hasn't failed, the logs aged out of the
short Logs retention, or a fork PR has no run association to resolve. Use pr-runs or workflow-jobs for the
run/job metadata behind these logs. Jobs carry ci_engine; null means unknown historical provenance.
hooks: engineeringAnalytics/redactFailureLogsHooks
feature_flag: engineering-analytics
engineering-analytics-depot-ci-usage:
operation: engineering_analytics_depot_ci_usage
enabled: false
disabled_reason: Billing totals for the weekly CI digest. Agents read cost per workflow or pull request.
engineering-analytics-dora:
operation: engineering_analytics_dora
enabled: false
disabled_reason: Shipped off until codegen ran for the new endpoint, then never enabled; enable after review
scopes:
- engineering_analytics:read
annotations:
readOnly: true
destructive: false
idempotent: true
title: DORA deploy metrics
description: >
DORA-style deploy metrics over the GitHub deployments + deployment_statuses warehouse pair, each headline
with its previous-window twin: deployment frequency (deploys whose first success status landed in the
window, per-day normalized, with a bucketed series), and lead time across three stages over the same
deployed-PR population and buckets: open_to_merge_series (PR open to merge), merge_to_deploy_series
(per-bucket distribution of seconds from a PR's merge to the first successful deployment that contains it,
bots and drafts excluded), and open_to_deploy_series (the full open-to-deploy span the other two compose
into): each a per-bucket min/p5/p25/p50/mean/p75/p95/max box-plot series, bucketed by series_granularity
(pass granularity to pin hour/day/week, otherwise it's chosen to fit the window), and two honest proxies:
failed_deployment_share (deployments with a failure/error status over deployments that reached an outcome:
NOT true change failure rate, since no incident data is linked) and
median_failed_deploy_to_next_success_seconds (first failure status to the next successful deploy in the same
environment: NOT true time-to-restore, since recovery by anything other than a deploy is invisible).
Environments default to all persistent production-marked or production-named environments (including
regional suffixes), falling back to the busiest persistent one when none match; ephemeral per-PR preview
environments never join a default scope. selected_environments lists the exact names used. Repeat
environment to scope to exact names; the API trims and deduplicates them, keeps real transient names, and
rejects blank or unknown names with a 400 response. Pass github_team (a slug from github_teams) to narrow
the PR-scoped lead-time figures to that team's authors: deploy counts stay repo-wide. deploy_data_available
is false when the deployments endpoints aren't synced on the GitHub source; the fix is enabling them, not
re-querying.
engineering-analytics-enable:
operation: engineering_analytics_enable
enabled: false
disabled_reason: The in-app setup state enables the product; an agent should not opt a project into background work.
engineering-analytics-flaky-tests:
operation: engineering_analytics_flaky_tests
enabled: true
scopes:
- engineering_analytics:read
annotations:
readOnly: true
destructive: false
idempotent: true
list: true
title: Active test health queue
description: >
Pytest and Jest tests worth acting on now, from the per-test CI spans over a window (date_from default -7d,
maximum 30 days), ranked by blast radius: master failures first, then how many distinct PRs the test broke.
Use this for "what is this failing test costing us" and to pick quarantine candidates. It does not measure
flake rates. This data sees the main Backend pytest and Frontend Jest suites; package-specific runners are
not included. Evidence is counted per CI run, never per span or run attempt: one run fans a test across
matrix legs, and re-run attempts re-test the same commit. classification is confirmed_flake only when one
commit both failed and passed the test (same_commit_recovery_run_count > 0): a "Re-run failed jobs" attempt
went green on the same commit, or an in-job pytest retry (tests hand-marked @pytest.mark.flaky(reruns=N))
recovered it. quarantined means a tolerated failure was recorded while the test was masked.
suspected_regression means only failures were recorded, which is absence of proof, not proof of a real
break: treat it as real until a recovery proves otherwise. A test qualifies on any same-commit recovery, any
master/main failure, a quarantined failure, or failures on at least min_failed_prs distinct PRs (default 3).
A CI setup break excludes every trial of the attempt, not only its failures: a run attempt whose tests
errored in three or more jobs or for three or more owning teams, or a job attempt with 100 or more distinct
failed or errored tests. Counts are absolute, never rates, because fast passing runs are not emitted. Use
selector to run or quarantine the test with its runner; nodeid is only the stable grouping key. To ask about
one test or one PR, pass test (a nodeid substring), branch, and/or pr_number: they apply before the limit,
and the counts still cover every PR and branch, so a row shows whether the test also fails elsewhere. Add
min_failed_prs=1 so a test that failed on that PR alone qualifies.
feature_flag: engineering-analytics
engineering-analytics-pr-cost:
operation: engineering_analytics_pr_cost
enabled: true
scopes:
- engineering_analytics:read
annotations:
readOnly: true
destructive: false
idempotent: true
title: Estimated CI cost of a PR
description: >
Estimated CI dollar cost of one pull request, summed over the jobs of all its workflow runs, with breakdowns
per workflow (by_workflow) and per run (by_run). Reach for this instead of summing runner minutes by hand:
the estimate already classifies runners (billable self-hosted Linux vs free GitHub-hosted), applies the
per-vCPU price ladder, and excludes non-Linux and provider-hosted jobs. jobs_available is false when the
job-level source isn't synced yet: every figure is then zero/null, so check it before trusting a $0.00.
estimated_cost_usd is null (not zero) when nothing was costable, and unsettled_jobs counts billable jobs
still running that are left out of the estimate. Key by_run by (ci_engine, run_id, run_attempt). This covers
curated work and excludes suppressed relay/wait overhead; it is not an invoice total.
feature_flag: engineering-analytics
engineering-analytics-run-failure-logs:
operation: engineering_analytics_run_failure_logs
enabled: true
scopes:
- engineering_analytics:read
annotations:
readOnly: true
destructive: false
idempotent: true
title: CI failure logs for a run
description: >
The thinned CI failure logs of one workflow run, grouped by failed job: the run-scoped twin of
ci-failure-logs, for when you have a run id rather than a PR (default-branch failures, the broken-tests
panel's latest_run_id, the run page). Each failed job returns its failure region (error lines plus
surrounding context, with "... N lines omitted ..." markers) as ordered lines, every line carrying its
original 1-based line number in the full pre-thinning log. logs_available is false when the run didn't fail
or its logs aged out of the short Logs retention. Use ci-failure-logs instead when you have a PR number and
want all its runs' failures. Pass ci_engine with run_id (latest_ci_engine for a triage row). Ambiguous
integer IDs return 400. Null engine means unknown historical provenance.
hooks: engineeringAnalytics/redactFailureLogsHooks
feature_flag: engineering-analytics
engineering-analytics-sources:
operation: engineering_analytics_sources
enabled: true
scopes:
- engineering_analytics:read
annotations:
readOnly: true
destructive: false
idempotent: true
list: true
title: List connected GitHub sources
description: >
The team's selectable GitHub repositories, oldest source first: one entry per repository a source syncs,
each with its source id, repo (owner/name), table prefix, and a synced flag. A source can sync several
repositories, so entries can share an id and differ only by repo. Call this first when a team may have more
than one source or repo, then pass a chosen entry's id as source_id AND its repo as repo to the
pull-requests, workflow-health, ci-cards, pr-lifecycle, or team-ci-health tools to read that specific repo:
omitting repo on a multi-repo source reads the source's default repo, not the one you picked. With a single
source/repo you can skip it: the other tools default to the oldest source whose entry is synced. The prefix
also names the underlying warehouse tables, which you can query directly with SQL or save as SQL insights
and subscriptions when the named tools don't fit. The source's original repo uses bare names (prefix +
github_pull_requests / github_workflow_runs / github_workflow_jobs); every other repo of a multi-repo source
is repo-qualified: the name `owner/repo.endpoint` is flattened for the table (each '/' becomes '_' and each
'.' becomes '__', lower-cased), e.g. posthog/posthog.com's pull_requests is prefix +
github_posthog_posthog__com__pull_requests: so pick the table for the repo you mean (timestamps land as
strings; parse with parseDateTimeBestEffort).
feature_flag: engineering-analytics
engineering-analytics-team-ci-health:
operation: engineering_analytics_team_ci_health
enabled: true
scopes:
- engineering_analytics:read
annotations:
readOnly: true
destructive: false
idempotent: true
list: true
title: Team CI health roster
description: >
Per-owning-team rollup of the CI test surfaces each team owns, over a window (date_from default -14d,
maximum 30 days). It reads the same per-run evidence as engineering-analytics-flaky-tests and means the same
thing by flaky: flaky_test_count is owned tests one commit was seen both failing and passing in the window
(a re-run attempt went green, or an in-job retry recovered it), regression_test_count is owned tests that
failed with no such proof and still hit min_failed_prs distinct PRs or master, and failed_run_count,
same_commit_recovery_run_count and quarantined_failed_run_count count distinct CI runs across the team's
owned tests, so one run with many failing tests counts once. A CI setup break excludes every trial of the
attempt, not only its failures: a run attempt whose tests errored in three or more jobs or for three or more
owning teams, or a job attempt with 100 or more distinct failed or errored tests. Each has an equal-length
previous-window twin (*_prior) for honest deltas. Use this for "which team owns the flakiest surfaces right
now" and to route a flaky-test or CI Signal to its owning team. Ownership is stamped on the spans at CI
emission time from the repo's ownership map (the distributed owners.yaml files); spans without a stamp
aggregate under the literal team 'unowned' (a nonempty 'unowned' row means ownership gaps, not a real team),
and a re-stamped test lands under its latest owner only. Each row also carries test_file_count (the daily
owners.yaml census denominator, with a window-start twin) and merged_pr_count (merged PRs by the team's
members, bots excluded); teams with census counts but no CI signal appear with zero signal counts and a null
last_seen_at. Teams are organizational owners of code surfaces, never authors. Read all counts as absolute
signal, never rates: fast passing runs are not emitted, so there is no honest denominator.
feature_flag: engineering-analytics
engineering-analytics-workflow-jobs:
operation: engineering_analytics_workflow_jobs
enabled: true
scopes:
- engineering_analytics:read
annotations:
readOnly: true
destructive: false
idempotent: true
list: true
title: Jobs of a workflow run
description: >
The jobs of a single workflow run attempt, each with its status, conclusion, wall-clock duration_seconds,
runner tier (runner_provider / runner_label), and estimated_cost_usd. Use this to see where a run spent its
time and money job-by-job, or to find the slow or expensive job in a run. Scoped to one run_attempt (the
latest unless you pass run_attempt) so a re-run's attempts don't merge. estimated_cost_usd is null for free
GitHub-hosted, non-Linux, or unfinished jobs. Each job carries its steps in run order (name, status,
conclusion, timestamps, duration_seconds), so you can find the step that failed or took the time; steps is
empty for a Depot CI job. Returns an empty list when the job-level source isn't synced yet. Pass ci_engine
with run_id when following an engine-qualified run; ambiguous integer IDs return 400.
feature_flag: engineering-analytics
engineering-analytics-workflow-runner-costs:
operation: engineering_analytics_workflow_runner_costs
enabled: true
scopes:
- engineering_analytics:read
annotations:
readOnly: true
destructive: false
idempotent: true
list: true
title: Workflow cost by runner tier
description: >
One workflow's estimated CI cost broken down by runner tier over a window (date_from default -30d), highest
spend first. Each row is a tier (provider + runner_label) with its job_count, billable_minutes, and
estimated_cost_usd. Use this for "which runner tier drives this workflow's CI bill" and to spot oversized
runners. Billable self-hosted Linux tiers carry a cost; github-hosted and non-Linux rows are null. Returns
an empty list when the job-level source isn't synced yet. Suppressed relay and wait overhead is excluded;
these curated estimates are not invoice totals.
feature_flag: engineering-analytics
pr-lifecycle:
operation: engineering_analytics_pr_lifecycle
enabled: true
scopes:
- engineering_analytics:read
annotations:
readOnly: true
destructive: false
idempotent: true
title: PR lifecycle timeline
description: >
The timeline of a single pull request: a header (number, title, author, state, draft, timestamps) plus
ordered events: opened, ready for review, converted to draft, CI started, CI finished, and merged or closed.
Use this to answer "where is this PR stuck and what happened to it". Draft/ready transitions appear only
when the issue-events table is synced and cover a bounded recent window. metric_quality is "partial": review
and comment events are not yet available. For aggregate questions use the pull-requests and workflow-health
tools.
feature_flag: engineering-analytics
pull-requests:
operation: engineering_analytics_pull_requests
enabled: true
scopes:
- engineering_analytics:read
annotations:
readOnly: true
destructive: false
idempotent: true
list: true
title: List pull requests with CI status
description: >
Open pull requests plus any merged or closed within the window, newest first, each with its head-SHA CI
rollup (runs / passing / failing / pending). Use this for "which PRs are stuck", "which open PRs have
failing or pending CI", and per-author or per-repo PR triage. open_to_merge_seconds is coarse: it is
merged_at minus created_at and fuses draft and ready-for-review time, so report it as "open to merge", never
"cycle time". ready_to_merge_seconds is the true cycle time: merged_at minus the last observed
ready-for-review transition (only the last draft/ready switch counts), or minus created_at for a merged PR
verifiably never drafted. Null means "not observed" (the issue-events window is bounded), never zero. CI
counts come from the latest run per workflow on the head SHA and can lag until the workflow_run webhook
settles a late completion. Cost and billable minutes can lag new CI by up to 5 minutes. Bots and drafts are
present; exclude them (author.is_bot, is_draft) for throughput questions. To list every PR merged in a
window, pass state=merged: open PRs then do not count against the page, and rows come newest merged_at first
with number as the tie-breaker. Pages hold up to 1000 rows; while truncated is true, call again with offset
increased by limit, and keep date_to fixed across pages so new merges do not move rows between pages.
param_overrides:
date_from:
description: >
Recency floor for merged/closed PRs: relative ('-30d', '-8w') or ISO8601. Open PRs are always
included regardless of age. Defaults to -30d.
feature_flag: engineering-analytics
workflow-health:
operation: engineering_analytics_workflow_health
enabled: true
scopes:
- engineering_analytics:read
annotations:
readOnly: true
destructive: false
idempotent: true
list: true
title: Per-workflow CI health
description: >
Per-workflow CI health over a window: run count, success rate, p50/p95 duration, and last failure time. Use
this for "is CI getting faster or slower", "which GitHub Actions workflow is the slow or flaky long pole",
and CI duration trends. Success rate covers runs that succeeded or ended in failure, timeout, startup
failure, or staleness. Skipped, cancelled, neutral, and action-required runs are excluded. p50/p95 are over
successful runs only, so superseded cancels and failed runs never bias the duration percentile. Runs under
10 seconds that did no work are excluded when longer successful runs exist. An all-fast workflow uses every
successful run. Pass run_scope to report on one group of runs: default_branch, pull_request (PR branches
only, same-repo PRs: fork PRs carry no PR attribution), merge_queue, or all. To get a trend, call this over
two adjacent windows and compare.
param_overrides:
date_from:
description: "Window start: relative ('-24h', '-7d') or ISO8601. Defaults to -24h."
date_to:
description: 'Window end: relative or ISO8601. Defaults to now.'
feature_flag: engineering-analytics
url_prefix: /engineering-analytics/workflows