:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(aeo): add a prompts tab to content autopilot (#122449)
Lucas Ricoycc29ad24e520
diff --git a/frontend/snapshots.yml b/frontend/snapshots.ymlindex c55915ccaeb3..48494e3b341d 100644--- a/frontend/snapshots.yml+++ b/frontend/snapshots.yml@@ -8161,9 +8161,9 @@ snapshots:products-web-analytics-content-autopilot-workspace--drafts-tab--light:hash: v1.k794b7964.68ef987ab841b7e5c388e882cbac22abafa8b029dd6ca7a6ed6f414e942259ac.FRDpQmpmPl51OBC3tAVrA1u3TWepGDCtKf9u5OEbERwproducts-web-analytics-content-autopilot-workspace--failed-run--dark:- hash: v1.k794b7964.3d8a16bf937d62edab696e840497fad20cb5065d7d706833bec861357516a490.brmGznNIzalassalSt8p3-pUe09akPMMOqgToVi7GV4+ hash: v1.k794b7964.c48edafc8558bc87f734cf385d527caaec50e6a9821a417c27ca244bcc4d88fe.pj4tRZDDFnab-PcNz0bNz65-kKNsIsIKSjL9M1Be4uEproducts-web-analytics-content-autopilot-workspace--failed-run--light:- hash: v1.k794b7964.5b838524626886f5d22ea592a8be8815bd4ba492b281f07679072a21099c9e87.COVxz4-01XJAxdibxcT5UdSyWYVL-g-8c0TPe0DSMKM+ hash: v1.k794b7964.7761dbde30e29ff2df4e8f2b769f47d07e1bc4bef49f8f02496e02cd92c8bf7b.nVtN7AZtGaCMqtG3f4GZtKVxN5zATkeu-9jPNIodxnIproducts-web-analytics-content-autopilot-workspace--multiple-sites--dark:hash: v1.k794b7964.b59bcd4873161e4d1a649a7acf03a84e86f1e4bd063a0418833288e4b4995422.1P0EVXKwtUq4LrVS9jlKO_tbPn6kLp8nmpiAyVZ8GG0products-web-analytics-content-autopilot-workspace--multiple-sites--light:diff --git a/frontend/src/initKea.ts b/frontend/src/initKea.tsindex 22afad8e652f..c7270c2119be 100644--- a/frontend/src/initKea.ts+++ b/frontend/src/initKea.ts@@ -121,6 +121,9 @@ const ERROR_FILTER_ALLOW_LIST_BY_LOGIC_PATH: Record<string, string> = {loadProposals: 'products.webAnalytics.contentAutopilot.contentAutopilotLogic',loadOpportunities: 'products.webAnalytics.contentAutopilot.contentAutopilotLogic',refreshOpportunities: 'products.webAnalytics.contentAutopilot.contentAutopilotLogic',+ // The prompts view shows its own error, and falls back to an unfiltered list when topics fail to load+ loadAeoPrompts: 'products.webAnalytics.contentAutopilot.contentAutopilotPromptsLogic',+ loadAeoPromptTopics: 'products.webAnalytics.contentAutopilot.contentAutopilotPromptsLogic',// llmProviderKeysLogic toasts each of these failures itself, or shows a rejected field under its inputloadProviderKeys: 'products.ai_observability.settings.llmProviderKeysLogic',loadEvaluationConfig: 'products.ai_observability.settings.llmProviderKeysLogic',diff --git a/products/aeo/README.md b/products/aeo/README.mdindex 62acd35f08f7..271ec6fa8414 100644--- a/products/aeo/README.md+++ b/products/aeo/README.md@@ -41,7 +41,7 @@ python manage.py run_aeo_citation_checks --team-id <id>### Import over the API-`/api/projects/<id>/aeo_prompts/` lists the prompt set, turns a prompt on or off (`PATCH {"active": false}`), and imports prompts with `POST bulk_import/`. It is gated by the `aeo-citation-tracking` flag and accepts personal API keys with `web_analytics:write`:+`/api/projects/<id>/aeo_prompts/` lists the prompt set (filter with `?topic=` and `?active=true|false`; each prompt carries `latest_checks`, its newest check per engine), lists topics with `GET topics/`, turns a prompt on or off (`PATCH {"active": false}`), and imports prompts with `POST bulk_import/`. It is gated by the `aeo-citation-tracking` flag and accepts personal API keys with `web_analytics:write`:```bashcurl -X POST "$HOST/api/projects/<id>/aeo_prompts/bulk_import/" \@@ -51,6 +51,7 @@ curl -X POST "$HOST/api/projects/<id>/aeo_prompts/bulk_import/" \- A request takes up to 2,000 prompts. Send larger sets in batches of a few hundred.- A re-import matches prompts by normalized text. It updates the topic and the evidence, and keeps each prompt's on/off state.+- The same operations are PostHog MCP tools (`products/aeo/mcp/tools.yaml`), gated by the same flag: `aeo-prompts-list`, `aeo-prompts-get`, `aeo-prompts-update`, `aeo-prompts-import` and `aeo-prompts-topics-list`.- `external_visibility` (0-100) is the share of answers that mention the brand, as reported by the tool the set came from. It is kept in the prompt's evidence to benchmark against our own rates.### Which prompts a run takes@@ -98,7 +99,7 @@ WHERE $channel_type = 'AI' AND $entry_pathname = '/docs/session-replay'## Content autopilot-Web analytics' Content autopilot (`/web/content-autopilot`, flag `web-analytics-content-autopilot`) reads this table through `list_citation_gaps` in `facade/api.py`. Each active prompt that an engine answered without citing a target domain becomes a content opportunity, and a person picks which ones to draft. Drafting and its review flow live in `products/web_analytics/backend/content_autopilot/`.+Web analytics' Content autopilot (`/web/content-autopilot`, flag `web-analytics-content-autopilot`) reads this table through `list_citation_gaps` in `facade/api.py`. Each active prompt that an engine answered without citing a target domain becomes a content opportunity, and a person picks which ones to draft. Its Prompts tab lists the prompt set with each prompt's latest result per engine, and a switch on each row turns the prompt on or off. Adding prompts or changing their text goes through the API or the MCP tools. Drafting and its review flow live in `products/web_analytics/backend/content_autopilot/`.## Cost@@ -106,4 +107,4 @@ Roughly \$2–4/day at 50 prompts × 3 engines × 1 run/day: provider web-search## Out of scope-Seeding prompts from first-party data, a UI for the prompt set, sentiment/quality scoring, rank tracking, competitor share-of-voice, new ClickHouse tables, consumer-surface (web UI) checking, multi-tenant rollout.+Seeding prompts from first-party data, editing the prompt set in the UI, sentiment/quality scoring, rank tracking, competitor share-of-voice, new ClickHouse tables, consumer-surface (web UI) checking, multi-tenant rollout.diff --git a/products/aeo/backend/facade/api.py b/products/aeo/backend/facade/api.pyindex 24c95cb8f584..be5863f44505 100644--- a/products/aeo/backend/facade/api.py+++ b/products/aeo/backend/facade/api.py@@ -6,9 +6,12 @@ capabilities. Celery wiring is re-exported from facade/tasks.py.from __future__ import annotations+import uuidimport datetime as dt+from collections import defaultdictfrom django.core.exceptions import ValidationError+from django.db.models import Countfrom posthog.models.team import Team@@ -17,11 +20,13 @@ from products.aeo.backend.facade.contracts import (CitationGap,CitationRunSummary,Prompt,+ PromptEngineCheck,PromptImport,PromptImportResult,PromptNotFoundError,+ PromptTopic,)-from products.aeo.backend.models import AEOPrompt+from products.aeo.backend.models import AEOCitationCheck, AEOPromptfrom products.aeo.backend.runner import run_citation_checksfrom products.aeo.backend.seeding import PromptCandidate, upsert_prompts@@ -38,22 +43,35 @@ def list_citation_gaps(team_id: int, *, since: dt.datetime) -> list[CitationGap]return gaps.list_citation_gaps(team_id, since=since)-def list_prompts(team_id: int, *, limit: int, offset: int) -> tuple[list[Prompt], int]:- """One page of the team's prompt set and the total count, paged in SQL."""+def list_prompts(+ team_id: int, *, limit: int, offset: int, topic: str | None = None, active: bool | None = None+) -> tuple[list[Prompt], int]:+ """One page of the team's prompt set and the total count, paged in SQL.+ A topic of "" selects the prompts that have no topic."""prompts = AEOPrompt.objects.for_team(team_id)- page = prompts.order_by("topic", "-rank", "created_at", "id")[offset : offset + limit]- return [_to_prompt(p) for p in page], prompts.count()+ if topic is not None:+ prompts = prompts.filter(topic=topic)+ if active is not None:+ prompts = prompts.filter(active=active)+ page = list(prompts.order_by("topic", "-rank", "created_at", "id")[offset : offset + limit])+ latest_checks = _latest_checks(team_id, [prompt.id for prompt in page])+ return [_to_prompt(prompt, latest_checks.get(prompt.id, ())) for prompt in page], prompts.count()+++def list_prompt_topics(team_id: int) -> list[PromptTopic]:+ rows = AEOPrompt.objects.for_team(team_id).values("topic").annotate(prompts=Count("id")).order_by("topic")+ return [PromptTopic(topic=row["topic"], prompts=row["prompts"]) for row in rows]def get_prompt(team_id: int, prompt_id: str) -> Prompt:- return _to_prompt(_get_prompt_row(team_id, prompt_id))+ return _with_latest_checks(team_id, _get_prompt_row(team_id, prompt_id))def set_prompt_active(team_id: int, prompt_id: str, active: bool) -> Prompt:prompt = _get_prompt_row(team_id, prompt_id)prompt.active = activeprompt.save(update_fields=["active", "updated_at"])- return _to_prompt(prompt)+ return _with_latest_checks(team_id, prompt)def import_prompts(team_id: int, prompts: list[PromptImport]) -> PromptImportResult:@@ -81,7 +99,34 @@ def _get_prompt_row(team_id: int, prompt_id: str) -> AEOPrompt:raise PromptNotFoundError(prompt_id) from error-def _to_prompt(prompt: AEOPrompt) -> Prompt:+def _latest_checks(team_id: int, prompt_ids: list[uuid.UUID]) -> dict[uuid.UUID, tuple[PromptEngineCheck, ...]]:+ if not prompt_ids:+ return {}+ rows = (+ AEOCitationCheck.objects.for_team(team_id)+ .filter(prompt_id__in=prompt_ids)+ .order_by("prompt_id", "engine", "-created_at")+ .distinct("prompt_id", "engine")+ .values("prompt_id", "engine", "cited", "check_failed", "created_at")+ )+ checks: dict[uuid.UUID, list[PromptEngineCheck]] = defaultdict(list)+ for row in rows:+ checks[row["prompt_id"]].append(+ PromptEngineCheck(+ engine=row["engine"],+ cited=row["cited"],+ check_failed=row["check_failed"],+ checked_at=row["created_at"],+ )+ )+ return {prompt_id: tuple(engine_checks) for prompt_id, engine_checks in checks.items()}+++def _with_latest_checks(team_id: int, prompt: AEOPrompt) -> Prompt:+ return _to_prompt(prompt, _latest_checks(team_id, [prompt.id]).get(prompt.id, ()))+++def _to_prompt(prompt: AEOPrompt, latest_checks: tuple[PromptEngineCheck, ...]) -> Prompt:visibility = (prompt.evidence or {}).get("external_visibility")return Prompt(id=str(prompt.id),@@ -94,4 +139,5 @@ def _to_prompt(prompt: AEOPrompt) -> Prompt:external_visibility=float(visibility) if isinstance(visibility, int | float) else None,created_at=prompt.created_at,updated_at=prompt.updated_at,+ latest_checks=latest_checks,)diff --git a/products/aeo/backend/facade/contracts.py b/products/aeo/backend/facade/contracts.pyindex 142c33c7dcb9..a959103138c0 100644--- a/products/aeo/backend/facade/contracts.py+++ b/products/aeo/backend/facade/contracts.py@@ -59,6 +59,16 @@ AEO_CITATION_TRACKING_FLAG = "aeo-citation-tracking"PROMPT_TOPIC_MAX_LENGTH = 200+@frozen+class PromptEngineCheck:+ """The most recent check of one prompt against one answer engine."""++ engine: str+ cited: bool+ check_failed: bool+ checked_at: dt.datetime++@frozenclass Prompt:id: str@@ -71,6 +81,13 @@ class Prompt:external_visibility: float | Nonecreated_at: dt.datetimeupdated_at: dt.datetime | None+ latest_checks: tuple[PromptEngineCheck, ...]+++@frozen+class PromptTopic:+ topic: str+ prompts: int@frozendiff --git a/products/aeo/backend/presentation/views.py b/products/aeo/backend/presentation/views.pyindex 7ed448ba4f94..192a141aee79 100644--- a/products/aeo/backend/presentation/views.py+++ b/products/aeo/backend/presentation/views.py@@ -1,7 +1,7 @@import mathfrom typing import Any, cast-from drf_spectacular.utils import OpenApiResponse, extend_schema+from drf_spectacular.utils import OpenApiResponsefrom rest_framework import serializers, viewsetsfrom rest_framework.decorators import actionfrom rest_framework.exceptions import NotFound, PermissionDenied@@ -14,14 +14,22 @@ from posthog.api.pagination import PrecountedLimitOffsetPaginationfrom posthog.api.routing import TeamAndOrgViewSetMixinfrom posthog.permissions import posthog_feature_flag_enabled-from products.aeo.backend.facade.api import get_prompt, import_prompts, list_prompts, set_prompt_active+from products.aeo.backend.facade.api import (+ get_prompt,+ import_prompts,+ list_prompt_topics,+ list_prompts,+ set_prompt_active,+)from products.aeo.backend.facade.contracts import (AEO_CITATION_TRACKING_FLAG,PROMPT_TOPIC_MAX_LENGTH,Prompt,+ PromptEngineCheck,PromptImport,PromptImportResult,PromptNotFoundError,+ PromptTopic,)MAX_PROMPTS_PER_IMPORT = 2000@@ -35,7 +43,28 @@ EXTERNAL_VISIBILITY_HELP = ()+class AEOPromptEngineCheckSerializer(DataclassSerializer):+ class Meta:+ dataclass = PromptEngineCheck+ ref_name = "AEOPromptEngineCheck"+ extra_kwargs = {+ "engine": {+ "help_text": "Answer engine that ran the prompt, for example 'claude-web-search', 'openai-web-search' or 'exa-answer'."+ },+ "cited": {"help_text": "Whether the answer cited a URL on one of the target domains."},+ "check_failed": {+ "help_text": "The engine did not answer. When true, 'cited' is false and says nothing about citations."+ },+ "checked_at": {"help_text": "When the check ran."},+ }++class AEOPromptSerializer(DataclassSerializer):+ latest_checks = AEOPromptEngineCheckSerializer(+ many=True,+ help_text="The most recent check for each answer engine that ran the prompt. Empty when the prompt never ran.",+ )+class Meta:dataclass = Promptref_name = "AEOPrompt"@@ -53,6 +82,32 @@ class AEOPromptSerializer(DataclassSerializer):}+class AEOPromptListQuerySerializer(serializers.Serializer):+ topic = serializers.CharField(+ required=False,+ allow_blank=True,+ max_length=PROMPT_TOPIC_MAX_LENGTH,+ help_text="Only list prompts with this topic. An empty value lists the prompts that have no topic.",+ )+ # A QueryDict reads a missing boolean as false, so the default keeps "no filter" distinct from active=false.+ active = serializers.BooleanField(+ required=False,+ allow_null=True,+ default=None,+ help_text="Only list prompts that are on (true) or off (false). Omit to list both.",+ )+++class AEOPromptTopicSerializer(DataclassSerializer):+ class Meta:+ dataclass = PromptTopic+ ref_name = "AEOPromptTopic"+ extra_kwargs = {+ "topic": {"help_text": "Topic name. Empty for prompts that have no topic."},+ "prompts": {"help_text": "How many prompts in the set have this topic."},+ }++class AEOPromptUpdateRequestSerializer(serializers.Serializer):active = serializers.BooleanField(help_text="Turn the prompt on or off for future runs.")@@ -104,7 +159,7 @@ class AEOPromptImportResultSerializer(DataclassSerializer):class AEOPromptViewSet(TeamAndOrgViewSetMixin, viewsets.GenericViewSet):scope_object = "web_analytics"- scope_object_read_actions = ["list", "retrieve"]+ scope_object_read_actions = ["list", "retrieve", "topics"]scope_object_write_actions = ["partial_update", "bulk_import"]pagination_class = PrecountedLimitOffsetPagination@@ -116,17 +171,22 @@ class AEOPromptViewSet(TeamAndOrgViewSetMixin, viewsets.GenericViewSet):):raise PermissionDenied("This feature is not available.")- @extend_schema(+ @validated_request(+ query_serializer=AEOPromptListQuerySerializer,operation_id="aeo_prompts_list",summary="List AEO prompts",- description="Lists the project's AEO prompt set, grouped by topic, one page at a time.",+ description="Lists the project's AEO prompt set, ordered by topic and then rank, one page at a time. Filter by topic or by on/off state. Each prompt carries its latest check per answer engine.",responses={200: AEOPromptSerializer(many=True)},tags=["web_analytics"],)- def list(self, request: Request, **kwargs: Any) -> Response:+ def list(self, request: ValidatedRequest, **kwargs: Any) -> Response:paginator = cast(PrecountedLimitOffsetPagination, self.paginator)prompts, count = list_prompts(- self.team_id, limit=paginator.get_limit(request) or DEFAULT_PAGE_SIZE, offset=paginator.get_offset(request)+ self.team_id,+ limit=paginator.get_limit(request) or DEFAULT_PAGE_SIZE,+ offset=paginator.get_offset(request),+ topic=request.validated_query_data.get("topic"),+ active=request.validated_query_data.get("active"),)paginator.set_count(count)page = self.paginate_queryset(prompts)@@ -135,6 +195,7 @@ class AEOPromptViewSet(TeamAndOrgViewSetMixin, viewsets.GenericViewSet):@validated_request(operation_id="aeo_prompts_retrieve",summary="Get an AEO prompt",+ description="Gets one prompt from the project's AEO prompt set, with its latest check per answer engine.",responses={200: OpenApiResponse(response=AEOPromptSerializer)},tags=["web_analytics"],)@@ -148,6 +209,7 @@ class AEOPromptViewSet(TeamAndOrgViewSetMixin, viewsets.GenericViewSet):request_serializer=AEOPromptUpdateRequestSerializer,operation_id="aeo_prompts_partial_update",summary="Turn an AEO prompt on or off",+ description="Turns a prompt on or off. Only active prompts run against the answer engines. Past checks are kept either way.",responses={200: OpenApiResponse(response=AEOPromptSerializer)},tags=["web_analytics"],)@@ -158,6 +220,17 @@ class AEOPromptViewSet(TeamAndOrgViewSetMixin, viewsets.GenericViewSet):raise NotFound()return Response(AEOPromptSerializer(prompt).data)+ @validated_request(+ operation_id="aeo_prompts_topics",+ summary="List AEO prompt topics",+ description="Lists the topics in the project's AEO prompt set, with the number of prompts in each.",+ responses={200: OpenApiResponse(response=AEOPromptTopicSerializer(many=True))},+ tags=["web_analytics"],+ )+ @action(detail=False, methods=["get"], pagination_class=None)+ def topics(self, request: Request, **kwargs: Any) -> Response:+ return Response(AEOPromptTopicSerializer(list_prompt_topics(self.team_id), many=True).data)+@validated_request(request_serializer=AEOPromptBulkImportRequestSerializer,operation_id="aeo_prompts_bulk_import",diff --git a/products/aeo/backend/test/test_api.py b/products/aeo/backend/test/test_api.pyindex 0539dfb19abc..2ae4d3be7eb1 100644--- a/products/aeo/backend/test/test_api.py+++ b/products/aeo/backend/test/test_api.py@@ -1,13 +1,21 @@+import uuid+import datetime as dt+from posthog.test.base import APIBaseTestfrom unittest.mock import patch+from django.db import connection+from django.test.utils import CaptureQueriesContext+from django.utils import timezone+from parameterized import parameterizedfrom rest_framework import statusfrom posthog.models.personal_api_key import PersonalAPIKey, hash_key_value+from posthog.models.scoping import team_scopefrom posthog.models.utils import generate_random_token_personal-from products.aeo.backend.models import AEOPrompt+from products.aeo.backend.models import AEOCitationCheck, AEOPromptclass TestAEOPromptAPI(APIBaseTest):@@ -21,6 +29,27 @@ class TestAEOPromptAPI(APIBaseTest):self.addCleanup(flag_patcher.stop)self.url = f"/api/projects/{self.team.id}/aeo_prompts/"+ def _record_check(+ self, prompt_id: str, engine: str, *, minutes_ago: int, cited: bool = False, check_failed: bool = False+ ) -> None:+ with team_scope(self.team.pk):+ prompt = AEOPrompt.objects.get(id=prompt_id)+ check = AEOCitationCheck.objects.create(+ team=self.team,+ prompt=prompt,+ run_id=uuid.uuid4(),+ prompt_text=prompt.prompt,+ prompt_source=prompt.prompt_source,+ prompt_hash=prompt.prompt_hash,+ engine=engine,+ model=engine,+ cited=cited,+ check_failed=check_failed,+ )+ AEOCitationCheck.objects.filter(id=check.id).update(+ created_at=timezone.now() - dt.timedelta(minutes=minutes_ago)+ )+def test_requires_the_rollout_flag(self) -> None:self.feature_enabled.return_value = False@@ -56,17 +85,75 @@ class TestAEOPromptAPI(APIBaseTest):assert response.json() == {"created": 0, "updated": 2, "skipped": 1}assert AEOPrompt.objects.for_team(self.team.id).get(id=replay_id).active is False- def test_list_pages_without_overlap(self) -> None:- prompts = [{"prompt": f"q{i}?", "topic": "same topic"} for i in range(5)]+ @parameterized.expand(+ [+ ("no_filter", "", 6, None),+ ("one_topic", "&topic=same topic", 5, ("topic", "same topic")),+ ("no_topic", "&topic=", 1, ("topic", "")),+ ("active", "&active=true", 5, ("active", True)),+ ("inactive", "&active=false", 1, ("active", False)),+ ]+ )+ def test_list_pages_without_overlap(+ self, _name: str, filters: str, expected: int, field_value: tuple[str, object] | None+ ) -> None:+ prompts = [{"prompt": f"q{i}?", "topic": "same topic"} for i in range(5)] + [{"prompt": "untopiced?"}]self.client.post(f"{self.url}bulk_import/", {"prompts": prompts}, format="json")+ untopiced = next(row for row in self.client.get(self.url).json()["results"] if row["topic"] == "")+ self.client.patch(f"{self.url}{untopiced['id']}/", {"active": False}, format="json")- first = self.client.get(f"{self.url}?limit=3").json()- second = self.client.get(f"{self.url}?limit=3&offset=3").json()+ first = self.client.get(f"{self.url}?limit=3{filters}").json()+ second = self.client.get(f"{self.url}?limit=3&offset=3{filters}").json()- assert first["count"] == 5- assert first["next"] is not None and second["next"] is None+ assert first["count"] == expectedids = [row["id"] for row in first["results"] + second["results"]]- assert len(ids) == len(set(ids)) == 5+ assert len(ids) == len(set(ids)) == expected+ if field_value is not None:+ field, value = field_value+ assert {row[field] for row in first["results"] + second["results"]} == {value}++ def test_topics_lists_each_topic_with_its_prompt_count(self) -> None:+ prompts = [{"prompt": "a?", "topic": "Replay"}, {"prompt": "b?", "topic": "Replay"}, {"prompt": "c?"}]+ self.client.post(f"{self.url}bulk_import/", {"prompts": prompts}, format="json")++ response = self.client.get(f"{self.url}topics/")++ assert response.status_code == status.HTTP_200_OK, response.json()+ assert response.json() == [{"topic": "", "prompts": 1}, {"topic": "Replay", "prompts": 2}]++ def test_list_returns_the_latest_check_per_engine(self) -> None:+ prompts = [{"prompt": "checked?", "topic": "a"}, {"prompt": "never checked?", "topic": "b"}]+ self.client.post(f"{self.url}bulk_import/", {"prompts": prompts}, format="json")+ ids = {row["prompt"]: row["id"] for row in self.client.get(self.url).json()["results"]}+ checked_id = ids["checked?"]+ self._record_check(checked_id, "exa-answer", minutes_ago=60, cited=True)+ self._record_check(checked_id, "exa-answer", minutes_ago=5, cited=False)+ self._record_check(checked_id, "claude-web-search", minutes_ago=90, check_failed=True)+ self._record_check(checked_id, "claude-web-search", minutes_ago=30, cited=True)++ listed = {row["prompt"]: row for row in self.client.get(self.url).json()["results"]}+ retrieved = self.client.get(f"{self.url}{checked_id}/").json()++ for row in (listed["checked?"], retrieved):+ latest = {check["engine"]: check for check in row["latest_checks"]}+ assert len(row["latest_checks"]) == 2+ assert (latest["exa-answer"]["cited"], latest["exa-answer"]["check_failed"]) == (False, False)+ assert (latest["claude-web-search"]["cited"], latest["claude-web-search"]["check_failed"]) == (True, False)+ assert listed["never checked?"]["latest_checks"] == []++ def test_list_query_count_does_not_grow_with_page_size(self) -> None:+ prompts = [{"prompt": f"q{i}?"} for i in range(4)]+ self.client.post(f"{self.url}bulk_import/", {"prompts": prompts}, format="json")+ for row in self.client.get(self.url).json()["results"]:+ self._record_check(row["id"], "exa-answer", minutes_ago=1)+ self._record_check(row["id"], "claude-web-search", minutes_ago=1)++ with CaptureQueriesContext(connection) as one_prompt:+ self.client.get(f"{self.url}?limit=1")+ with CaptureQueriesContext(connection) as four_prompts:+ self.client.get(f"{self.url}?limit=4")++ assert len(four_prompts.captured_queries) == len(one_prompt.captured_queries)@parameterized.expand([("above_range", 140), ("not_a_number", "NaN"), ("infinite", "Infinity")])def test_bulk_import_rejects_invalid_visibility(self, _name: str, visibility: object) -> None:diff --git a/products/aeo/mcp/tools.yaml b/products/aeo/mcp/tools.yamlnew file mode 100644index 000000000000..09c2eb7a318f--- /dev/null+++ b/products/aeo/mcp/tools.yaml@@ -0,0 +1,77 @@+# MCP tool definitions. Tools are opt-in: an operation is exposed only when it has an entry here.+# List operations without an entry: pnpm --filter=@posthog/mcp run scaffold-yaml -- --candidates --product aeo+# Add one: pnpm --filter=@posthog/mcp run scaffold-yaml -- --add <operationId> --product aeo+# To keep an operation off on purpose, set enabled: false and give a disabled_reason.+category: AI citation tracking+feature: aeo+url_prefix: /web/content-autopilot+ui_apps: {}+tools:+ aeo-prompts-get:+ operation: aeo_prompts_retrieve+ enabled: true+ title: Get a tracked AI prompt+ description: >+ Get one AI citation tracking prompt by id, with its topic, on/off state, when it last ran, and+ `latest_checks`, the newest result per answer engine. Get the id from `aeo-prompts-list`.+ param_overrides:+ id:+ description: The prompt id (a UUID), from `aeo-prompts-list`.+ feature_flag: aeo-citation-tracking+ aeo-prompts-import:+ operation: aeo_prompts_bulk_import+ enabled: true+ annotations:+ readOnly: false+ destructive: false+ idempotent: true+ title: Import AI prompts to track+ description: >+ Add prompts to the project's AI citation tracking set, up to 2000 per call. Each item has the `prompt` text,+ an optional `topic` and an optional `external_visibility` (0-100). Prompts are matched to existing ones by+ normalized text (case and whitespace ignored), so importing the same set again does not create duplicates: a+ match updates the topic and visibility and keeps the prompt's on/off state. Empty prompts, and prompts too+ long for the answer engines, are skipped. Returns the `created`, `updated` and `skipped` counts. New prompts+ run in the next daily check. Write prompts the way a person would ask an AI assistant, and ask the user to+ confirm the list before you import it.+ feature_flag: aeo-citation-tracking+ aeo-prompts-list:+ operation: aeo_prompts_list+ enabled: true+ list: true+ title: List tracked AI prompts+ description: >+ List the project's AI citation tracking prompts: the questions PostHog asks AI answer engines every day to+ check whether their answers cite the project's site. Each prompt has its text, topic, on/off state+ (`active`), when it last ran (`last_checked_at`) and `latest_checks`, the newest result per engine (`cited`,+ `check_failed`, `checked_at`). Filter by `topic` (use `aeo-prompts-topics-list` for the valid values; an+ empty string selects prompts with no topic) and by `active`. Results are paginated with `limit`/`offset` and+ ordered by topic, then rank. Use it to find prompts that are not cited, to audit the prompt set, or to find+ a prompt id before calling `aeo-prompts-update`.+ feature_flag: aeo-citation-tracking+ aeo-prompts-topics-list:+ operation: aeo_prompts_topics+ enabled: true+ title: List tracked AI prompt topics+ description: >+ List the topics in the project's AI citation tracking prompt set, with the number of prompts in each. An+ empty topic groups the prompts that have no topic. Use the values as the `topic` filter of+ `aeo-prompts-list`.+ feature_flag: aeo-citation-tracking+ aeo-prompts-update:+ operation: aeo_prompts_partial_update+ enabled: true+ annotations:+ readOnly: false+ destructive: false+ idempotent: true+ title: Turn a tracked AI prompt on or off+ description: >+ Turn one AI citation tracking prompt on (`active: true`) or off (`active: false`). Only active prompts run+ in the daily check. Turning a prompt off keeps its past results, and turning it on again resumes the checks.+ This is the only field you can change; the prompt text and topic change only through `aeo-prompts-import`.+ Returns the updated prompt.+ param_overrides:+ id:+ description: The prompt id (a UUID), from `aeo-prompts-list`.+ feature_flag: aeo-citation-trackingdiff --git a/products/web_analytics/frontend/contentAutopilot/ContentAutopilot.tsx b/products/web_analytics/frontend/contentAutopilot/ContentAutopilot.tsxindex c9e86240529e..f54016a03acf 100644--- a/products/web_analytics/frontend/contentAutopilot/ContentAutopilot.tsx+++ b/products/web_analytics/frontend/contentAutopilot/ContentAutopilot.tsx@@ -15,6 +15,7 @@ import {import { ContentAutopilotDrafts } from './ContentAutopilotDrafts'import { contentAutopilotLogic } from './contentAutopilotLogic'import { ContentAutopilotOpportunities } from './ContentAutopilotOpportunities'+import { ContentAutopilotPrompts } from './ContentAutopilotPrompts'import { ContentAutopilotProposalDetail } from './ContentAutopilotProposalDetail'import { ContentAutopilotSetup } from './ContentAutopilotSetup'@@ -147,6 +148,11 @@ export const ContentAutopilot = (): JSX.Element => {: `Drafts (${draftRows.length})`,content: <ContentAutopilotDrafts />,},+ {+ key: 'prompts',+ label: 'Prompts',+ content: <ContentAutopilotPrompts />,+ },]}/></>diff --git a/products/web_analytics/frontend/contentAutopilot/ContentAutopilotOpportunities.tsx b/products/web_analytics/frontend/contentAutopilot/ContentAutopilotOpportunities.tsxindex c0b56977c6ed..0d6d34a143d1 100644--- a/products/web_analytics/frontend/contentAutopilot/ContentAutopilotOpportunities.tsx+++ b/products/web_analytics/frontend/contentAutopilot/ContentAutopilotOpportunities.tsx@@ -22,12 +22,7 @@ import type { ContentAutopilotOpportunityApi } from 'products/web_analytics/fronimport { MAX_DRAFTS_PER_RUN, contentAutopilotLogic } from './contentAutopilotLogic'import { ContentAutopilotPageToUpdate } from './ContentAutopilotPageToUpdate'import { ContentAutopilotProposalStatusTag } from './ContentAutopilotProposalStatusTag'--const ENGINE_LABELS: Record<string, string> = {- 'claude-web-search': 'Claude',- 'openai-web-search': 'ChatGPT',- 'exa-answer': 'Exa',-}+import { engineLabel } from './utils'const STATUS_TAGS: Record<ContentAutopilotOpportunityApi['status'], JSX.Element | null> = {new: null,@@ -40,8 +35,6 @@ const STATUS_TAGS: Record<ContentAutopilotOpportunityApi['status'], JSX.Elementdismissed: <LemonTag type="muted">Dismissed</LemonTag>,}-const engineLabel = (engine: string): string => ENGINE_LABELS[engine] ?? engine-const visibilityTag = (gap: ContentAutopilotOpportunityApi['gap']): JSX.Element => {const answers = pluralize(gap.checks, 'answer')if (gap.cited_checks > 0) {diff --git a/products/web_analytics/frontend/contentAutopilot/ContentAutopilotPrompts.tsx b/products/web_analytics/frontend/contentAutopilot/ContentAutopilotPrompts.tsxnew file mode 100644index 000000000000..e9e81bdfaa5a--- /dev/null+++ b/products/web_analytics/frontend/contentAutopilot/ContentAutopilotPrompts.tsx@@ -0,0 +1,171 @@+import { useActions, useValues } from 'kea'++import {+ LemonBanner,+ LemonCard,+ LemonSelect,+ LemonSkeleton,+ LemonSwitch,+ LemonTable,+ LemonTag,+} from '@posthog/lemon-ui'++import { TZLabel } from 'lib/components/TZLabel'++import type { AEOPromptApi } from 'products/web_analytics/frontend/generated/api.schemas'++import { PROMPTS_PAGE_SIZE, contentAutopilotPromptsLogic } from './contentAutopilotPromptsLogic'+import { engineLabel } from './utils'++const checkTag = (prompt: AEOPromptApi, engine: string): JSX.Element => {+ const check = prompt.latest_checks.find((latest) => latest.engine === engine)+ if (!check) {+ return <span className="text-muted whitespace-nowrap">Not checked</span>+ }+ if (check.check_failed) {+ return <LemonTag type="danger">Failed</LemonTag>+ }+ return check.cited ? <LemonTag type="success">Cited</LemonTag> : <LemonTag type="warning">Not cited</LemonTag>+}++export const ContentAutopilotPrompts = (): JSX.Element => {+ const {+ prompts,+ promptsLoading,+ promptsError,+ promptsPage,+ promptsTopic,+ promptTopics,+ promptEngines,+ promptActiveUpdating,+ } = useValues(contentAutopilotPromptsLogic)+ const { loadAeoPrompts, setPromptsPage, setPromptsTopic, setPromptActive } =+ useActions(contentAutopilotPromptsLogic)++ if (promptsError === 'forbidden') {+ return (+ <LemonCard hoverEffect={false} className="p-6 text-center">+ <h3 className="m-0">AI citation tracking is not turned on</h3>+ <p className="m-0 mt-2 text-muted max-w-xl mx-auto">+ This project doesn't have access to prompt tracking yet. Ask PostHog support to turn on AI citation+ tracking for it.+ </p>+ </LemonCard>+ )+ }++ const count = prompts?.count ?? 0++ return (+ <section className="flex flex-col gap-3">+ <p className="m-0 text-muted max-w-3xl">+ The questions PostHog asks AI assistants every day, and whether each one cited your site the last time+ it ran.+ </p>+ <p className="m-0 text-muted max-w-3xl">+ You can view tracked prompts and turn them on or off here. To add prompts or change their text, use the+ API (<code>POST /api/projects/:project_id/aeo_prompts/bulk_import/</code>) or the PostHog MCP tools.+ </p>+ <div className="flex flex-wrap items-center gap-2">+ <LemonSelect+ size="small"+ aria-label="Topic"+ value={promptsTopic}+ onChange={setPromptsTopic}+ options={[+ { value: null, label: 'All topics' },+ ...promptTopics.map(({ topic, prompts: topicCount }) => ({+ value: topic,+ label: `${topic || 'No topic'} (${topicCount})`,+ })),+ ]}+ data-attr="content-autopilot-prompts-topic-filter"+ />+ </div>++ {promptsError === 'failed' ? (+ <LemonBanner+ type="error"+ action={{ children: 'Try again', onClick: loadAeoPrompts, loading: promptsLoading }}+ >+ Couldn't load prompts. Try again, and if it keeps happening contact support.+ </LemonBanner>+ ) : null}++ {prompts === null ? (+ promptsError ? null : (+ <LemonSkeleton className="h-40 w-full" />+ )+ ) : count === 0 && promptsTopic === null ? (+ <LemonCard hoverEffect={false} className="p-6 text-center">+ <h3 className="m-0">No prompts yet</h3>+ <p className="m-0 mt-2 text-muted max-w-xl mx-auto">+ Add prompts with the API (<code>POST /api/projects/:project_id/aeo_prompts/bulk_import/</code>)+ or the PostHog MCP tools. They're checked against AI assistants on the next daily run.+ </p>+ </LemonCard>+ ) : (+ <LemonTable+ dataSource={prompts.results}+ rowKey="id"+ loading={promptsLoading}+ emptyState="No prompts have this topic"+ pagination={{+ controlled: true,+ pageSize: PROMPTS_PAGE_SIZE,+ currentPage: promptsPage,+ entryCount: count,+ onForward:+ promptsPage * PROMPTS_PAGE_SIZE < count ? () => setPromptsPage(promptsPage + 1) : undefined,+ onBackward: promptsPage > 1 ? () => setPromptsPage(promptsPage - 1) : undefined,+ }}+ columns={[+ {+ title: 'Prompt',+ key: 'prompt',+ render: (_, prompt) => <div className="py-2 min-w-60 break-words">{prompt.prompt}</div>,+ },+ {+ title: 'Topic',+ key: 'topic',+ render: (_, { topic }) =>+ topic ? (+ <span className="whitespace-nowrap">{topic}</span>+ ) : (+ <span className="text-muted whitespace-nowrap">No topic</span>+ ),+ },+ ...promptEngines.map((engine) => ({+ title: engineLabel(engine),+ key: `engine-${engine}`,+ render: (_: unknown, prompt: AEOPromptApi) => checkTag(prompt, engine),+ })),+ {+ title: 'Last checked',+ key: 'last_checked_at',+ render: (_, { last_checked_at }) =>+ last_checked_at ? (+ <TZLabel time={last_checked_at} className="whitespace-nowrap" />+ ) : (+ <span className="text-muted whitespace-nowrap">Never</span>+ ),+ },+ {+ title: 'Active',+ key: 'active',+ render: (_, { id, active }) => (+ <LemonSwitch+ checked={active}+ onChange={(checked) => setPromptActive(id, checked)}+ loading={!!promptActiveUpdating[id]}+ aria-label={active ? 'Turn prompt off' : 'Turn prompt on'}+ data-attr="content-autopilot-prompt-active-toggle"+ />+ ),+ },+ ]}+ />+ )}+ </section>+ )+}diff --git a/products/web_analytics/frontend/contentAutopilot/contentAutopilotLogic.ts b/products/web_analytics/frontend/contentAutopilot/contentAutopilotLogic.tsindex 61a286c160f5..caa168b191e4 100644--- a/products/web_analytics/frontend/contentAutopilot/contentAutopilotLogic.ts+++ b/products/web_analytics/frontend/contentAutopilot/contentAutopilotLogic.ts@@ -24,7 +24,7 @@ import { ContentAutopilotPageTarget, opportunitiesNeedRefresh, pageToUpdate, proexport type ContentAutopilotOnboardingStep = 'site' | 'sources'export type ContentAutopilotProposalTab = 'preview' | 'draft' | 'changes' | 'brief' | 'sources'-export type ContentAutopilotWorkspaceTab = 'opportunities' | 'drafts'+export type ContentAutopilotWorkspaceTab = 'opportunities' | 'drafts' | 'prompts'export type ContentAutopilotWorkspaceResource = 'profiles' | 'runs' | 'proposals'export type ContentAutopilotProfileResource = 'runs' | 'proposals'export type ContentAutopilotWorkspaceErrors = Partial<Record<ContentAutopilotWorkspaceResource, string>>diff --git a/products/web_analytics/frontend/contentAutopilot/contentAutopilotPromptsLogic.test.ts b/products/web_analytics/frontend/contentAutopilot/contentAutopilotPromptsLogic.test.tsnew file mode 100644index 000000000000..df7a6ce93d9f--- /dev/null+++ b/products/web_analytics/frontend/contentAutopilot/contentAutopilotPromptsLogic.test.ts@@ -0,0 +1,107 @@+import { MOCK_DEFAULT_TEAM } from 'lib/api.mock'++import { expectLogic } from 'kea-test-utils'++import { ApiError } from 'lib/api-error'+import { lemonToast } from 'lib/lemon-ui/LemonToast/LemonToast'++import { resumeKeaLoadersErrors, silenceKeaLoadersErrors } from '~/initKea'+import { initKeaTests } from '~/test/init'++import { aeoPromptsList, aeoPromptsPartialUpdate, aeoPromptsTopics } from '../generated/api'+import type { AEOPromptApi } from '../generated/api.schemas'+import { PROMPTS_PAGE_SIZE, contentAutopilotPromptsLogic } from './contentAutopilotPromptsLogic'+import { EXAMPLE_PROMPT_TOPICS, EXAMPLE_PROMPTS } from './contentAutopilotStoryFixtures'++jest.mock('../generated/api', () => ({+ aeoPromptsList: jest.fn(),+ aeoPromptsTopics: jest.fn(),+ aeoPromptsPartialUpdate: jest.fn(),+}))++jest.mock('lib/lemon-ui/LemonToast/LemonToast', () => ({ lemonToast: { error: jest.fn() } }))++const mockPromptsList = jest.mocked(aeoPromptsList)+const mockPromptsTopics = jest.mocked(aeoPromptsTopics)+const mockPromptsPartialUpdate = jest.mocked(aeoPromptsPartialUpdate)++describe('contentAutopilotPromptsLogic', () => {+ let logic: ReturnType<typeof contentAutopilotPromptsLogic.build>++ beforeEach(() => {+ jest.clearAllMocks()+ initKeaTests()+ mockPromptsList.mockResolvedValue({ count: 120, next: null, previous: null, results: EXAMPLE_PROMPTS })+ mockPromptsTopics.mockResolvedValue(EXAMPLE_PROMPT_TOPICS)+ })++ afterEach(() => {+ logic?.unmount()+ })++ it('resets to the first page and sends the topic when the topic filter changes', async () => {+ logic = contentAutopilotPromptsLogic()+ logic.mount()+ await expectLogic(logic).toFinishAllListeners()++ logic.actions.setPromptsPage(3)+ await expectLogic(logic).toFinishAllListeners()+ expect(mockPromptsList).toHaveBeenLastCalledWith(String(MOCK_DEFAULT_TEAM.id), {+ limit: PROMPTS_PAGE_SIZE,+ offset: 2 * PROMPTS_PAGE_SIZE,+ topic: undefined,+ })++ logic.actions.setPromptsTopic('')+ await expectLogic(logic).toFinishAllListeners().toMatchValues({ promptsPage: 1 })+ expect(mockPromptsList).toHaveBeenLastCalledWith(String(MOCK_DEFAULT_TEAM.id), {+ limit: PROMPTS_PAGE_SIZE,+ offset: 0,+ topic: '',+ })+ })++ it.each([+ ['forbidden', new ApiError('This feature is not available.', 403)],+ ['failed', new ApiError('Server error', 500)],+ ])('reports a %s error when the prompts fail to load', async (expected, error) => {+ mockPromptsList.mockRejectedValue(error)+ silenceKeaLoadersErrors()+ try {+ logic = contentAutopilotPromptsLogic()+ logic.mount()+ await expectLogic(logic).toFinishAllListeners().toMatchValues({ promptsError: expected, prompts: null })+ } finally {+ resumeKeaLoadersErrors()+ }+ })++ it.each([+ ['saves', true, false],+ ['rolls back', false, true],+ ])('turns a prompt off optimistically, then %s', async (_name, succeeds, expectedActive) => {+ const prompt = EXAMPLE_PROMPTS[0]+ let settle: () => void = () => {}+ mockPromptsPartialUpdate.mockReturnValue(+ new Promise<AEOPromptApi>((resolve, reject) => {+ settle = () => (succeeds ? resolve({ ...prompt, active: false }) : reject(new Error('Server error')))+ })+ )+ logic = contentAutopilotPromptsLogic()+ logic.mount()+ await expectLogic(logic).toFinishAllListeners()++ logic.actions.setPromptActive(prompt.id, false)+ expect(logic.values.prompts?.results[0].active).toBe(false)+ expect(logic.values.promptActiveUpdating).toEqual({ [prompt.id]: true })++ settle()+ await expectLogic(logic).toFinishAllListeners()+ expect(mockPromptsPartialUpdate).toHaveBeenCalledWith(String(MOCK_DEFAULT_TEAM.id), prompt.id, {+ active: false,+ })+ expect(logic.values.prompts?.results[0].active).toBe(expectedActive)+ expect(logic.values.promptActiveUpdating).toEqual({})+ expect(lemonToast.error).toHaveBeenCalledTimes(succeeds ? 0 : 1)+ })+})diff --git a/products/web_analytics/frontend/contentAutopilot/contentAutopilotPromptsLogic.ts b/products/web_analytics/frontend/contentAutopilot/contentAutopilotPromptsLogic.tsnew file mode 100644index 000000000000..033d80668e19--- /dev/null+++ b/products/web_analytics/frontend/contentAutopilot/contentAutopilotPromptsLogic.ts@@ -0,0 +1,241 @@+import { MakeLogicType, actions, afterMount, connect, kea, listeners, path, reducers, selectors } from 'kea'+import { loaders } from 'kea-loaders'+import posthog from 'posthog-js'++import { ApiError } from 'lib/api-error'+import { lemonToast } from 'lib/lemon-ui/LemonToast/LemonToast'+import { teamLogic } from 'scenes/teamLogic'++import {+ aeoPromptsList,+ aeoPromptsPartialUpdate,+ aeoPromptsTopics,+} from 'products/web_analytics/frontend/generated/api'+import type {+ AEOPromptApi,+ AEOPromptTopicApi,+ PaginatedAEOPromptListApi,+} from 'products/web_analytics/frontend/generated/api.schemas'++import { sortEngines } from './utils'++export const PROMPTS_PAGE_SIZE = 50++export type ContentAutopilotPromptsError = 'forbidden' | 'failed'++const promptsErrorFrom = (error: unknown): ContentAutopilotPromptsError =>+ error instanceof ApiError && error.status === 403 ? 'forbidden' : 'failed'++const omitKey = (record: Record<string, boolean>, key: string): Record<string, boolean> => {+ const { [key]: _omitted, ...rest } = record+ return rest+}++const withPromptActive = (+ prompts: PaginatedAEOPromptListApi | null,+ promptId: string,+ active: boolean+): PaginatedAEOPromptListApi | null =>+ prompts && {+ ...prompts,+ results: prompts.results.map((prompt) => (prompt.id === promptId ? { ...prompt, active } : prompt)),+ }++// Generated by kea-typegen. Update if you're an agent, ignore if you're human.+export interface contentAutopilotPromptsLogicValues {+ currentTeamIdStrict: number | string // teamLogic+ promptActiveUpdating: Record<string, boolean>+ promptEngines: string[]+ promptTopics: AEOPromptTopicApi[]+ promptTopicsLoading: boolean+ prompts: PaginatedAEOPromptListApi | null+ promptsError: ContentAutopilotPromptsError | null+ promptsLoading: boolean+ promptsPage: number+ promptsTopic: string | null+}++// Generated by kea-typegen. Update if you're an agent, ignore if you're human.+export interface contentAutopilotPromptsLogicActions {+ loadAeoPromptTopics: () => any+ loadAeoPromptTopicsFailure: (+ error: string,+ errorObject?: any+ ) => {+ error: string+ errorObject?: any+ }+ loadAeoPromptTopicsSuccess: (+ promptTopics: AEOPromptTopicApi[],+ payload?: any+ ) => {+ promptTopics: AEOPromptTopicApi[]+ payload?: any+ }+ loadAeoPrompts: () => {+ value: true+ }+ loadAeoPromptsFailure: (+ error: string,+ errorObject?: any+ ) => {+ error: string+ errorObject?: any+ }+ loadAeoPromptsSuccess: (+ prompts: PaginatedAEOPromptListApi,+ payload?: {+ value: true+ }+ ) => {+ prompts: PaginatedAEOPromptListApi+ payload?: {+ value: true+ }+ }+ setPromptActive: (+ promptId: string,+ active: boolean+ ) => {+ active: boolean+ promptId: string+ }+ setPromptActiveFailure: (+ promptId: string,+ active: boolean+ ) => {+ active: boolean+ promptId: string+ }+ setPromptActiveSuccess: (prompt: AEOPromptApi) => {+ prompt: AEOPromptApi+ }+ setPromptsPage: (page: number) => {+ page: number+ }+ setPromptsTopic: (topic: string | null) => {+ topic: string | null+ }+}++// Generated by kea-typegen. Update if you're an agent, ignore if you're human.+export interface contentAutopilotPromptsLogicMeta {+ __keaTypeGenInternalSelectorTypes: {+ promptEngines: (prompts: PaginatedAEOPromptListApi | null) => string[]+ }+}++export type contentAutopilotPromptsLogicType = MakeLogicType<+ contentAutopilotPromptsLogicValues,+ contentAutopilotPromptsLogicActions,+ Record<string, any>,+ contentAutopilotPromptsLogicMeta+>++export const contentAutopilotPromptsLogic = kea<contentAutopilotPromptsLogicType>([+ path(['products', 'webAnalytics', 'contentAutopilot', 'contentAutopilotPromptsLogic']),+ connect(() => ({ values: [teamLogic, ['currentTeamIdStrict']] })),+ actions({+ loadAeoPrompts: true,+ setPromptsPage: (page: number) => ({ page }),+ setPromptsTopic: (topic: string | null) => ({ topic }),+ setPromptActive: (promptId: string, active: boolean) => ({ promptId, active }),+ setPromptActiveSuccess: (prompt: AEOPromptApi) => ({ prompt }),+ setPromptActiveFailure: (promptId: string, active: boolean) => ({ promptId, active }),+ }),+ loaders(({ values }) => ({+ prompts: [+ null as PaginatedAEOPromptListApi | null,+ {+ loadAeoPrompts: async (_, breakpoint) => {+ const response = await aeoPromptsList(String(values.currentTeamIdStrict), {+ limit: PROMPTS_PAGE_SIZE,+ offset: (values.promptsPage - 1) * PROMPTS_PAGE_SIZE,+ topic: values.promptsTopic ?? undefined,+ })+ breakpoint()+ return response+ },+ },+ ],+ promptTopics: [+ [] as AEOPromptTopicApi[],+ {+ loadAeoPromptTopics: async () => aeoPromptsTopics(String(values.currentTeamIdStrict)),+ },+ ],+ })),+ reducers({+ prompts: {+ setPromptActive: (state, { promptId, active }) => withPromptActive(state, promptId, active),+ setPromptActiveSuccess: (state, { prompt }) => withPromptActive(state, prompt.id, prompt.active),+ setPromptActiveFailure: (state, { promptId, active }) => withPromptActive(state, promptId, !active),+ },+ promptActiveUpdating: [+ {} as Record<string, boolean>,+ {+ setPromptActive: (state, { promptId }) => ({ ...state, [promptId]: true }),+ setPromptActiveSuccess: (state, { prompt }) => omitKey(state, prompt.id),+ setPromptActiveFailure: (state, { promptId }) => omitKey(state, promptId),+ },+ ],+ promptsPage: [+ 1,+ {+ setPromptsPage: (_, { page }) => page,+ setPromptsTopic: () => 1,+ },+ ],+ promptsTopic: [+ null as string | null,+ {+ setPromptsTopic: (_, { topic }) => topic,+ },+ ],+ promptsError: [+ null as ContentAutopilotPromptsError | null,+ {+ loadAeoPrompts: () => null,+ loadAeoPromptsFailure: (_, { errorObject }) => promptsErrorFrom(errorObject),+ },+ ],+ }),+ selectors({+ promptEngines: [+ (s) => [s.prompts],+ (prompts: PaginatedAEOPromptListApi | null): string[] =>+ sortEngines(+ (prompts?.results ?? []).flatMap((prompt) => prompt.latest_checks.map((check) => check.engine))+ ),+ ],+ }),+ listeners(({ actions, values }) => ({+ setPromptActive: async ({ promptId, active }) => {+ try {+ const prompt = await aeoPromptsPartialUpdate(String(values.currentTeamIdStrict), promptId, { active })+ actions.setPromptActiveSuccess(prompt)+ // pinned: analytics event name, renaming it breaks dashboards+ posthog.capture('web analytics content autopilot prompt toggled', { active })+ } catch {+ actions.setPromptActiveFailure(promptId, active)+ lemonToast.error(+ active ? "Couldn't turn the prompt on. Try again." : "Couldn't turn the prompt off. Try again."+ )+ }+ },+ setPromptsPage: () => actions.loadAeoPrompts(),+ setPromptsTopic: ({ topic }) => {+ // pinned: analytics event name, renaming it breaks dashboards+ posthog.capture('web analytics content autopilot prompts topic changed', {+ filtered: topic !== null,+ })+ actions.loadAeoPrompts()+ },+ })),+ afterMount(({ actions }) => {+ // pinned: analytics event name, renaming it breaks dashboards+ posthog.capture('web analytics content autopilot prompts viewed')+ actions.loadAeoPrompts()+ actions.loadAeoPromptTopics()+ }),+])diff --git a/products/web_analytics/frontend/contentAutopilot/contentAutopilotStoryFixtures.ts b/products/web_analytics/frontend/contentAutopilot/contentAutopilotStoryFixtures.tsindex 39df2dc83f73..0d1ad60c40e8 100644--- a/products/web_analytics/frontend/contentAutopilot/contentAutopilotStoryFixtures.ts+++ b/products/web_analytics/frontend/contentAutopilot/contentAutopilotStoryFixtures.ts@@ -1,4 +1,6 @@import type {+ AEOPromptApi,+ AEOPromptTopicApi,ContentAutopilotOpportunityApi,ContentAutopilotProposalApi,ContentAutopilotProposalListApi,@@ -231,3 +233,61 @@ export const EXAMPLE_FAILED_DRAFT_OPPORTUNITY: ContentAutopilotOpportunityApi =title: 'How do I track AI crawler traffic on my website?',status: 'drafted',}++const EXAMPLE_PROMPT: AEOPromptApi = {+ id: '00000000-0000-4000-8000-000000000301',+ prompt: 'What is the best open source product analytics tool for a small startup?',+ topic: 'Product analytics',+ prompt_source: 'imported',+ rank: 0,+ active: true,+ last_checked_at: '2026-09-29T09:00:00Z',+ external_visibility: 40,+ created_at: '2026-09-01T12:00:00Z',+ updated_at: '2026-09-01T12:00:00Z',+ latest_checks: [+ { engine: 'claude-web-search', cited: true, check_failed: false, checked_at: '2026-09-29T09:00:00Z' },+ { engine: 'openai-web-search', cited: false, check_failed: false, checked_at: '2026-09-29T09:00:00Z' },+ { engine: 'exa-answer', cited: true, check_failed: false, checked_at: '2026-09-29T09:00:00Z' },+ ],+}++export const EXAMPLE_PROMPTS: AEOPromptApi[] = [+ EXAMPLE_PROMPT,+ {+ ...EXAMPLE_PROMPT,+ id: '00000000-0000-4000-8000-000000000302',+ prompt: 'How do I track funnels without writing SQL, and can I break the results down by country and browser?',+ latest_checks: [+ { engine: 'claude-web-search', cited: false, check_failed: false, checked_at: '2026-09-28T09:00:00Z' },+ { engine: 'openai-web-search', cited: false, check_failed: true, checked_at: '2026-09-28T09:00:00Z' },+ { engine: 'gemini-google-search', cited: true, check_failed: false, checked_at: '2026-09-28T09:00:00Z' },+ ],+ last_checked_at: '2026-09-28T09:00:00Z',+ },+ {+ ...EXAMPLE_PROMPT,+ id: '00000000-0000-4000-8000-000000000303',+ prompt: 'Which session replay tools let me mask sensitive inputs by default?',+ topic: 'Session replay',+ active: false,+ latest_checks: [],+ last_checked_at: null,+ },+ {+ ...EXAMPLE_PROMPT,+ id: '00000000-0000-4000-8000-000000000304',+ prompt: 'Is there a free alternative to paid feature flag services?',+ topic: '',+ latest_checks: [+ { engine: 'exa-answer', cited: false, check_failed: false, checked_at: '2026-09-27T09:00:00Z' },+ ],+ last_checked_at: '2026-09-27T09:00:00Z',+ },+]++export const EXAMPLE_PROMPT_TOPICS: AEOPromptTopicApi[] = [+ { topic: '', prompts: 1 },+ { topic: 'Product analytics', prompts: 2 },+ { topic: 'Session replay', prompts: 1 },+]diff --git a/products/web_analytics/frontend/contentAutopilot/utils.ts b/products/web_analytics/frontend/contentAutopilot/utils.tsindex 84e5e39d224f..cf82839a1289 100644--- a/products/web_analytics/frontend/contentAutopilot/utils.ts+++ b/products/web_analytics/frontend/contentAutopilot/utils.ts@@ -7,6 +7,22 @@ import type {const OPPORTUNITIES_STALE_AFTER_MS = 60 * 60 * 1000+const ENGINE_LABELS: Record<string, string> = {+ 'claude-web-search': 'Claude',+ 'openai-web-search': 'ChatGPT',+ 'exa-answer': 'Exa',+ 'gemini-google-search': 'Gemini',+}++export const engineLabel = (engine: string): string => ENGINE_LABELS[engine] ?? engine++/** Known engines first, in a fixed order, so table columns don't move between pages. */+export const sortEngines = (engines: Iterable<string>): string[] => {+ const known = Object.keys(ENGINE_LABELS)+ const rank = (engine: string): number => (known.includes(engine) ? known.indexOf(engine) : known.length)+ return [...new Set(engines)].sort((a, b) => rank(a) - rank(b) || a.localeCompare(b))+}+export type ContentAutopilotPageTarget =| { isNewPage: true; path: string }| { isNewPage: false; path: string; url: string }diff --git a/products/web_analytics/frontend/generated/api.schemas.ts b/products/web_analytics/frontend/generated/api.schemas.tsindex 6b3736184a39..b2c6cb6adb5d 100644--- a/products/web_analytics/frontend/generated/api.schemas.ts+++ b/products/web_analytics/frontend/generated/api.schemas.ts@@ -7,7 +7,20 @@* PostHog API - generated* OpenAPI spec version: 1.0.0*/+export interface AEOPromptEngineCheckApi {+ /** Answer engine that ran the prompt, for example 'claude-web-search', 'openai-web-search' or 'exa-answer'. */+ engine: string+ /** Whether the answer cited a URL on one of the target domains. */+ cited: boolean+ /** The engine did not answer. When true, 'cited' is false and says nothing about citations. */+ check_failed: boolean+ /** When the check ran. */+ checked_at: string+}+export interface AEOPromptApi {+ /** The most recent check for each answer engine that ran the prompt. Empty when the prompt never ran. */+ latest_checks: AEOPromptEngineCheckApi[]/** Prompt id. */id: string/** The question asked to the answer engines, as a user would phrase it. */@@ -91,6 +104,13 @@ export interface AEOPromptImportResultApi {skipped: number}+export interface AEOPromptTopicApi {+ /** Topic name. Empty for prompts that have no topic. */+ topic: string+ /** How many prompts in the set have this topic. */+ prompts: number+}+export interface HeatmapCapturePageApi {/** A page URL that currently sends heatmap data. */url: string@@ -1726,6 +1746,11 @@ export interface GeneratePathCleaningSuggestionResponseApi {}export type AeoPromptsListParams = {+ /**+ * Only list prompts that are on (true) or off (false). Omit to list both.+ * @nullable+ */+ active?: boolean | null/*** Number of results to return per page.*/@@ -1734,6 +1759,11 @@ export type AeoPromptsListParams = {* The initial index from which to return the results.*/offset?: number+ /**+ * Only list prompts with this topic. An empty value lists the prompts that have no topic.+ * @maxLength 200+ */+ topic?: string}export type HeatmapScreenshotsContentRetrieveParams = {diff --git a/products/web_analytics/frontend/generated/api.ts b/products/web_analytics/frontend/generated/api.tsindex 61b8a913af43..d8ac45d5688f 100644--- a/products/web_analytics/frontend/generated/api.ts+++ b/products/web_analytics/frontend/generated/api.ts@@ -12,6 +12,7 @@ import type {AEOPromptApi,AEOPromptBulkImportRequestApi,AEOPromptImportResultApi,+ AEOPromptTopicApi,AchievementsListResponseApi,AcknowledgeCelebrationRequestApi,AcknowledgeCelebrationResponseApi,@@ -115,7 +116,7 @@ export const getAeoPromptsListUrl = (projectId: string, params?: AeoPromptsListP}/**- * Lists the project's AEO prompt set, grouped by topic, one page at a time.+ * Lists the project's AEO prompt set, ordered by topic and then rank, one page at a time. Filter by topic or by on/off state. Each prompt carries its latest check per answer engine.* @summary List AEO prompts*/export const aeoPromptsList = async (@@ -134,6 +135,7 @@ export const getAeoPromptsRetrieveUrl = (projectId: string, id: string) => {}/**+ * Gets one prompt from the project's AEO prompt set, with its latest check per answer engine.* @summary Get an AEO prompt*/export const aeoPromptsRetrieve = async (@@ -152,6 +154,7 @@ export const getAeoPromptsPartialUpdateUrl = (projectId: string, id: string) =>}/**+ * Turns a prompt on or off. Only active prompts run against the answer engines. Past checks are kept either way.* @summary Turn an AEO prompt on or off*/export const aeoPromptsPartialUpdate = async (@@ -189,6 +192,21 @@ export const aeoPromptsBulkImport = async (})}+export const getAeoPromptsTopicsUrl = (projectId: string) => {+ return `/api/projects/${projectId}/aeo_prompts/topics/`+}++/**+ * Lists the topics in the project's AEO prompt set, with the number of prompts in each.+ * @summary List AEO prompt topics+ */+export const aeoPromptsTopics = async (projectId: string, options?: RequestInit): Promise<AEOPromptTopicApi[]> => {+ return apiMutator<AEOPromptTopicApi[]>(getAeoPromptsTopicsUrl(projectId), {+ ...options,+ method: 'GET',+ })+}+export const getHeatmapCapturePagesRetrieveUrl = (projectId: string) => {return `/api/projects/${projectId}/heatmap_capture/pages/`}diff --git a/products/web_analytics/frontend/generated/api.zod.ts b/products/web_analytics/frontend/generated/api.zod.tsindex 5354843cc06f..9bac51610ff5 100644--- a/products/web_analytics/frontend/generated/api.zod.ts+++ b/products/web_analytics/frontend/generated/api.zod.ts@@ -10,6 +10,7 @@import * as zod from 'zod'/**+ * Turns a prompt on or off. Only active prompts run against the answer engines. Past checks are kept either way.* @summary Turn an AEO prompt on or off*/export const AeoPromptsPartialUpdateBody = /* @__PURE__ */ zod.object({diff --git a/services/mcp/schema/generated-tool-definitions.json b/services/mcp/schema/generated-tool-definitions.jsonindex 13728c0ef610..88289ef2aed4 100644--- a/services/mcp/schema/generated-tool-definitions.json+++ b/services/mcp/schema/generated-tool-definitions.json@@ -711,6 +711,86 @@},"feature_entitlement": "audit_logs"},+ "aeo-prompts-get": {+ "description": "Get one AI citation tracking prompt by id, with its topic, on/off state, when it last ran, and `latest_checks`, the newest result per answer engine. Get the id from `aeo-prompts-list`.",+ "category": "AI citation tracking",+ "feature": "aeo",+ "summary": "Get a tracked AI prompt",+ "title": "Get a tracked AI prompt",+ "required_scopes": ["web_analytics:read"],+ "api_operations": ["GET /api/projects/{project_id}/aeo_prompts/{id}/"],+ "annotations": {+ "destructiveHint": false,+ "idempotentHint": true,+ "openWorldHint": true,+ "readOnlyHint": true+ },+ "feature_flag": "aeo-citation-tracking"+ },+ "aeo-prompts-import": {+ "description": "Add prompts to the project's AI citation tracking set, up to 2000 per call. Each item has the `prompt` text, an optional `topic` and an optional `external_visibility` (0-100). Prompts are matched to existing ones by normalized text (case and whitespace ignored), so importing the same set again does not create duplicates: a match updates the topic and visibility and keeps the prompt's on/off state. Empty prompts, and prompts too long for the answer engines, are skipped. Returns the `created`, `updated` and `skipped` counts. New prompts run in the next daily check. Write prompts the way a person would ask an AI assistant, and ask the user to confirm the list before you import it.",+ "category": "AI citation tracking",+ "feature": "aeo",+ "summary": "Import AI prompts to track",+ "title": "Import AI prompts to track",+ "required_scopes": ["web_analytics:write"],+ "api_operations": ["POST /api/projects/{project_id}/aeo_prompts/bulk_import/"],+ "annotations": {+ "destructiveHint": false,+ "idempotentHint": true,+ "openWorldHint": true,+ "readOnlyHint": false+ },+ "feature_flag": "aeo-citation-tracking"+ },+ "aeo-prompts-list": {+ "description": "List the project's AI citation tracking prompts: the questions PostHog asks AI answer engines every day to check whether their answers cite the project's site. Each prompt has its text, topic, on/off state (`active`), when it last ran (`last_checked_at`) and `latest_checks`, the newest result per engine (`cited`, `check_failed`, `checked_at`). Filter by `topic` (use `aeo-prompts-topics-list` for the valid values; an empty string selects prompts with no topic) and by `active`. Results are paginated with `limit`/`offset` and ordered by topic, then rank. Use it to find prompts that are not cited, to audit the prompt set, or to find a prompt id before calling `aeo-prompts-update`.",+ "category": "AI citation tracking",+ "feature": "aeo",+ "summary": "List tracked AI prompts",+ "title": "List tracked AI prompts",+ "required_scopes": ["web_analytics:read"],+ "api_operations": ["GET /api/projects/{project_id}/aeo_prompts/"],+ "annotations": {+ "destructiveHint": false,+ "idempotentHint": true,+ "openWorldHint": true,+ "readOnlyHint": true+ },+ "feature_flag": "aeo-citation-tracking"+ },+ "aeo-prompts-topics-list": {+ "description": "List the topics in the project's AI citation tracking prompt set, with the number of prompts in each. An empty topic groups the prompts that have no topic. Use the values as the `topic` filter of `aeo-prompts-list`.",+ "category": "AI citation tracking",+ "feature": "aeo",+ "summary": "List tracked AI prompt topics",+ "title": "List tracked AI prompt topics",+ "required_scopes": ["web_analytics:read"],+ "api_operations": ["GET /api/projects/{project_id}/aeo_prompts/topics/"],+ "annotations": {+ "destructiveHint": false,+ "idempotentHint": true,+ "openWorldHint": true,+ "readOnlyHint": true+ },+ "feature_flag": "aeo-citation-tracking"+ },+ "aeo-prompts-update": {+ "description": "Turn one AI citation tracking prompt on (`active: true`) or off (`active: false`). Only active prompts run in the daily check. Turning a prompt off keeps its past results, and turning it on again resumes the checks. This is the only field you can change; the prompt text and topic change only through `aeo-prompts-import`. Returns the updated prompt.",+ "category": "AI citation tracking",+ "feature": "aeo",+ "summary": "Turn a tracked AI prompt on or off",+ "title": "Turn a tracked AI prompt on or off",+ "required_scopes": ["web_analytics:write"],+ "api_operations": ["PATCH /api/projects/{project_id}/aeo_prompts/{id}/"],+ "annotations": {+ "destructiveHint": false,+ "idempotentHint": true,+ "openWorldHint": true,+ "readOnlyHint": false+ },+ "feature_flag": "aeo-citation-tracking"+ },"alert-create": {"description": "Create a new alert on an insight. Alerts can use either threshold-based conditions or anomaly detection. For threshold alerts: set condition (absolute_value, relative_increase, relative_decrease) and threshold configuration with bounds — at least one of lower or upper is required (omit detector_config). For anomaly detection: set detector_config with a detector type (zscore, mad, iqr, threshold, copod, ecod, hbos, isolation_forest, knn, lof, ocsvm, pca) and parameters like threshold (sensitivity 0-1, default 0.9) and window size. Ensemble detectors combine 2+ sub-detectors with AND/OR logic. The llm detector type asks an AI model whether the latest point is anomalous, given its history, and fires when the model says yes. Its only field is optional instructions (plain text describing what counts as unusual, up to 2000 characters). The history it reads follows the insight interval, and it also sees the points that its earlier checks fired on, so instructions can refer to earlier alerts. It cannot be an ensemble member, cannot use preprocessing, cannot run on the real_time cadence, is not available on breakdown insights, is capped per project, and is only available where the AI detector is enabled for the organization. Always send detector_config.type explicitly: a config with no type is read as a z-score detector, and any llm-only field you sent with it is dropped without an error. Requires an insight ID and at least one subscribed user.\nThe config type must match the insight kind. Trends insights: TrendsAlertConfig (series_index, check_ongoing_interval). SQL/HogQL insights: HogQLAlertConfig — column picks which result column to evaluate (defaults to the single numeric column), evaluation is required and selects how rows are read: 'last_row' (query ordered oldest->newest, the last row is the current value), 'first_row' (query ordered newest->oldest, the first row is the current value — pair with a LIMIT, and it is unaffected by result truncation), or 'any_row' (every row is checked and the alert fires if any value breaches; absolute_value condition only). label_column names the evaluated row(s) in breach messages in every evaluation mode (defaults to the first non-evaluated column). Label values appear in delivered notifications — avoid label columns containing PII. Funnel insights: FunnelsAlertConfig — funnel_step picks the step to monitor (null for the overall last step), metric is 'conversion_from_start' or 'conversion_from_previous', and check_ongoing_interval (historical-trend funnels: also evaluate the current in-progress period). Steps funnels support only absolute_value conditions; historical-trend funnels also support relative_increase/relative_decrease. Anomaly detection (detector_config) is supported for trends and SQL insights (not funnels).\ncalculation_interval sets how often the alert runs: real_time, every_15_minutes, hourly, daily, weekly, or monthly. real_time needs a Scale or Enterprise plan. every_15_minutes needs a Boost, Scale, or Enterprise add-on. The field is optional and defaults to daily. If the user does not specify a cadence, ask before creating the alert.\nTo set the check time, pass schedule_start_time with a local HH:MM time. Updates take effect after the current next_check_at, so they cannot increase the alert check frequency.\nNote: subscribed_users only controls email recipients. To also post the alert to Slack, create it and then call alert-destinations-create with the workspace and channel. For HTTPS webhook or Discord delivery, see the recipe on cdp-functions-create — it covers integration lookup (integrations-channels-retrieve), dedupe (cdp-functions-list filtered by alert id, limit=1000), and the exact filters/inputs shape to pass.","category": "Alerts",diff --git a/services/mcp/schema/tool-definitions-all.json b/services/mcp/schema/tool-definitions-all.jsonindex a5a48c048428..e5d3278a7f9f 100644--- a/services/mcp/schema/tool-definitions-all.json+++ b/services/mcp/schema/tool-definitions-all.json@@ -711,6 +711,86 @@},"feature_entitlement": "audit_logs"},+ "aeo-prompts-get": {+ "description": "Get one AI citation tracking prompt by id, with its topic, on/off state, when it last ran, and `latest_checks`, the newest result per answer engine. Get the id from `aeo-prompts-list`.",+ "category": "AI citation tracking",+ "feature": "aeo",+ "summary": "Get a tracked AI prompt",+ "title": "Get a tracked AI prompt",+ "required_scopes": ["web_analytics:read"],+ "api_operations": ["GET /api/projects/{project_id}/aeo_prompts/{id}/"],+ "annotations": {+ "destructiveHint": false,+ "idempotentHint": true,+ "openWorldHint": true,+ "readOnlyHint": true+ },+ "feature_flag": "aeo-citation-tracking"+ },+ "aeo-prompts-import": {+ "description": "Add prompts to the project's AI citation tracking set, up to 2000 per call. Each item has the `prompt` text, an optional `topic` and an optional `external_visibility` (0-100). Prompts are matched to existing ones by normalized text (case and whitespace ignored), so importing the same set again does not create duplicates: a match updates the topic and visibility and keeps the prompt's on/off state. Empty prompts, and prompts too long for the answer engines, are skipped. Returns the `created`, `updated` and `skipped` counts. New prompts run in the next daily check. Write prompts the way a person would ask an AI assistant, and ask the user to confirm the list before you import it.",+ "category": "AI citation tracking",+ "feature": "aeo",+ "summary": "Import AI prompts to track",+ "title": "Import AI prompts to track",+ "required_scopes": ["web_analytics:write"],+ "api_operations": ["POST /api/projects/{project_id}/aeo_prompts/bulk_import/"],+ "annotations": {+ "destructiveHint": false,+ "idempotentHint": true,+ "openWorldHint": true,+ "readOnlyHint": false+ },+ "feature_flag": "aeo-citation-tracking"+ },+ "aeo-prompts-list": {+ "description": "List the project's AI citation tracking prompts: the questions PostHog asks AI answer engines every day to check whether their answers cite the project's site. Each prompt has its text, topic, on/off state (`active`), when it last ran (`last_checked_at`) and `latest_checks`, the newest result per engine (`cited`, `check_failed`, `checked_at`). Filter by `topic` (use `aeo-prompts-topics-list` for the valid values; an empty string selects prompts with no topic) and by `active`. Results are paginated with `limit`/`offset` and ordered by topic, then rank. Use it to find prompts that are not cited, to audit the prompt set, or to find a prompt id before calling `aeo-prompts-update`.",+ "category": "AI citation tracking",+ "feature": "aeo",+ "summary": "List tracked AI prompts",+ "title": "List tracked AI prompts",+ "required_scopes": ["web_analytics:read"],+ "api_operations": ["GET /api/projects/{project_id}/aeo_prompts/"],+ "annotations": {+ "destructiveHint": false,+ "idempotentHint": true,+ "openWorldHint": true,+ "readOnlyHint": true+ },+ "feature_flag": "aeo-citation-tracking"+ },+ "aeo-prompts-topics-list": {+ "description": "List the topics in the project's AI citation tracking prompt set, with the number of prompts in each. An empty topic groups the prompts that have no topic. Use the values as the `topic` filter of `aeo-prompts-list`.",+ "category": "AI citation tracking",+ "feature": "aeo",+ "summary": "List tracked AI prompt topics",+ "title": "List tracked AI prompt topics",+ "required_scopes": ["web_analytics:read"],+ "api_operations": ["GET /api/projects/{project_id}/aeo_prompts/topics/"],+ "annotations": {+ "destructiveHint": false,+ "idempotentHint": true,+ "openWorldHint": true,+ "readOnlyHint": true+ },+ "feature_flag": "aeo-citation-tracking"+ },+ "aeo-prompts-update": {+ "description": "Turn one AI citation tracking prompt on (`active: true`) or off (`active: false`). Only active prompts run in the daily check. Turning a prompt off keeps its past results, and turning it on again resumes the checks. This is the only field you can change; the prompt text and topic change only through `aeo-prompts-import`. Returns the updated prompt.",+ "category": "AI citation tracking",+ "feature": "aeo",+ "summary": "Turn a tracked AI prompt on or off",+ "title": "Turn a tracked AI prompt on or off",+ "required_scopes": ["web_analytics:write"],+ "api_operations": ["PATCH /api/projects/{project_id}/aeo_prompts/{id}/"],+ "annotations": {+ "destructiveHint": false,+ "idempotentHint": true,+ "openWorldHint": true,+ "readOnlyHint": false+ },+ "feature_flag": "aeo-citation-tracking"+ },"agent-feedback": {"description": "Send feedback about anything PostHog to the PostHog team. Set `feedback_type` to route it: `product` (any PostHog product or feature — insights, session replay, feature flags, the data warehouse, web analytics, error tracking, etc.; put the area in `product_area`), `mcp` (this MCP server itself — a tool, input schema, response format, error, or these instructions; set `category`), `docs`, `scout` (reserved for scheduled scout runs reporting an improvement opportunity in the canonical PostHog scout skill steering the run; set `scout_skill_name`, `scout_skill_version`, and `scout_category`, and generalize — describe the pattern, never the project's data), or `other`. ALL sentiments are welcome via `sentiment` (`positive`/`neutral`/`negative`/`mixed`) — praise and feature requests are useful signal, not just problems. Good triggers: a confusing or broken product experience, a papercut that slowed the task down, a missing capability you worked around, a feature request, an unhelpful error, or something that worked really well. Keep `summary` to one sentence and write the free-text fields as clear, concise bullet points, quoting the product surface, tool name, parameter, or error text where possible. Include a concrete `suggested_improvement` whenever you can name one. Do not include user PII or sensitive query content. The user can also ask you to send feedback directly (e.g. 'make a PostHog feedback for this'). IMPORTANT: submitting feedback does NOT mean your work is done — keep going and finish the user's task using the other available tools. This tool is a side channel to the PostHog team, not a way to end the conversation.","category": "Feedback",diff --git a/services/mcp/src/api/generated.ts b/services/mcp/src/api/generated.tsindex 3f868bb8472e..a4b03e648a39 100644--- a/services/mcp/src/api/generated.ts+++ b/services/mcp/src/api/generated.ts@@ -13,7 +13,20 @@ export namespace Schemas {/****/+ export interface AEOPromptEngineCheck {+ /** Answer engine that ran the prompt, for example 'claude-web-search', 'openai-web-search' or 'exa-answer'. */+ engine: string;+ /** Whether the answer cited a URL on one of the target domains. */+ cited: boolean;+ /** The engine did not answer. When true, 'cited' is false and says nothing about citations. */+ check_failed: boolean;+ /** When the check ran. */+ checked_at: string;+ }+export interface AEOPrompt {+ /** The most recent check for each answer engine that ran the prompt. Empty when the prompt never ran. */+ latest_checks: AEOPromptEngineCheck[];/** Prompt id. */id: string;/** The question asked to the answer engines, as a user would phrase it. */@@ -83,6 +96,13 @@ export namespace Schemas {skipped: number;}+ export interface AEOPromptTopic {+ /** Topic name. Empty for prompts that have no topic. */+ topic: string;+ /** How many prompts in the set have this topic. */+ prompts: number;+ }+export interface AIContextAccountProperty {/** Customer analytics account property definition id. */id: string;@@ -125492,6 +125512,11 @@ export namespace Schemas {} as const;export type AeoPromptsListParams = {+ /**+ * Only list prompts that are on (true) or off (false). Omit to list both.+ * @nullable+ */+ active?: boolean | null;/*** Number of results to return per page.*/@@ -125500,6 +125525,11 @@ export namespace Schemas {* The initial index from which to return the results.*/offset?: number;+ /**+ * Only list prompts with this topic. An empty value lists the prompts that have no topic.+ * @maxLength 200+ */+ topic?: string;};export type AiObservabilityInstrumentationChecklistRetrieveParams = {diff --git a/services/mcp/src/generated/aeo/api.ts b/services/mcp/src/generated/aeo/api.tsnew file mode 100644index 000000000000..eb5e7ebec876--- /dev/null+++ b/services/mcp/src/generated/aeo/api.ts@@ -0,0 +1,127 @@+/**+ * Auto-generated from the Django backend OpenAPI schema.+ * MCP service uses these Zod schemas for generated tool handlers.+ * To regenerate: hogli build:openapi+ *+ * PostHog API - MCP 5 enabled ops+ * OpenAPI spec version: 1.0.0+ */+import * as zod from 'zod'++/**+ * Lists the project's AEO prompt set, ordered by topic and then rank, one page at a time. Filter by topic or by on/off state. Each prompt carries its latest check per answer engine.+ * @summary List AEO prompts+ */+export const AeoPromptsListParams = () => zod.object({+ project_id: zod+ .string()+ .describe(+ "Project ID of the project you're trying to access. To find the ID of the project, make a call to \/api\/projects\/."+ ),+})++export const aeoPromptsListQueryTopicMax = 200++export const AeoPromptsListQueryParams = () => zod.object({+ active: zod.boolean().nullish().describe('Only list prompts that are on (true) or off (false). Omit to list both.'),+ limit: zod.number().optional().describe('Number of results to return per page.'),+ offset: zod.number().optional().describe('The initial index from which to return the results.'),+ topic: zod+ .string()+ .max(aeoPromptsListQueryTopicMax)+ .optional()+ .describe('Only list prompts with this topic. An empty value lists the prompts that have no topic.'),+})++/**+ * Gets one prompt from the project's AEO prompt set, with its latest check per answer engine.+ * @summary Get an AEO prompt+ */+export const AeoPromptsRetrieveParams = () => zod.object({+ id: zod.string(),+ project_id: zod+ .string()+ .describe(+ "Project ID of the project you're trying to access. To find the ID of the project, make a call to \/api\/projects\/."+ ),+})++/**+ * Turns a prompt on or off. Only active prompts run against the answer engines. Past checks are kept either way.+ * @summary Turn an AEO prompt on or off+ */+export const AeoPromptsPartialUpdateParams = () => zod.object({+ id: zod.string(),+ project_id: zod+ .string()+ .describe(+ "Project ID of the project you're trying to access. To find the ID of the project, make a call to \/api\/projects\/."+ ),+})++export const AeoPromptsPartialUpdateBody = () => zod.object({+ active: zod.boolean().optional().describe('Turn the prompt on or off for future runs.'),+})++/**+ * Adds prompts to the project's AEO prompt set, or updates the topic and visibility of prompts that already exist. Existing prompts keep their active state.+ * @summary Import AEO prompts+ */+export const AeoPromptsBulkImportParams = () => zod.object({+ project_id: zod+ .string()+ .describe(+ "Project ID of the project you're trying to access. To find the ID of the project, make a call to \/api\/projects\/."+ ),+})++export const aeoPromptsBulkImportBodyPromptsItemPromptMax = 10000++export const aeoPromptsBulkImportBodyPromptsItemTopicMax = 200++export const aeoPromptsBulkImportBodyPromptsItemExternalVisibilityMin = 0+export const aeoPromptsBulkImportBodyPromptsItemExternalVisibilityMax = 100++export const aeoPromptsBulkImportBodyPromptsMax = 2000++export const AeoPromptsBulkImportBody = () => zod.object({+ prompts: zod+ .array(+ zod.object({+ prompt: zod+ .string()+ .max(aeoPromptsBulkImportBodyPromptsItemPromptMax)+ .describe(+ 'The question to ask the answer engines, as a user would phrase it. Matching ignores case and whitespace, so a re-import updates the existing prompt.'+ ),+ topic: zod+ .string()+ .max(aeoPromptsBulkImportBodyPromptsItemTopicMax)+ .optional()+ .describe('Theme the prompt belongs to. The runner alternates between topics on each run.'),+ external_visibility: zod+ .number()+ .min(aeoPromptsBulkImportBodyPromptsItemExternalVisibilityMin)+ .max(aeoPromptsBulkImportBodyPromptsItemExternalVisibilityMax)+ .nullish()+ .describe(+ 'Share of answers (0-100) that mention the brand for this prompt, as reported by the tool the prompt set came from. Kept to benchmark against our own citation rate.'+ ),+ })+ )+ .min(1)+ .max(aeoPromptsBulkImportBodyPromptsMax)+ .describe('Prompts to add or update, at most 2000 per request.'),+})++/**+ * Lists the topics in the project's AEO prompt set, with the number of prompts in each.+ * @summary List AEO prompt topics+ */+export const AeoPromptsTopicsParams = () => zod.object({+ project_id: zod+ .string()+ .describe(+ "Project ID of the project you're trying to access. To find the ID of the project, make a call to \/api\/projects\/."+ ),+})diff --git a/services/mcp/src/lib/instructions.ts b/services/mcp/src/lib/instructions.tsindex aad3f1595836..4a22152b1a32 100644--- a/services/mcp/src/lib/instructions.ts+++ b/services/mcp/src/lib/instructions.ts@@ -225,6 +225,7 @@ export class ToolDomainExtractor {'estimate','fail','freeze',+ 'import','launch','migrate','move',diff --git a/services/mcp/src/tools/generated/aeo.ts b/services/mcp/src/tools/generated/aeo.tsnew file mode 100644index 000000000000..4d352483102e--- /dev/null+++ b/services/mcp/src/tools/generated/aeo.ts@@ -0,0 +1,128 @@+// AUTO-GENERATED from products/aeo/mcp/tools.yaml + OpenAPI — do not edit+import { z } from 'zod'++import type { Schemas } from '@/api/generated'+import * as orvalSchemas from '@/generated/aeo/api'+import { withPostHogUrl, type WithPostHogUrl } from '@/tools/tool-utils'+import type { Context, ToolBase, ZodObjectAny } from '@/tools/types'++const AeoPromptsGetSchema = () => {+ const AeoPromptsRetrieveParams = orvalSchemas.AeoPromptsRetrieveParams()+ return AeoPromptsRetrieveParams.omit({ project_id: true }).extend({+ id: AeoPromptsRetrieveParams.shape['id'].describe('The prompt id (a UUID), from `aeo-prompts-list`.'),+ })+}++const aeoPromptsGet = (): ToolBase<ReturnType<typeof AeoPromptsGetSchema>, Schemas.AEOPrompt> => ({+ name: 'aeo-prompts-get',+ schema: AeoPromptsGetSchema(),+ handler: async (context: Context, params: z.infer<ReturnType<typeof AeoPromptsGetSchema>>) => {+ const projectId = await context.stateManager.getProjectId()+ const result = await context.api.request<Schemas.AEOPrompt>({+ method: 'GET',+ path: `/api/projects/${encodeURIComponent(String(projectId))}/aeo_prompts/${encodeURIComponent(String(params.id))}/`,+ })+ return result+ },+})++const AeoPromptsImportSchema = () => {+ const AeoPromptsBulkImportBody = orvalSchemas.AeoPromptsBulkImportBody()+ return AeoPromptsBulkImportBody+}++const aeoPromptsImport = (): ToolBase<ReturnType<typeof AeoPromptsImportSchema>, Schemas.AEOPromptImportResult> => ({+ name: 'aeo-prompts-import',+ schema: AeoPromptsImportSchema(),+ handler: async (context: Context, params: z.infer<ReturnType<typeof AeoPromptsImportSchema>>) => {+ const projectId = await context.stateManager.getProjectId()+ const body: Record<string, unknown> = {}+ if (params.prompts !== undefined) {+ body['prompts'] = params.prompts+ }+ const result = await context.api.request<Schemas.AEOPromptImportResult>({+ method: 'POST',+ path: `/api/projects/${encodeURIComponent(String(projectId))}/aeo_prompts/bulk_import/`,+ body,+ })+ return result+ },+})++const AeoPromptsListSchema = () => {+ const AeoPromptsListQueryParams = orvalSchemas.AeoPromptsListQueryParams()+ return AeoPromptsListQueryParams+}++const aeoPromptsList = (): ToolBase<+ ReturnType<typeof AeoPromptsListSchema>,+ WithPostHogUrl<Schemas.PaginatedAEOPromptList>+> => ({+ name: 'aeo-prompts-list',+ schema: AeoPromptsListSchema(),+ handler: async (context: Context, params: z.infer<ReturnType<typeof AeoPromptsListSchema>>) => {+ const projectId = await context.stateManager.getProjectId()+ const result = await context.api.request<Schemas.PaginatedAEOPromptList>({+ method: 'GET',+ path: `/api/projects/${encodeURIComponent(String(projectId))}/aeo_prompts/`,+ query: {+ active: params.active,+ limit: params.limit,+ offset: params.offset,+ topic: params.topic,+ },+ })+ return await withPostHogUrl(context, result, '/web/content-autopilot')+ },+})++const AeoPromptsTopicsListSchema = () => z.object({})++const aeoPromptsTopicsList = (): ToolBase<ReturnType<typeof AeoPromptsTopicsListSchema>, Schemas.AEOPromptTopic[]> => ({+ name: 'aeo-prompts-topics-list',+ schema: AeoPromptsTopicsListSchema(),+ handler: async (context: Context, _params: z.infer<ReturnType<typeof AeoPromptsTopicsListSchema>>) => {+ const projectId = await context.stateManager.getProjectId()+ const result = await context.api.request<Schemas.AEOPromptTopic[]>({+ method: 'GET',+ path: `/api/projects/${encodeURIComponent(String(projectId))}/aeo_prompts/topics/`,+ })+ return result+ },+})++const AeoPromptsUpdateSchema = () => {+ const AeoPromptsPartialUpdateBody = orvalSchemas.AeoPromptsPartialUpdateBody()+ const AeoPromptsPartialUpdateParams = orvalSchemas.AeoPromptsPartialUpdateParams()+ return AeoPromptsPartialUpdateParams.omit({ project_id: true })+ .extend(AeoPromptsPartialUpdateBody.shape)+ .extend({+ id: AeoPromptsPartialUpdateParams.shape['id'].describe('The prompt id (a UUID), from `aeo-prompts-list`.'),+ })+}++const aeoPromptsUpdate = (): ToolBase<ReturnType<typeof AeoPromptsUpdateSchema>, Schemas.AEOPrompt> => ({+ name: 'aeo-prompts-update',+ schema: AeoPromptsUpdateSchema(),+ handler: async (context: Context, params: z.infer<ReturnType<typeof AeoPromptsUpdateSchema>>) => {+ const projectId = await context.stateManager.getProjectId()+ const body: Record<string, unknown> = {}+ if (params.active !== undefined) {+ body['active'] = params.active+ }+ const result = await context.api.request<Schemas.AEOPrompt>({+ method: 'PATCH',+ path: `/api/projects/${encodeURIComponent(String(projectId))}/aeo_prompts/${encodeURIComponent(String(params.id))}/`,+ body,+ })+ return result+ },+})++export const GENERATED_TOOLS: Record<string, () => ToolBase<ZodObjectAny>> = {+ 'aeo-prompts-get': aeoPromptsGet,+ 'aeo-prompts-import': aeoPromptsImport,+ 'aeo-prompts-list': aeoPromptsList,+ 'aeo-prompts-topics-list': aeoPromptsTopicsList,+ 'aeo-prompts-update': aeoPromptsUpdate,+}diff --git a/services/mcp/src/tools/generated/index.ts b/services/mcp/src/tools/generated/index.tsindex 7b91db4461d1..6cfe3a6e7714 100644--- a/services/mcp/src/tools/generated/index.ts+++ b/services/mcp/src/tools/generated/index.ts@@ -3,6 +3,7 @@ import type { ToolBase, ZodObjectAny } from '@/tools/types'import { GENERATED_TOOLS as access_control } from './access_control'import { GENERATED_TOOLS as actions } from './actions'+import { GENERATED_TOOLS as aeo } from './aeo'import { GENERATED_TOOLS as ai_observability } from './ai_observability'import { GENERATED_TOOLS as alerts } from './alerts'import { GENERATED_TOOLS as annotations } from './annotations'@@ -72,6 +73,7 @@ import { GENERATED_TOOLS as workflows } from './workflows'export const GENERATED_TOOL_MAP: Record<string, () => ToolBase<ZodObjectAny>> = {...access_control,...actions,+ ...aeo,...ai_observability,...alerts,...annotations,diff --git a/services/mcp/tests/unit/__snapshots__/tool-schemas/aeo-prompts-get.json b/services/mcp/tests/unit/__snapshots__/tool-schemas/aeo-prompts-get.jsonnew file mode 100644index 000000000000..86e86e954730--- /dev/null+++ b/services/mcp/tests/unit/__snapshots__/tool-schemas/aeo-prompts-get.json@@ -0,0 +1,11 @@+{+ "$schema": "https://json-schema.org/draft/2020-12/schema",+ "properties": {+ "id": {+ "description": "The prompt id (a UUID), from `aeo-prompts-list`.",+ "type": "string"+ }+ },+ "required": ["id"],+ "type": "object"+}diff --git a/services/mcp/tests/unit/__snapshots__/tool-schemas/aeo-prompts-import.json b/services/mcp/tests/unit/__snapshots__/tool-schemas/aeo-prompts-import.jsonnew file mode 100644index 000000000000..dbe14fcc7193--- /dev/null+++ b/services/mcp/tests/unit/__snapshots__/tool-schemas/aeo-prompts-import.json@@ -0,0 +1,42 @@+{+ "$schema": "https://json-schema.org/draft/2020-12/schema",+ "properties": {+ "prompts": {+ "description": "Prompts to add or update, at most 2000 per request.",+ "items": {+ "properties": {+ "external_visibility": {+ "anyOf": [+ {+ "maximum": 100,+ "minimum": 0,+ "type": "number"+ },+ {+ "type": "null"+ }+ ],+ "description": "Share of answers (0-100) that mention the brand for this prompt, as reported by the tool the prompt set came from. Kept to benchmark against our own citation rate."+ },+ "prompt": {+ "description": "The question to ask the answer engines, as a user would phrase it. Matching ignores case and whitespace, so a re-import updates the existing prompt.",+ "maxLength": 10000,+ "type": "string"+ },+ "topic": {+ "description": "Theme the prompt belongs to. The runner alternates between topics on each run.",+ "maxLength": 200,+ "type": "string"+ }+ },+ "required": ["prompt"],+ "type": "object"+ },+ "maxItems": 2000,+ "minItems": 1,+ "type": "array"+ }+ },+ "required": ["prompts"],+ "type": "object"+}diff --git a/services/mcp/tests/unit/__snapshots__/tool-schemas/aeo-prompts-list.json b/services/mcp/tests/unit/__snapshots__/tool-schemas/aeo-prompts-list.jsonnew file mode 100644index 000000000000..e5c265b455e6--- /dev/null+++ b/services/mcp/tests/unit/__snapshots__/tool-schemas/aeo-prompts-list.json@@ -0,0 +1,30 @@+{+ "$schema": "https://json-schema.org/draft/2020-12/schema",+ "properties": {+ "active": {+ "anyOf": [+ {+ "type": "boolean"+ },+ {+ "type": "null"+ }+ ],+ "description": "Only list prompts that are on (true) or off (false). Omit to list both."+ },+ "limit": {+ "description": "Number of results to return per page.",+ "type": "number"+ },+ "offset": {+ "description": "The initial index from which to return the results.",+ "type": "number"+ },+ "topic": {+ "description": "Only list prompts with this topic. An empty value lists the prompts that have no topic.",+ "maxLength": 200,+ "type": "string"+ }+ },+ "type": "object"+}diff --git a/services/mcp/tests/unit/__snapshots__/tool-schemas/aeo-prompts-topics-list.json b/services/mcp/tests/unit/__snapshots__/tool-schemas/aeo-prompts-topics-list.jsonnew file mode 100644index 000000000000..7b7ea9186078--- /dev/null+++ b/services/mcp/tests/unit/__snapshots__/tool-schemas/aeo-prompts-topics-list.json@@ -0,0 +1,5 @@+{+ "$schema": "https://json-schema.org/draft/2020-12/schema",+ "properties": {},+ "type": "object"+}diff --git a/services/mcp/tests/unit/__snapshots__/tool-schemas/aeo-prompts-update.json b/services/mcp/tests/unit/__snapshots__/tool-schemas/aeo-prompts-update.jsonnew file mode 100644index 000000000000..9fa4a43d098f--- /dev/null+++ b/services/mcp/tests/unit/__snapshots__/tool-schemas/aeo-prompts-update.json@@ -0,0 +1,15 @@+{+ "$schema": "https://json-schema.org/draft/2020-12/schema",+ "properties": {+ "active": {+ "description": "Turn the prompt on or off for future runs.",+ "type": "boolean"+ },+ "id": {+ "description": "The prompt id (a UUID), from `aeo-prompts-list`.",+ "type": "string"+ }+ },+ "required": ["id"],+ "type": "object"+}diff --git a/services/mcp/tests/unit/instructions.test.ts b/services/mcp/tests/unit/instructions.test.tsindex 710eaa1ce3c3..222d6a229e62 100644--- a/services/mcp/tests/unit/instructions.test.ts+++ b/services/mcp/tests/unit/instructions.test.ts@@ -140,6 +140,7 @@ describe('buildToolDomainsBlock', () => {'estimate','fail','freeze',+ 'import','launch','migrate','move',diff --git a/services/mcp/tests/unit/tool-filtering.test.ts b/services/mcp/tests/unit/tool-filtering.test.tsindex 76b1976407fd..463d6e51245c 100644--- a/services/mcp/tests/unit/tool-filtering.test.ts+++ b/services/mcp/tests/unit/tool-filtering.test.ts@@ -1153,11 +1153,12 @@ describe('Tool Filtering - Feature Flags', () => {'marketing-analytics-setup-plan-mcp','subscription-source-summaries','workflows-mcp-onboarding',+ 'aeo-citation-tracking',]))expect(flags).toContain('ai-observability-offline-evaluations')expect(allFlags).not.toContain('dashboard-widgets')- expect(flags).toHaveLength(34)+ expect(flags).toHaveLength(35)})it('keeps human and task context wiki tools on separate scopes', () => {diff --git a/services/mcp/tests/unit/tool-schema-snapshots.test.ts b/services/mcp/tests/unit/tool-schema-snapshots.test.tsindex ddae52ea4b48..e7dacf4c2f06 100644--- a/services/mcp/tests/unit/tool-schema-snapshots.test.ts+++ b/services/mcp/tests/unit/tool-schema-snapshots.test.ts@@ -109,6 +109,7 @@ async function loadSnapshotTools(): Promise<Tool<ZodObjectAny>[]> {'insight-data-checks': true,'workflows-transactional-email-api': true,'activity-summary': true,+ 'aeo-citation-tracking': true,}return [...(await getToolsFromContext(createMockContext(), { featureFlags }))].sort((a, b) =>a.name.localeCompare(b.name)