:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(signals): replace the activity summary with a one-page profile (#121378)
posthog[bot]a0970a31aec0
diff --git a/products/signals/backend/activity_summary/logic.py b/products/signals/backend/activity_summary/logic.pyindex 783e9a78842f..a638f8986501 100644--- a/products/signals/backend/activity_summary/logic.py+++ b/products/signals/backend/activity_summary/logic.py@@ -207,7 +207,7 @@ def update_activity_summary(model=model,as_of=now.date(),max_completion_tokens=limits.max_completion_tokens,- summary_max_words=limits.summary_max_words,+ profile_max_chars=limits.profile_max_chars,)cost = cost_usd(outcome.tokens_in, outcome.tokens_out, MODEL_PRICES.get(model))result = ActivitySummaryUpdate(diff --git a/products/signals/backend/activity_summary/prompt.py b/products/signals/backend/activity_summary/prompt.pyindex e3e2324cc93f..010f67fbdd7e 100644--- a/products/signals/backend/activity_summary/prompt.py+++ b/products/signals/backend/activity_summary/prompt.py@@ -9,9 +9,9 @@ import jsonimport datetimefrom typing import Any-from products.signals.backend.activity_summary.render import SUMMARY_MAX_WORDS, cap_line+from products.signals.backend.activity_summary.render import PROFILE_MAX_CHARS, cap_line-PROMPT_VERSION = 4+PROMPT_VERSION = 5MAX_FOCUS_AREAS = 6MAX_PRODUCTS = 10@@ -19,22 +19,35 @@ MAX_PRODUCTS = 10MAX_TOPIC_CHARS = 80MAX_PRODUCT_CHARS = 40MAX_RECENT_SHIFT_CHARS = 300+MAX_HEADLINE_CHARS = 600STRENGTHS = ("high", "medium", "low")DEFAULT_STRENGTH = "low"# Strings a model writes in place of a JSON null._NULL_STRINGS = frozenset({"", "none", "null", "n/a", "na", "nil"})-_SYSTEM_PROMPT = """You maintain a short summary of what one person has been working on in one PostHog project.-You get their previous summary, if any, and a sample of their recent activity in up to three sections: tasks they gave to PostHog AI or the Slack agent, things they created or changed in PostHog, with the reason their coding agent gave when an agent made the change, and things they opened in PostHog.-Update the summary to reflect what they are focused on now. Keep what is still true, drop what has gone quiet, and add what is new. Weigh themes that repeat across sections above one-off items. Treat what they open on many days as part of their work, even when they changed nothing. Treat bulk changes of the same kind as one theme, not a list.-Name concrete products, features, metrics, pipelines and workflows. Never write generic phrases like "data analysis".-Write about the work, not the person: no names, emails, IDs, secrets, or quotes from the input. Open with the work itself.-Return JSON: `summary` (at most {max_words} words), `focus_areas` (up to 6, each with `topic`, `strength` high, medium or low, and `last_seen` as YYYY-MM-DD), `products`, and `recent_shift` (one sentence, or null).-Set `recent_shift` only when there is a previous summary and the activity shows a real change from it. Otherwise it is null."""+_SYSTEM_PROMPT = """You write an activity summary: what one person has been doing in one PostHog project, built only from the activity below. People and their agents read it to understand what this person works on and cares about.+The input has up to three sections, plus the previous summary if there is one:+- Tasks: work the person gave to PostHog AI or the Slack agent. Each line has the day, the tags in brackets, and the stated goal.+- Things they created or changed in PostHog: each line has the day, the action, the type and the name of the object. "(xN)" is the number of changes. "agent" is the reason their coding agent gave for the change.+- Things they opened in PostHog: one line per object, or per product for its pages. Each line has the number of distinct days they opened it, the last day, the number of days they edited it if any, and "not theirs" when someone else made it. Treat what they open on many days as part of their work, even when they changed nothing.-def system_prompt(max_words: int = SUMMARY_MAX_WORDS) -> str:- return _SYSTEM_PROMPT.format(max_words=max_words)+Each section shows a different side of the same person, and the evidence can overlap. Use your own judgment about what matters. Stay with what the input shows, and say so when something is unclear.+The activity covers only the time since the previous summary. Keep what the previous summary says unless the new activity changes it.+Give a line to every thread that has repeated evidence. Use the concrete names that the input uses, such as features, dashboards and experiments.++Write about the work, not the person: no names, emails, IDs, secrets, or quotes from the input.++Return JSON:+- `profile`: a one-page markdown profile of the person's work in this project, at most {profile_max_chars} characters. Choose your own sections.+- `headline`: two or three plain sentences that capture the profile.+- `focus_areas`: the main threads, up to 6, each with `topic`, `strength` (high, medium or low) and `last_seen` as YYYY-MM-DD.+- `products`: the PostHog products they use, by name.+- `recent_shift`: one sentence about what changed since the previous summary, or null when there is no previous summary or no real change."""+++def system_prompt(profile_max_chars: int = PROFILE_MAX_CHARS) -> str:+ return _SYSTEM_PROMPT.format(profile_max_chars=profile_max_chars)# Strict structured output. The normalization below still checks every rule, because a provider@@ -47,7 +60,8 @@ RESPONSE_FORMAT: Any = {"schema": {"type": "object","properties": {- "summary": {"type": "string"},+ "profile": {"type": "string"},+ "headline": {"type": "string"},"focus_areas": {"type": "array","items": {@@ -64,7 +78,7 @@ RESPONSE_FORMAT: Any = {"products": {"type": "array", "items": {"type": "string"}},"recent_shift": {"type": ["string", "null"]},},- "required": ["summary", "focus_areas", "products", "recent_shift"],+ "required": ["profile", "headline", "focus_areas", "products", "recent_shift"],"additionalProperties": False,},},@@ -75,17 +89,43 @@ class MalformedSummaryError(ValueError):"""The model's answer cannot be turned into a summary. The message never quotes the answer."""-def user_message(rendered_input: str, previous_summary: dict[str, Any] | None, as_of: datetime.date) -> str:+def _profile(value: Any, max_chars: int) -> str | None:+ """The markdown profile with its line breaks, cut at a word boundary when it is longer than `max_chars`."""+ if not isinstance(value, str):+ return None+ text = value.strip()+ if text.lower() in _NULL_STRINGS:+ return None+ if len(text) <= max_chars:+ return text+ cut = text[: max_chars - 1]+ boundary = max(cut.rfind(" "), cut.rfind("\n"))+ return (cut[:boundary] if boundary > 0 else cut).rstrip() + "…"+++def _previous(previous_summary: dict[str, Any], profile_max_chars: int) -> dict[str, Any]:+ # A summary stored before prompt version 3 has a `summary` and no `headline` or `profile`.+ # A profile stored under a larger limit gets cut to the current one, so the input stays inside the cost bound.+ return {+ "headline": previous_summary.get("headline") or previous_summary.get("summary"),+ "profile": _profile(previous_summary.get("profile"), profile_max_chars),+ "focus_areas": previous_summary.get("focus_areas"),+ "products": previous_summary.get("products"),+ }+++def user_message(+ rendered_input: str,+ previous_summary: dict[str, Any] | None,+ as_of: datetime.date,+ profile_max_chars: int = PROFILE_MAX_CHARS,+) -> str:"""The user turn: the day the activity ends, the previous summary and the rendered input.The rendered input comes from `build_input` as is, and the previous summary is this pipeline'sown stored output, so no other text reaches the model."""previous = (- json.dumps(- {key: previous_summary.get(key) for key in ("summary", "focus_areas", "products")},- ensure_ascii=False,- sort_keys=True,- )+ json.dumps(_previous(previous_summary, profile_max_chars), ensure_ascii=False, sort_keys=True)if previous_summaryelse "None")@@ -101,11 +141,6 @@ def _clean(value: Any, max_chars: int | None = None) -> str | None:return cap_line(text, max_chars) if max_chars is not None else text-def cap_words(text: str, max_words: int = SUMMARY_MAX_WORDS) -> str:- words = text.split()- return text if len(words) <= max_words else " ".join(words[:max_words])--def _last_seen(value: Any) -> str | None:text = _clean(value)if text is None:@@ -148,12 +183,12 @@ def _products(value: Any) -> list[str]:def normalize_summary(- content: str | None, *, had_previous_summary: bool, max_words: int = SUMMARY_MAX_WORDS+ content: str | None, *, had_previous_summary: bool, profile_max_chars: int = PROFILE_MAX_CHARS) -> dict[str, Any]:"""The model's JSON answer under the output rules, or `MalformedSummaryError`.- Only a missing or empty `summary` makes an answer malformed. Every other field degrades to an- empty or null value, so one bad focus area does not cost the row."""+ Only a missing or empty `profile` or `headline` makes an answer malformed. Every other field+ degrades to an empty or null value, so one bad focus area does not cost the row."""if not content:raise MalformedSummaryError("empty response")try:@@ -162,12 +197,16 @@ def normalize_summary(raise MalformedSummaryError("response is not JSON") from errorif not isinstance(parsed, dict):raise MalformedSummaryError("response is not a JSON object")- summary = _clean(parsed.get("summary"))- if summary is None:- raise MalformedSummaryError("response has no summary")+ profile = _profile(parsed.get("profile"), profile_max_chars)+ if profile is None:+ raise MalformedSummaryError("response has no profile")+ headline = _clean(parsed.get("headline"))+ if headline is None:+ raise MalformedSummaryError("response has no headline")recent_shift = _clean(parsed.get("recent_shift"), MAX_RECENT_SHIFT_CHARS) if had_previous_summary else Nonereturn {- "summary": cap_words(summary, max_words),+ "profile": profile,+ "headline": cap_line(headline, MAX_HEADLINE_CHARS),"focus_areas": _focus_areas(parsed.get("focus_areas")),"products": _products(parsed.get("products")),"recent_shift": recent_shift,@@ -175,6 +214,6 @@ def normalize_summary(def summary_text(summary: dict[str, Any]) -> str:- """The text that gets embedded: the summary, then the focus area topics in the model's order."""+ """The text that gets embedded: the headline, then the focus area topics in the model's order."""topics = [area["topic"] for area in summary.get("focus_areas", [])]- return f"{summary['summary']}\nFocus areas: {'; '.join(topics)}" if topics else summary["summary"]+ return f"{summary['headline']}\nFocus areas: {'; '.join(topics)}" if topics else summary["headline"]diff --git a/products/signals/backend/activity_summary/queue.py b/products/signals/backend/activity_summary/queue.pyindex f5016d87a970..c499adf3523f 100644--- a/products/signals/backend/activity_summary/queue.py+++ b/products/signals/backend/activity_summary/queue.py@@ -57,7 +57,13 @@ class DailyUsage:def max_update_cost_usd(limits: RenderLimits = DEFAULT_LIMITS, model: str = DEFAULT_MODEL) -> float:"""The most one update can cost: every attempt with the largest input and output."""- input_chars = limits.sections_max_chars + MAX_SUMMARY_CHARS + len(system_prompt(limits.summary_max_words))+ # The previous summary is a profile, cut to `limits.profile_max_chars`, plus fields that `MAX_SUMMARY_CHARS` covers.+ input_chars = (+ limits.sections_max_chars+ + limits.profile_max_chars+ + MAX_SUMMARY_CHARS+ + len(system_prompt(limits.profile_max_chars))+ )bound = cost_usd(MAX_ATTEMPTS * input_chars // _BOUND_CHARS_PER_TOKEN,MAX_ATTEMPTS * limits.max_completion_tokens,diff --git a/products/signals/backend/activity_summary/render.py b/products/signals/backend/activity_summary/render.pyindex 324eef2c99bb..7c2c634d7e04 100644--- a/products/signals/backend/activity_summary/render.py+++ b/products/signals/backend/activity_summary/render.py@@ -22,9 +22,6 @@ from urllib.parse import urlsplitfrom posthog.dataclasses import frozen-# Words of the stored summary. It lives here and not in `prompt`, because `prompt` imports from this module.-SUMMARY_MAX_WORDS = 60-# Bump on any code change to the filters, the default caps, the sampling or the rendered layout, so summaries# stay comparable. Limits from the flag payload go into `inputs_used` instead.RENDER_VERSION = 6@@ -61,6 +58,8 @@ VIEW_LINE_CHARS = 160SECTIONS_MAX_CHARS = 8000# Hidden reasoning tokens count against the cap too, so a cap near the answer size truncates answers.MAX_COMPLETION_TOKENS = 4000+# About 1,500 output tokens: one page of markdown. The prompt asks for it, and the normalization cuts a longer profile.+PROFILE_MAX_CHARS = 6000# Masking and whitespace folding shorten a text, so the patterns read this many times its cap.RAW_TEXT_FACTOR = 4@@ -159,7 +158,7 @@ class RenderLimits:task_goal_chars: int = _limit(TASK_GOAL_CHARS, max_value=800)activity_line_chars: int = _limit(ACTIVITY_LINE_CHARS, max_value=1000)activity_intent_chars: int = _limit(ACTIVITY_INTENT_CHARS, max_value=500)- summary_max_words: int = _limit(SUMMARY_MAX_WORDS, max_value=300)+ profile_max_chars: int = _limit(PROFILE_MAX_CHARS, max_value=20000)max_completion_tokens: int = _limit(MAX_COMPLETION_TOKENS, max_value=16000)diff --git a/products/signals/backend/activity_summary/summaries.py b/products/signals/backend/activity_summary/summaries.pyindex b47d584f41a0..2d7b5fe6fabd 100644--- a/products/signals/backend/activity_summary/summaries.py+++ b/products/signals/backend/activity_summary/summaries.py@@ -21,7 +21,7 @@ from products.signals.backend.activity_summary.prompt import (system_prompt,user_message,)-from products.signals.backend.activity_summary.render import MAX_COMPLETION_TOKENS, SUMMARY_MAX_WORDS+from products.signals.backend.activity_summary.render import MAX_COMPLETION_TOKENS, PROFILE_MAX_CHARSAI_PRODUCT = "activity_summary"# Not a person: the gateway attributes every call to the pipeline.@@ -88,12 +88,12 @@ def summarize(model: str,as_of: datetime.date,max_completion_tokens: int = MAX_COMPLETION_TOKENS,- summary_max_words: int = SUMMARY_MAX_WORDS,+ profile_max_chars: int = PROFILE_MAX_CHARS,) -> SummaryOutcome:"""One summary. Tokens and latency add up over the attempts, because the retry costs too."""messages: list[ChatCompletionMessageParam] = [- {"role": "system", "content": system_prompt(summary_max_words)},- {"role": "user", "content": user_message(rendered_input, previous_summary, as_of)},+ {"role": "system", "content": system_prompt(profile_max_chars)},+ {"role": "user", "content": user_message(rendered_input, previous_summary, as_of, profile_max_chars)},]tokens_in: int | None = Nonetokens_out: int | None = None@@ -122,7 +122,7 @@ def summarize(summary = normalize_summary(choice.message.content if choice is not None else None,had_previous_summary=previous_summary is not None,- max_words=summary_max_words,+ profile_max_chars=profile_max_chars,)except MalformedSummaryError as malformed:error = f"malformed_output: {malformed}"diff --git a/products/signals/backend/test/test_activity_summary_measure.py b/products/signals/backend/test/test_activity_summary_measure.pyindex 63b0d8e7a4da..dd418c526515 100644--- a/products/signals/backend/test/test_activity_summary_measure.py+++ b/products/signals/backend/test/test_activity_summary_measure.py@@ -23,7 +23,8 @@ from products.signals.backend.test.test_activity_summary_render import FakeLLMNOW = datetime.datetime(2026, 10, 19, 12, tzinfo=datetime.UTC)ANSWER = json.dumps({- "summary": "Builds the signup funnel.",+ "profile": "## Funnels\n- Builds the signup funnel.",+ "headline": "Builds the signup funnel.","focus_areas": [{"topic": "signup funnel", "strength": "high", "last_seen": "2026-10-18"}],"products": ["product_analytics"],"recent_shift": None,diff --git a/products/signals/backend/test/test_activity_summary_render.py b/products/signals/backend/test/test_activity_summary_render.pyindex 0f7db9c48bd4..5b7a020ffd58 100644--- a/products/signals/backend/test/test_activity_summary_render.py+++ b/products/signals/backend/test/test_activity_summary_render.py@@ -453,26 +453,42 @@ def test_build_input_drops_every_unit_for_a_budget_below_the_section_overhead()assert (rendered.text, rendered.units, rendered.drops[Drop.SECTIONS_BUDGET]) == ("", 0, 2)-PREVIOUS = {"summary": "Builds the signup funnel.", "focus_areas": [], "products": ["product_analytics"]}+PREVIOUS = {+ "profile": "## Funnels\n- Builds the signup funnel.",+ "headline": "Builds the signup funnel.",+ "focus_areas": [],+ "products": ["product_analytics"],+ "recent_shift": "Moved from flags to funnels.",+}+PREVIOUS_BEFORE_PROFILES = {"summary": "Builds the signup funnel.", "focus_areas": [], "products": []}[email protected]([("with a previous summary", PREVIOUS), ("cold start", None)])+ [+ (+ "with a previous summary",+ PREVIOUS,+ ["## Funnels\\n- Builds the signup funnel.", "Builds the signup funnel."],+ ),+ ("with a summary from before profiles", PREVIOUS_BEFORE_PROFILES, ['"headline": "Builds the signup funnel."']),+ ("cold start", None, ["Previous summary:\nNone"]),+ ]+)def test_user_message_sends_only_the_rendered_input_and_the_previous_summary(- _name: str, previous: dict[str, Any] | None+ _name: str, previous: dict[str, Any] | None, expected: list[str]) -> None:message = user_message("- 2026-10-17 Build a funnel", previous, datetime.date(2026, 10, 18))assert message.startswith("Today is 2026-10-18.")assert message.endswith("Recent activity:\n- 2026-10-17 Build a funnel")- if previous:- assert json.dumps(PREVIOUS["summary"]) in message- else:- assert "Previous summary:\nNone" in message+ assert all(text in message for text in expected)+ assert "Moved from flags" not in messagedef _answer(**overrides: Any) -> str:return json.dumps({- "summary": "Builds the signup funnel.",+ "profile": "## Funnels\n- Builds the signup funnel.",+ "headline": "Builds the signup funnel.","focus_areas": [{"topic": "signup funnel", "strength": "high", "last_seen": "2026-10-17"}],"products": ["product_analytics"],"recent_shift": "Moved from flags to funnels.",@@ -493,10 +509,10 @@ def _answer(**overrides: Any) -> str:("a 'None' recent_shift becomes null", _answer(recent_shift="None"), True, {"recent_shift": None}),("a 'null' recent_shift becomes null", _answer(recent_shift=" null "), True, {"recent_shift": None}),(- "the summary is cut to the word cap",- _answer(summary="word " * 80),+ "a long profile keeps its line breaks and is cut at a word boundary",+ _answer(profile="## Funnels\n" + "word " * 2000),False,- {"summary": " ".join(["word"] * 60)},+ {"profile": "## Funnels\n" + " ".join(["word"] * 1197) + "…"},),("at most six focus areas, an unknown strength is low and a bad date is null",@@ -547,7 +563,8 @@ def test_normalize_summary(_name: str, content: str, had_previous: bool, expecte("empty", None),("not JSON", "Here is the summary: ..."),("not an object", "[1, 2]"),- ("no summary", _answer(summary=" ")),+ ("no profile", _answer(profile=" ")),+ ("no headline", _answer(headline=None)),])def test_normalize_summary_rejects_a_malformed_answer(_name: str, content: str | None) -> None:@@ -681,3 +698,23 @@ def test_summarize_reports_null_tokens_when_the_response_has_no_usage() -> None:)assert outcome.summary is not Noneassert (outcome.tokens_in, outcome.tokens_out) == (None, None)+++def test_summarize_cuts_a_previous_profile_to_a_lowered_limit() -> None:+ stored_profile = "## Funnels\n" + "Builds the signup funnel. " * 230+ assert len(stored_profile) > 5000+ llm = FakeLLM([_answer()])++ summarize(+ llm, # type: ignore[arg-type]+ rendered_input="- a line",+ previous_summary={**PREVIOUS, "profile": stored_profile},+ model="gpt-6-luna",+ as_of=datetime.date(2026, 10, 18),+ profile_max_chars=500,+ )++ user_turn = llm.requests[0]["messages"][1]["content"]+ previous = json.loads(user_turn.split("Previous summary:\n", 1)[1].split("\n\nRecent activity:", 1)[0])+ assert len(previous["profile"]) <= 500+ assert previous["profile"].startswith("## Funnels\nBuilds the signup funnel.")diff --git a/products/signals/backend/test/test_activity_summary_tick.py b/products/signals/backend/test/test_activity_summary_tick.pyindex d1c7f282806f..2c63ce12c416 100644--- a/products/signals/backend/test/test_activity_summary_tick.py+++ b/products/signals/backend/test/test_activity_summary_tick.py@@ -59,7 +59,8 @@ from products.tasks.backend.facade.user_task_summaries import UserTaskTimeNOW = datetime.datetime(2026, 10, 19, 12, tzinfo=datetime.UTC)ANSWER = json.dumps({- "summary": "Builds the signup funnel.",+ "profile": "## Funnels\n- Builds the signup funnel.",+ "headline": "Builds the signup funnel.","focus_areas": [{"topic": "signup funnel", "strength": "high", "last_seen": "2026-10-18"}],"products": ["product_analytics"],"recent_shift": None,diff --git a/products/signals/backend/test/test_activity_summary_update.py b/products/signals/backend/test/test_activity_summary_update.pyindex 217bf84155c4..c7d7a198a08c 100644--- a/products/signals/backend/test/test_activity_summary_update.py+++ b/products/signals/backend/test/test_activity_summary_update.py@@ -23,7 +23,8 @@ NOW = datetime.datetime(2026, 10, 19, 12, tzinfo=datetime.UTC)PATH_SECRET = "https://hooks.example.com/services/T000/B000/s3cr3tPathValue"ANSWER = json.dumps({- "summary": "Builds the signup funnel.",+ "profile": "## Funnels\n- Builds the signup funnel.",+ "headline": "Builds the signup funnel.","focus_areas": [{"topic": "signup funnel", "strength": "high", "last_seen": "2026-10-18"}],"products": ["product_analytics"],"recent_shift": None,@@ -130,13 +131,13 @@ class TestUpdateActivitySummary(BaseTest):self._change("Old funnel", days_ago=10)llm = FakeLLM([ANSWER])- self._update(llm, limits=RenderLimits(window_days=7, summary_max_words=2, max_completion_tokens=1234))+ self._update(llm, limits=RenderLimits(window_days=7, profile_max_chars=12, max_completion_tokens=1234))stored = self._stored()assert stored is not None and stored.summary is not Noneassert llm.requests[0]["max_completion_tokens"] == 1234- assert "at most 2 words" in llm.requests[0]["messages"][0]["content"]- assert stored.summary["summary"] == "Builds the"+ assert "at most 12 characters" in llm.requests[0]["messages"][0]["content"]+ assert stored.summary["profile"] == "## Funnels…"assert stored.inputs_used["activity_lines"] == 3assert stored.inputs_used["limits"]["max_completion_tokens"] == 1234