:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
chore(signals): remove the dagster activity summary pipeline (#120468)
posthog[bot]e84a8069a589
diff --git a/posthog/dags/locations/signals.py b/posthog/dags/locations/signals.pyindex 63adf44281b1..26a106e47fc1 100644--- a/posthog/dags/locations/signals.py+++ b/posthog/dags/locations/signals.py@@ -1,6 +1,5 @@import dagster-from products.signals.dags.inbox_ranking.activity_summary import dag as activity_summaryfrom products.signals.dags.inbox_ranking.common import is_inbox_ranking_registeredfrom products.signals.dags.inbox_ranking.dataset import dag as inbox_ranking_datasetfrom products.signals.dags.inbox_ranking.training import (@@ -31,22 +30,16 @@ if is_inbox_ranking_registered():inbox_ranking_training.inbox_ranking_serving_manifest,inbox_ranking_served.inbox_ranking_served_scores,inbox_ranking_training.inbox_ranking_unseen_graded,- activity_summary.activity_summary_inputs,- activity_summary.activity_summary_cost_projection,- activity_summary.activity_summaries,- activity_summary.activity_summary_embeddings,],jobs=[inbox_ranking_dataset.inbox_ranking_dataset_job,inbox_ranking_dataset.inbox_ranking_labels_refresh_job,inbox_ranking_training.inbox_ranking_training_job,inbox_ranking_promote.inbox_ranking_promote_job,- activity_summary.activity_summary_job,],schedules=[inbox_ranking_dataset.inbox_ranking_dataset_schedule,inbox_ranking_training.inbox_ranking_training_schedule,- activity_summary.activity_summary_schedule,],sensors=[inbox_ranking_dataset.inbox_ranking_labels_refresh_sensor],loggers=loggers,diff --git a/products/signals/dags/inbox_ranking/README.md b/products/signals/dags/inbox_ranking/README.mdindex eb6cd89b8e8d..556625803e96 100644--- a/products/signals/dags/inbox_ranking/README.md+++ b/products/signals/dags/inbox_ranking/README.md@@ -18,10 +18,11 @@ inbox_ranking/│ └── promotion.py # the candidate grading rule├── multihead/ # the multi-head network, its ONNX export, and the offline prototype├── nn/ # the `nn` family: examples → candidate → champion assets, on the `signals_ml` location-├── activity_summary/ # per-person activity summaries: inputs, cost projection, LLM summaries and their vectors└── tests/```+Per-person activity summaries are not a dag. They live in `products/signals/backend/activity_summary/`.+Registered via `posthog/dags/locations/signals.py` (US only) and loaded locally through `.dagster_home/workspace.yaml`. The `nn` family is registered via `posthog/dags/locations/signals_ml.py` instead, because it needs torch (see [The nn family](#the-nn-family)).## The dataset dag@@ -374,47 +375,6 @@ dagster job launch -w .dagster_home/workspace.yaml --location posthog.dags.locat-j inbox_ranking_nn_training_job --tags '{"dagster/partition": "2026-08-25"}'```-## The activity summary dag--`activity_summary/` builds per-person activity summaries and their vectors for inbox ranking.-`activity_summary_job` runs four assets on the shared daily partition, over the 28 days that end with D.-Its weekly schedule ships stopped, so run the job by hand.--- `activity_summary_inputs/v1/dt=D/` holds one row per (project, member) for active members of the dogfood organization with an `@posthog.com` email, in that organization's own projects only.- `rendered_input` has up to three sections, built by `render.py`:- - tasks the member started in PostHog AI, Slack or by hand, as `[tags] goal` from the latest run's `task_summary` and `task_tags`, read through the tasks facade;- - the member's own MCP sessions (`source` `mcp` with the `user` preset or none) from the dogfood project's events, as `first step → last step`;- - the member's activity log, one line per changed object.-- The sampling seed comes from (partition, team, user), so a re-run writes the same object.-- `activity_summary_cost_projection/v1/dt=D/` covers every MCP (project, distinct id) pair in the events and holds counts only, never free text.- It has one row per `MIN_UNITS` and `MIN_DAYS` option, with eligible pairs, input token percentiles and monthly volume.- The task and activity sections go in at their capped maximum (`capped_sections_tokens`), so the projection errs high.- Set `input_price_per_million_tokens` and `output_price_per_million_tokens` in the run config to fill `projected_monthly_cost_usd`.-- `activity_summaries/v1/dt=D/` holds one LLM summary per inputs row, made through `build_openai_client` with the prompt in `prompt.py`.- The model gets the rendered input and the pair's previous summary: its successful row in the newest earlier partition, within `PREVIOUS_SUMMARY_MAX_AGE_DAYS`.- A malformed answer is retried once, then the row is written with `status = failed`. When every row fails, the asset fails and leaves the partition as it was.- `summary_text` is the text that gets embedded, so bump `PROMPT_VERSION` with any change to the prompt, the normalization or `summary_text`.- Run config: `model`, `use_previous_summary` (off compares the stateless summaries), the two token prices for `cost_usd`, and `max_concurrency`.- Each row also captures an `activity summary generated` event into the dogfood project, with counts only unless `capture_summary_text` is set.-- `activity_summary_embeddings/v1/dt=D/` holds the `text-embedding-3-small` vector of each successful summary.- The asset sends a summary to the embedding worker only when `document_embeddings` has no vector of its current text, waits up to `wait_seconds`, and copies out what landed.- A vector that lands later is picked up by the next materialization; `missing_vectors` in the metadata counts them.- The source table drops a row three months after the partition day, so this Parquet is the durable copy, and a re-run that would write fewer rows is refused.--Most activity recorded under a person is not theirs: scouts, agents and scheduled jobs act with the person's identity.-Every filter in `render.py` is a named rule in `Drop`, and each inputs row records in `drops` how many units or calls each rule removed.-When a rule drops everything for a person, or stops dropping anything, start there.--The input size is bounded for every person, in this order:--1. Rows read per person, newest first: `MAX_TASKS_PER_PERSON`, `MAX_MCP_SESSIONS_PER_PERSON`, `MAX_ACTIVITY_ROWS_PER_PERSON`.-2. The named filters.-3. At most `MAX_UNITS_PER_DAY` tasks and sessions per UTC day (activity lines are already one per object and skip this).-4. A recency-weighted sample: `TASK_LINES`, `MCP_LINES` and `ACTIVITY_LINES`.-5. A character cap on every line, and `SECTIONS_MAX_CHARS` as a hard stop on all sections.--Bump `RENDER_VERSION` with any change to a filter, a cap or the layout.-## Configuration| Setting | Default | Meaning |diff --git a/products/signals/dags/inbox_ranking/activity_summary/dag.py b/products/signals/dags/inbox_ranking/activity_summary/dag.pydeleted file mode 100644index 4c2c9e87430d..000000000000--- a/products/signals/dags/inbox_ranking/activity_summary/dag.py+++ /dev/null@@ -1,850 +0,0 @@-"""Per-person activity summaries: their inputs, a cost projection, the summaries and their vectors.--Four assets on the shared daily partition, each writing Parquet under the dataset prefix:-- activity_summary_inputs/v1/dt=D/ one filtered, rendered input per (project, member)- activity_summary_cost_projection/v1/dt=D/ eligible pairs, token percentiles and monthly volume- activity_summaries/v1/dt=D/ one LLM summary per inputs row- activity_summary_embeddings/v1/dt=D/ one vector per summary, kept past the source table's TTL--The inputs asset covers only members of the dogfood project's organization with an `@posthog.com`-email, in that organization's own projects, so no customer project's activity gets in. It reads-three sources over the 28 days that end with D: the person's own MCP sessions from the dogfood-project's events, the tasks they started in PostHog AI, Slack or the desktop app, and their-activity log. The projection covers every MCP pair in the events and writes counts only.-"""--import json-import time-import datetime-from collections import defaultdict-from collections.abc import Collection-from typing import Any--from django.db.models.fields.json import KeyTextTransform--import dagster-import pyarrow as pa--from posthog import settings-from posthog.clickhouse.query_tagging import Feature, Product, get_query_tags, tag_queries-from posthog.dags.common import dagster_tags-from posthog.dataclasses import frozen-from posthog.models import ActivityLog, OrganizationMembership, Team, User--from products.signals.dags.inbox_ranking.activity_summary import queries-from products.signals.dags.inbox_ranking.activity_summary.embeddings import (- EMBEDDING_MODEL,- LandedVector,- SummaryDocument,- document_id,- emit_summary_embedding,- flush_embedding_requests,- landed_vectors,- pending_documents,- within_source_ttl,-)-from products.signals.dags.inbox_ranking.activity_summary.prompt import PROMPT_VERSION-from products.signals.dags.inbox_ranking.activity_summary.render import (- MAX_ACTIVITY_ROWS_PER_PERSON,- MAX_TASKS_PER_PERSON,- RENDER_VERSION,- WINDOW_DAYS,- ActivityItem,- McpCall,- McpSession,- PairActivity,- ProjectionPrices,- TaskItem,- build_input,- calibration_ratio,- capped_sections_tokens,- estimated_input_chars,- job_intents,- projection_rows,- sampling_seed,-)-from products.signals.dags.inbox_ranking.activity_summary.summaries import (- DEFAULT_MODEL,- PIPELINE_DISTINCT_ID,- STATUS_OK,- SummaryPrices,- SummaryRequest,- gateway_client,- summarize,- summarize_all,- summary_events,- summary_row,-)-from products.signals.dags.inbox_ranking.common import (- DATASET_VERSION,- PARQUET_PART_NAME,- S3_BUCKET_ENV,- dataset_bucket,- dataset_unconfigured,- ensure_utc,- object_row_count,- owner_tags,- partition_def,- partition_object_key,- partition_write_allowed,- read_parquet_if_exists,- s3_client,- skip_unconfigured,- snapshot_bounds,- write_parquet,-)-from products.signals.dags.inbox_ranking.dataset.queries import LABELS_TEAM_ID, labels_team-from products.signals.dags.inbox_ranking.training.telemetry import capture_training_events-from products.tasks.backend.facade.user_task_summaries import list_user_task_summaries--INPUTS_TABLE = "activity_summary_inputs"-COST_PROJECTION_TABLE = "activity_summary_cost_projection"-SUMMARIES_TABLE = "activity_summaries"-SUMMARY_EMBEDDINGS_TABLE = "activity_summary_embeddings"--# The newest earlier summary of a pair counts as its previous summary only within this many days. An-# older one describes work that has gone quiet, so the pair starts over.-PREVIOUS_SUMMARY_MAX_AGE_DAYS = 60--COHORT_EMAIL_DOMAIN = "@posthog.com"-# Tasks a person started: in PostHog AI, in Slack, or by hand in the web or desktop app. Scouts,-# reports and workflows start tasks under a person's name too, and those are not the person's intent.-TASK_ORIGIN_PRODUCTS = ("posthog_ai", "slack", "user_created")--_TIMESTAMP = pa.timestamp("us", tz="UTC")--_INPUTS_FIELDS: list[tuple[str, pa.DataType]] = [- ("team_id", pa.int64()),- ("user_id", pa.int64()),- ("window_start", _TIMESTAMP),- ("window_end", _TIMESTAMP),- ("n_tasks", pa.int32()),- ("n_mcp_sessions", pa.int32()),- ("n_activity_lines", pa.int32()),- ("rendered_input", pa.string()),- ("task_chars", pa.int32()),- ("mcp_chars", pa.int32()),- ("activity_chars", pa.int32()),- ("input_chars", pa.int32()),- ("input_tokens_estimate", pa.int32()),- # JSON object of filter rule name to the units or calls it removed for this pair.- ("drops", pa.string()),- ("render_version", pa.int32()),-]-INPUTS_SCHEMA = pa.schema(_INPUTS_FIELDS)--# No free-text column, and none may be added: the projection covers every person in the events.-_COST_PROJECTION_FIELDS: list[tuple[str, pa.DataType]] = [- ("window_start", _TIMESTAMP),- ("window_end", _TIMESTAMP),- ("min_units", pa.int32()),- ("min_days", pa.int32()),- ("eligible_pairs", pa.int64()),- ("input_tokens_p50", pa.int64()),- ("input_tokens_p90", pa.int64()),- ("input_tokens_p99", pa.int64()),- ("updates_per_month", pa.float64()),- ("input_tokens_per_month", pa.float64()),- ("output_tokens_per_month", pa.float64()),- ("projected_monthly_cost_usd", pa.float64()),- ("calibration_ratio", pa.float64()),-]-COST_PROJECTION_SCHEMA = pa.schema(_COST_PROJECTION_FIELDS)--_SUMMARIES_FIELDS: list[tuple[str, pa.DataType]] = [- ("team_id", pa.int64()),- ("user_id", pa.int64()),- # The normalized answer: summary, focus_areas, products and recent_shift. Null when status is failed.- ("summary_json", pa.string()),- ("summary_text", pa.string()),- ("used_previous_summary", pa.bool_()),- ("model", pa.string()),- ("prompt_version", pa.int32()),- ("tokens_in", pa.int64()),- ("tokens_out", pa.int64()),- ("latency_ms", pa.int64()),- ("cost_usd", pa.float64()),- ("status", pa.string()),- ("error", pa.string()),-]-SUMMARIES_SCHEMA = pa.schema(_SUMMARIES_FIELDS)--_SUMMARY_EMBEDDINGS_FIELDS: list[tuple[str, pa.DataType]] = [- ("team_id", pa.int64()),- ("user_id", pa.int64()),- ("prompt_version", pa.int32()),- ("embedding", pa.list_(pa.float32())),- ("embedding_model", pa.string()),-]-SUMMARY_EMBEDDINGS_SCHEMA = pa.schema(_SUMMARY_EMBEDDINGS_FIELDS)--ASSET_KWARGS: dict[str, Any] = {- "group_name": "activity_summary",- "partitions_def": partition_def,- "tags": owner_tags,- "retry_policy": dagster.RetryPolicy(max_retries=2, delay=60),- "pool": "inbox_ranking_etl",-}---class CostProjectionConfig(dagster.Config):- """Prices are per million tokens and stay unset by default, so a run fills in the current price of- the model it projects for. Without both prices the cost column stays null."""-- input_price_per_million_tokens: float | None = None- output_price_per_million_tokens: float | None = None- # The previous summary (~400 tokens) and the instructions (~500) that every update sends.- prompt_overhead_tokens: int = 900- # The task and activity sections, which the projection does not read. Their caps bound them, and- # the default is that bound, so the projection errs high.- capped_sections_tokens: int = capped_sections_tokens()- output_tokens_per_update: int = 500---class SummariesConfig(dagster.Config):- """Prices are per million tokens and stay unset by default, like the cost projection's, so- `cost_usd` stays null until a run fills in the current price of `model`."""-- model: str = DEFAULT_MODEL- # Off to compare the stateful and the stateless summaries on the same inputs.- use_previous_summary: bool = True- # Puts the summary text and focus topics on the events. Off by default, because an event cannot be- # edited or deleted the way a partition can.- capture_summary_text: bool = False- input_price_per_million_tokens: float | None = None- output_price_per_million_tokens: float | None = None- max_concurrency: int = 8---class SummaryEmbeddingsConfig(dagster.Config):- # How long the asset waits for the embedding worker to write the vectors it asked for. A vector- # that lands later is picked up by the next materialization of the partition.- wait_seconds: int = 600- poll_interval_seconds: int = 30---@frozen-class Window:- start: datetime.datetime- end: datetime.datetime---def activity_window(partition_key: str) -> Window:- """The `WINDOW_DAYS` days that end with the partition day."""- _, end = snapshot_bounds(partition_key)- return Window(start=end - datetime.timedelta(days=WINDOW_DAYS), end=end)---def _tag_queries(context: dagster.AssetExecutionContext, query_type: str) -> None:- tag_queries(product=Product.SIGNALS, feature=Feature.DATA_MODELING, team_id=LABELS_TEAM_ID, query_type=query_type)- get_query_tags().with_dagster(dagster_tags(context))---def cohort_users(organization_id: Any) -> dict[str, int]:- """The active members of the organization with a cohort email, keyed by their distinct id."""- return {- str(distinct_id): user_id- for distinct_id, user_id in OrganizationMembership.objects.filter(- organization_id=organization_id,- user__is_active=True,- user__email__iendswith=COHORT_EMAIL_DOMAIN,- user__distinct_id__isnull=False,- )- .exclude(user__distinct_id="")- .values_list("user__distinct_id", "user_id")- }---def organization_projects(organization_id: Any) -> dict[str, int]:- """The organization's teams, keyed by the uuid that the events carry as their project group key."""- return {- str(team_uuid): team_id- for team_id, team_uuid in Team.objects.filter(organization_id=organization_id).values_list("id", "uuid")- }---@frozen(order=True)-class PairKey:- """A (project, person) pair as the events name it: the team uuid and the distinct id."""-- project: str- distinct_id: str---@frozen(order=True)-class MemberProject:- """A (project, person) pair as Postgres names it: the team id and the user id."""-- team_id: int- user_id: int---def mcp_sessions_by_pair(rows: list[tuple[Any, ...]]) -> dict[PairKey, list[McpSession]]:- sessions: dict[PairKey, list[McpSession]] = defaultdict(list)- for distinct_id, project, _session_key, timestamps, tools, intents in rows:- calls = tuple(- McpCall(at=ensure_utc(at) or at, tool=str(tool), intent=str(intent))- for at, tool, intent in zip(timestamps, tools, intents, strict=True)- )- sessions[PairKey(project=str(project), distinct_id=str(distinct_id))].append(McpSession(calls=calls))- return sessions---def member_tasks(- user_ids: Collection[int], team_ids: Collection[int], window: Window-) -> dict[MemberProject, list[TaskItem]]:- """Each member's own tasks, newest `MAX_TASKS_PER_PERSON` per member, keyed by project."""- tasks: dict[MemberProject, list[TaskItem]] = defaultdict(list)- for user_id in user_ids:- for task in list_user_task_summaries(- team_ids=team_ids,- user_id=user_id,- start=window.start,- end=window.end,- origin_products=TASK_ORIGIN_PRODUCTS,- limit=MAX_TASKS_PER_PERSON,- ):- tasks[MemberProject(team_id=task.team_id, user_id=user_id)].append(- TaskItem(- at=ensure_utc(task.created_at) or task.created_at,- title=task.title,- summary=task.summary,- tags=task.tags,- )- )- return tasks---def member_activity(- user_ids: Collection[int], team_ids: Collection[int], window: Window-) -> dict[MemberProject, list[ActivityItem]]:- """Each member's own activity log rows, newest `MAX_ACTIVITY_ROWS_PER_PERSON` per member.-- System rows and rows written while someone impersonated the member are not the member's."""- activity: dict[MemberProject, list[ActivityItem]] = defaultdict(list)- for user_id in user_ids:- rows = (- ActivityLog.objects.filter(- team_id__in=list(team_ids),- user_id=user_id,- created_at__gte=window.start,- created_at__lt=window.end,- )- .exclude(is_system=True)- .exclude(was_impersonated=True)- .order_by("-created_at")- .annotate(name=KeyTextTransform("name", "detail"))- .values("team_id", "scope", "activity", "client", "created_at", "name")[:MAX_ACTIVITY_ROWS_PER_PERSON]- )- for row in rows:- activity[MemberProject(team_id=row["team_id"], user_id=user_id)].append(- ActivityItem(- at=ensure_utc(row["created_at"]) or row["created_at"],- scope=row["scope"],- activity=row["activity"],- name=row["name"],- client=row["client"],- )- )- return activity---def input_rows(- *,- sessions: dict[PairKey, list[McpSession]],- tasks: dict[MemberProject, list[TaskItem]],- activity: dict[MemberProject, list[ActivityItem]],- users: dict[str, int],- projects: dict[str, int],- window: Window,- partition_key: str,-) -> list[dict[str, Any]]:- """One row per (project, member) with anything left after the filters."""- sessions_by_person: dict[str, list[McpSession]] = defaultdict(list)- for pair, person_sessions in sessions.items():- sessions_by_person[pair.distinct_id].extend(person_sessions)- jobs = {distinct_id: job_intents(person_sessions) for distinct_id, person_sessions in sessions_by_person.items()}-- member_sessions: dict[MemberProject, list[McpSession]] = defaultdict(list)- member_jobs: dict[int, frozenset[tuple[datetime.date, str]]] = {}- for pair, pair_sessions in sessions.items():- user_id = users.get(pair.distinct_id)- team_id = projects.get(pair.project)- if user_id is None or team_id is None:- continue- member_sessions[MemberProject(team_id=team_id, user_id=user_id)].extend(pair_sessions)- member_jobs[user_id] = jobs.get(pair.distinct_id, frozenset())-- allowed_teams = set(projects.values())- rows: list[dict[str, Any]] = []- for member in sorted(member_sessions.keys() | tasks.keys() | activity.keys()):- if member.team_id not in allowed_teams:- continue- rendered = build_input(- member_sessions.get(member, []),- tasks.get(member, []),- activity.get(member, []),- jobs=member_jobs.get(member.user_id, frozenset()),- as_of=window.end,- seed=sampling_seed(partition_key, member.team_id, member.user_id),- )- if rendered is None:- continue- rows.append(- {- "team_id": member.team_id,- "user_id": member.user_id,- "window_start": window.start,- "window_end": window.end,- "n_tasks": rendered.n_tasks,- "n_mcp_sessions": rendered.n_mcp_sessions,- "n_activity_lines": rendered.n_activity_lines,- "rendered_input": rendered.text,- "task_chars": rendered.task_chars,- "mcp_chars": rendered.mcp_chars,- "activity_chars": rendered.activity_chars,- "input_chars": rendered.input_chars,- "input_tokens_estimate": rendered.input_tokens_estimate,- "drops": json.dumps(rendered.drops, sort_keys=True),- "render_version": RENDER_VERSION,- }- )- return rows--[email protected](name=INPUTS_TABLE, **ASSET_KWARGS)-def activity_summary_inputs(context: dagster.AssetExecutionContext) -> None:- if skip_unconfigured(context):- return- partition_key = context.partition_key- window = activity_window(partition_key)- team = labels_team()- users = cohort_users(team.organization_id)- projects = organization_projects(team.organization_id)-- _tag_queries(context, "activity_summary_inputs")- session_rows = queries.input_rows(- queries.INPUT_MCP_SESSIONS_SQL,- query_type="activity_summary_input_mcp_sessions",- team=team,- window_start=window.start,- window_end=window.end,- distinct_ids=users.keys(),- projects=projects.keys(),- )- tasks = member_tasks(users.values(), projects.values(), window)- activity = member_activity(users.values(), projects.values(), window)-- rows = input_rows(- sessions=mcp_sessions_by_pair(session_rows),- tasks=tasks,- activity=activity,- users=users,- projects=projects,- window=window,- partition_key=partition_key,- )- bucket = dataset_bucket()- key = partition_object_key(settings.INBOX_RANKING_DATASET_S3_PREFIX, INPUTS_TABLE, partition_key)- write_parquet(s3_client(), bucket, key, pa.Table.from_pylist(rows, schema=INPUTS_SCHEMA))- context.add_output_metadata(- {- "rows": dagster.MetadataValue.int(len(rows)),- "cohort_users": dagster.MetadataValue.int(len(users)),- "mcp_session_rows": dagster.MetadataValue.int(len(session_rows)),- "task_pairs": dagster.MetadataValue.int(len(tasks)),- "activity_pairs": dagster.MetadataValue.int(len(activity)),- "s3_key": dagster.MetadataValue.text(f"s3://{bucket}/{key}"),- }- )---def pair_activities(mcp_rows: list[tuple[Any, ...]]) -> dict[PairKey, PairActivity]:- """The projection query's rows keyed by pair. The key never leaves the run."""- return {- PairKey(project=str(project), distinct_id=str(distinct_id)): PairActivity(- mcp_sessions=int(sessions), mcp_avg_line_chars=float(chars or 0)- )- for distinct_id, project, sessions, chars in mcp_rows- }---def cohort_calibration(- activities: dict[PairKey, PairActivity], inputs: pa.Table | None, projects: dict[str, int]-) -> float:- """The rendered-to-estimated MCP section length ratio over the cohort pairs both assets cover."""- if inputs is None or inputs.num_rows == 0:- return 1.0- distinct_ids = {- user_id: str(distinct_id)- for user_id, distinct_id in User.objects.filter(id__in=set(inputs.column("user_id").to_pylist()))- .exclude(distinct_id__isnull=True)- .values_list("id", "distinct_id")- }- team_uuids = {team_id: team_uuid for team_uuid, team_id in projects.items()}- actual: dict[PairKey, int] = {}- estimated: dict[PairKey, int] = {}- for row in inputs.select(["team_id", "user_id", "mcp_chars"]).to_pylist():- distinct_id = distinct_ids.get(row["user_id"])- team_uuid = team_uuids.get(row["team_id"])- if distinct_id is None or team_uuid is None or not row["mcp_chars"]:- continue- pair = PairKey(project=team_uuid, distinct_id=distinct_id)- if pair not in activities:- continue- actual[pair] = row["mcp_chars"]- estimated[pair] = estimated_input_chars(activities[pair])- return calibration_ratio(actual, estimated)--[email protected](name=COST_PROJECTION_TABLE, deps=[INPUTS_TABLE], **ASSET_KWARGS)-def activity_summary_cost_projection(context: dagster.AssetExecutionContext, config: CostProjectionConfig) -> None:- if skip_unconfigured(context):- return- partition_key = context.partition_key- window = activity_window(partition_key)- team = labels_team()-- _tag_queries(context, "activity_summary_cost_projection")- mcp_rows = queries.projection_query_rows(- queries.PROJECTION_MCP_SQL,- query_type="activity_summary_projection_mcp",- team=team,- window_start=window.start,- window_end=window.end,- )- activities = pair_activities(mcp_rows)-- client = s3_client()- bucket = dataset_bucket()- prefix = settings.INBOX_RANKING_DATASET_S3_PREFIX- inputs = read_parquet_if_exists(- client, bucket, partition_object_key(prefix, INPUTS_TABLE, partition_key), ["team_id", "user_id", "mcp_chars"]- )- ratio = cohort_calibration(activities, inputs, organization_projects(team.organization_id))-- rows = [- {"window_start": window.start, "window_end": window.end, **row}- for row in projection_rows(- list(activities.values()),- ratio=ratio,- prices=ProjectionPrices(- prompt_overhead_tokens=config.prompt_overhead_tokens + config.capped_sections_tokens,- output_tokens_per_update=config.output_tokens_per_update,- input_price_per_million_tokens=config.input_price_per_million_tokens,- output_price_per_million_tokens=config.output_price_per_million_tokens,- ),- )- ]- key = partition_object_key(prefix, COST_PROJECTION_TABLE, partition_key)- write_parquet(client, bucket, key, pa.Table.from_pylist(rows, schema=COST_PROJECTION_SCHEMA))- context.add_output_metadata(- {- "pairs": dagster.MetadataValue.int(len(activities)),- "calibration_ratio": dagster.MetadataValue.float(ratio),- "calibrated_on_inputs": dagster.MetadataValue.bool(inputs is not None and inputs.num_rows > 0),- "s3_key": dagster.MetadataValue.text(f"s3://{bucket}/{key}"),- }- )---def summary_partitions_before(client, bucket: str, prefix: str, partition_key: str) -> list[str]:- """The written summaries partitions in the `PREVIOUS_SUMMARY_MAX_AGE_DAYS` before `partition_key`,- newest first."""- folder = f"{prefix}/{SUMMARIES_TABLE}/{DATASET_VERSION}/dt="- earliest = (- datetime.date.fromisoformat(partition_key) - datetime.timedelta(days=PREVIOUS_SUMMARY_MAX_AGE_DAYS)- ).isoformat()- partitions: set[str] = set()- for page in client.get_paginator("list_objects_v2").paginate(Bucket=bucket, Prefix=folder):- for item in page.get("Contents", []):- partition, _, name = item["Key"].removeprefix(folder).partition("/")- if name == PARQUET_PART_NAME and earliest <= partition < partition_key:- partitions.add(partition)- return sorted(partitions, reverse=True)---def usable_summaries(rows: list[dict[str, Any]], wanted: Collection[MemberProject]) -> dict[MemberProject, dict]:- """The successful summaries among one partition's rows, for the pairs in `wanted`."""- return {- member: json.loads(row["summary_json"])- for row in rows- if row["status"] == STATUS_OK- and row["summary_json"]- and (member := MemberProject(team_id=row["team_id"], user_id=row["user_id"])) in wanted- }---def previous_summaries(- client, bucket: str, prefix: str, partition_key: str, pairs: Collection[MemberProject]-) -> dict[MemberProject, dict]:- """Each pair's summary from the newest earlier partition that holds one for it. Only earlier- partitions are read, so a re-run of a partition sees the same previous summaries."""- wanted = set(pairs)- found: dict[MemberProject, dict] = {}- for partition in summary_partitions_before(client, bucket, prefix, partition_key):- if not wanted:- break- table = read_parquet_if_exists(- client,- bucket,- partition_object_key(prefix, SUMMARIES_TABLE, partition),- ["team_id", "user_id", "summary_json", "status"],- )- if table is None:- continue- newer = usable_summaries(table.to_pylist(), wanted)- found |= newer- wanted -= newer.keys()- return found---def keep_earlier_successes(- rows: list[dict[str, Any]], existing: list[dict[str, Any]]-) -> tuple[list[dict[str, Any]], int]:- """A re-run's rows, with each failed row replaced by the pair's successful row from the earlier- write of the partition, so a transient failure does not destroy a summary. Returns the rows and- how many were kept from the earlier write."""- earlier = {- MemberProject(team_id=row["team_id"], user_id=row["user_id"]): row- for row in existing- if row["status"] == STATUS_OK- }- merged: list[dict[str, Any]] = []- kept = 0- for row in rows:- previous = earlier.get(MemberProject(team_id=row["team_id"], user_id=row["user_id"]))- if row["status"] != STATUS_OK and previous is not None:- merged.append(previous)- kept += 1- else:- merged.append(row)- return merged, kept--[email protected](name=SUMMARIES_TABLE, deps=[INPUTS_TABLE], **ASSET_KWARGS)-def activity_summaries(context: dagster.AssetExecutionContext, config: SummariesConfig) -> None:- """One LLM summary per `activity_summary_inputs` row, with its tokens, latency and cost.-- A row whose answer stays malformed after one retry, or whose request fails, is written with- `status = failed`, unless an earlier write of the partition holds a successful row for that pair.- When every row fails, the asset fails and leaves the partition as it was."""- if skip_unconfigured(context):- return- partition_key = context.partition_key- client = s3_client()- bucket = dataset_bucket()- prefix = settings.INBOX_RANKING_DATASET_S3_PREFIX- inputs = read_parquet_if_exists(- client,- bucket,- partition_object_key(prefix, INPUTS_TABLE, partition_key),- ["team_id", "user_id", "rendered_input", "n_tasks", "n_mcp_sessions", "n_activity_lines", "drops"],- )- if inputs is None:- context.log.warning(f"{INPUTS_TABLE} has no object for dt={partition_key}; nothing to summarize")- return- input_rows_ = inputs.to_pylist()- members = [MemberProject(team_id=row["team_id"], user_id=row["user_id"]) for row in input_rows_]- previous = previous_summaries(client, bucket, prefix, partition_key, members) if config.use_previous_summary else {}- requests = [- SummaryRequest(- team_id=member.team_id,- user_id=member.user_id,- rendered_input=row["rendered_input"],- previous_summary=previous.get(member),- )- for member, row in zip(members, input_rows_, strict=True)- ]-- llm = gateway_client(partition_key)- as_of = datetime.date.fromisoformat(partition_key)- outcomes = summarize_all(- requests, lambda request: summarize(llm, request, model=config.model, as_of=as_of), config.max_concurrency- )- prices = SummaryPrices(- input_price_per_million_tokens=config.input_price_per_million_tokens,- output_price_per_million_tokens=config.output_price_per_million_tokens,- )- rows = [- summary_row(request, outcome, model=config.model, prices=prices)- for request, outcome in zip(requests, outcomes, strict=True)- ]- failed = sum(1 for row in rows if row["status"] != STATUS_OK)- if rows and failed == len(rows):- raise dagster.Failure(- f"Every summary failed for dt={partition_key}, so the partition was not written",- metadata={"rows": len(rows), "first_error": str(rows[0]["error"])},- )-- key = partition_object_key(prefix, SUMMARIES_TABLE, partition_key)- existing = read_parquet_if_exists(client, bucket, key)- written_rows, kept = keep_earlier_successes(rows, [] if existing is None else existing.to_pylist())- write_parquet(client, bucket, key, pa.Table.from_pylist(written_rows, schema=SUMMARIES_SCHEMA))- capture_training_events(- context,- partition_key,- summary_events(rows, input_rows_, capture_summary_text=config.capture_summary_text),- distinct_id=PIPELINE_DISTINCT_ID,- )- costs = [row["cost_usd"] for row in rows if row["cost_usd"] is not None]- context.add_output_metadata(- {- "rows": dagster.MetadataValue.int(len(rows)),- "failed": dagster.MetadataValue.int(failed),- "kept_from_earlier_run": dagster.MetadataValue.int(kept),- "used_previous_summary": dagster.MetadataValue.int(sum(1 for row in rows if row["used_previous_summary"])),- "tokens_in": dagster.MetadataValue.int(sum(row["tokens_in"] for row in rows)),- "tokens_out": dagster.MetadataValue.int(sum(row["tokens_out"] for row in rows)),- "cost_usd": dagster.MetadataValue.float(sum(costs)) if costs else dagster.MetadataValue.null(),- "model": dagster.MetadataValue.text(config.model),- "prompt_version": dagster.MetadataValue.int(PROMPT_VERSION),- "s3_key": dagster.MetadataValue.text(f"s3://{bucket}/{key}"),- }- )---@frozen-class EmbeddingTarget:- user_id: int- prompt_version: int- text: str---def embedding_targets(rows: list[dict[str, Any]], partition_key: str) -> dict[SummaryDocument, EmbeddingTarget]:- """The successful summaries of a partition, keyed by their embedding document."""- return {- SummaryDocument(- team_id=row["team_id"], document_id=document_id(row["user_id"], partition_key)- ): EmbeddingTarget(user_id=row["user_id"], prompt_version=row["prompt_version"], text=row["summary_text"])- for row in rows- if row["status"] == STATUS_OK and row["summary_text"]- }---def summary_embedding_rows(- targets: dict[SummaryDocument, EmbeddingTarget], landed: dict[SummaryDocument, LandedVector]-) -> list[dict[str, Any]]:- """One row per summary whose vector landed and embeds the summary's current text."""- rows: list[dict[str, Any]] = []- for document, target in sorted(targets.items()):- vector = landed.get(document)- if vector is None or vector.content != target.text:- continue- rows.append(- {- "team_id": document.team_id,- "user_id": target.user_id,- "prompt_version": target.prompt_version,- "embedding": vector.embedding,- "embedding_model": EMBEDDING_MODEL.value,- }- )- return rows--- name=SUMMARY_EMBEDDINGS_TABLE,- # Reads the summaries partition, and also keeps the vectors out of the LLM step's run pod memory.- deps=[SUMMARIES_TABLE],- **ASSET_KWARGS,-)-def activity_summary_embeddings(context: dagster.AssetExecutionContext, config: SummaryEmbeddingsConfig) -> None:- """The vector of each successful summary, copied out of `document_embeddings`.-- The source table drops a row three months after its timestamp, which is the partition day, so- this Parquet is the history a backtest reads. Only a summary with no vector yet, or a vector of- other text, is sent to the worker, so a re-run does not embed the same text twice. A re-run that- would write fewer rows than the partition holds is refused: past the source TTL, the vectors it- lacks cannot come back."""- if skip_unconfigured(context):- return- partition_key = context.partition_key- if not within_source_ttl(partition_key, datetime.datetime.now(datetime.UTC).date()):- context.log.warning(- f"dt={partition_key} is too old: its vectors would expire from document_embeddings before they "- "could be read back; nothing to embed"- )- return- client = s3_client()- bucket = dataset_bucket()- prefix = settings.INBOX_RANKING_DATASET_S3_PREFIX- summaries = read_parquet_if_exists(- client,- bucket,- partition_object_key(prefix, SUMMARIES_TABLE, partition_key),- ["team_id", "user_id", "prompt_version", "summary_text", "status"],- )- if summaries is None:- context.log.warning(f"{SUMMARIES_TABLE} has no object for dt={partition_key}; nothing to embed")- return- targets = embedding_targets(summaries.to_pylist(), partition_key)- wanted = {document: target.text for document, target in targets.items()}- day_start, _ = snapshot_bounds(partition_key)-- landed = landed_vectors(wanted.keys(), day_start)- pending = pending_documents(wanted, landed)- emitted = len(pending)- if pending:- for document in pending:- target = targets[document]- emit_summary_embedding(- team_id=document.team_id,- user_id=target.user_id,- partition_key=partition_key,- prompt_version=target.prompt_version,- text=target.text,- day_start=day_start,- )- flush_embedding_requests()- deadline = time.monotonic() + config.wait_seconds- while pending and time.monotonic() < deadline:- time.sleep(config.poll_interval_seconds)- landed |= landed_vectors(pending, day_start)- pending = pending_documents({document: wanted[document] for document in pending}, landed)-- rows = summary_embedding_rows(targets, landed)- key = partition_object_key(prefix, SUMMARY_EMBEDDINGS_TABLE, partition_key)- existing_rows = object_row_count(client, bucket, key)- written = partition_write_allowed(existing_rows, len(rows))- if written:- write_parquet(client, bucket, key, pa.Table.from_pylist(rows, schema=SUMMARY_EMBEDDINGS_SCHEMA))- else:- context.log.warning(- f"dt={partition_key} already holds {existing_rows} vectors and this run has {len(rows)}; "- "the partition was not overwritten"- )- context.add_output_metadata(- {- "summaries": dagster.MetadataValue.int(summaries.num_rows),- "skipped_failed_summaries": dagster.MetadataValue.int(summaries.num_rows - len(targets)),- "emitted": dagster.MetadataValue.int(emitted),- "missing_vectors": dagster.MetadataValue.int(len(pending)),- "rows": dagster.MetadataValue.int(len(rows)),- "written": dagster.MetadataValue.bool(written),- "s3_key": dagster.MetadataValue.text(f"s3://{bucket}/{key}"),- }- )---activity_summary_job = dagster.define_asset_job(- name="activity_summary_job",- selection=[INPUTS_TABLE, COST_PROJECTION_TABLE, SUMMARIES_TABLE, SUMMARY_EMBEDDINGS_TABLE],- tags={**owner_tags, "dagster/max_runtime": str(60 * 60)},-)---# Stopped everywhere until someone turns it on: the summaries are an experiment, run by hand for now.- cron_schedule="30 3 * * 1",- job=activity_summary_job,- execution_timezone="UTC",- default_status=dagster.DefaultScheduleStatus.STOPPED,- tags=owner_tags,-)-def activity_summary_schedule(context: dagster.ScheduleEvaluationContext) -> dagster.RunRequest | dagster.SkipReason:- if dataset_unconfigured():- return dagster.SkipReason(f"{S3_BUCKET_ENV} is not set; skipping until the dedicated bucket is provisioned")- previous_day = context.scheduled_execution_time.date() - datetime.timedelta(days=1)- return dagster.RunRequest(partition_key=previous_day.isoformat(), run_key=previous_day.isoformat())diff --git a/products/signals/dags/inbox_ranking/activity_summary/embeddings.py b/products/signals/dags/inbox_ranking/activity_summary/embeddings.pydeleted file mode 100644index c22fa9600445..000000000000--- a/products/signals/dags/inbox_ranking/activity_summary/embeddings.py+++ /dev/null@@ -1,153 +0,0 @@-"""The embedding round trip behind `activity_summary_embeddings`.--Each summary goes through the embedding worker, the path the report embeddings take, so a summary-vector shares its model and space with the report vectors. The worker writes to-`document_embeddings` asynchronously, so the asset emits, waits, and reads back what landed.-"""--import datetime-from collections.abc import Collection-from typing import Any, cast--from posthog.api.embedding_worker import emit_embedding_request-from posthog.clickhouse.client import sync_execute-from posthog.clickhouse.client.connection import ClickHouseUser-from posthog.dataclasses import frozen-from posthog.kafka_client.routing import get_producer-from posthog.kafka_client.topics import KAFKA_DOCUMENT_EMBEDDINGS_INPUT_TOPIC-from posthog.schema_enums import EmbeddingModelName--from products.signals.backend.report_embeddings import EMBEDDING_PRODUCT-from products.signals.dags.inbox_ranking.dataset.queries import EMBEDDINGS_TABLE, etl_workload--EMBEDDING_DOCUMENT_TYPE = "activity_summary"-EMBEDDING_RENDERING = "summary_text"-# The model `EMBEDDINGS_TABLE` holds. The worker also writes the other models, like the report rows.-EMBEDDING_MODEL = EmbeddingModelName.TEXT_EMBEDDING_3_SMALL_1536-KAFKA_FLUSH_TIMEOUT_SECONDS = 60.0-# `document_embeddings` drops a row three months after its timestamp, which is the partition day. Three-# months is never less than 89 days, and the margin leaves room for the worker and the read-back.-SOURCE_TTL_DAYS = 89-SOURCE_TTL_MARGIN_DAYS = 7--# `timestamp` is in the table's sort key as `toDate(timestamp)`, and every row of a partition carries-# the partition day, so the day bound prunes the read to one day of the table.-SUMMARY_EMBEDDINGS_SQL = f"""-SELECT- team_id,- document_id,- argMax(embedding, inserted_at) AS embedding,- argMax(content, inserted_at) AS content-FROM {EMBEDDINGS_TABLE}-WHERE product = %(product)s- AND document_type = %(document_type)s- AND rendering = %(rendering)s- AND team_id IN %(team_ids)s- AND document_id IN %(document_ids)s- AND timestamp >= %(day_start)s- AND timestamp < %(day_end)s-GROUP BY team_id, document_id-"""-SUMMARY_EMBEDDINGS_QUERY_SETTINGS: dict[str, int] = {"max_execution_time": 120}---@frozen(order=True)-class SummaryDocument:- team_id: int- document_id: str---@frozen-class LandedVector:- embedding: list[float]- content: str---def within_source_ttl(partition_key: str, today: datetime.date) -> bool:- """Whether a vector stamped with this partition day still lives in `document_embeddings` long- enough to be read back. An older backfill would lose its vectors and write a partial partition."""- age_days = (today - datetime.date.fromisoformat(partition_key)).days- return age_days <= SOURCE_TTL_DAYS - SOURCE_TTL_MARGIN_DAYS---def document_id(user_id: int, partition_key: str) -> str:- """One document per (user, partition). The team is part of the table's key on its own."""- return f"{user_id}:{partition_key}"---def emit_summary_embedding(- *, team_id: int, user_id: int, partition_key: str, prompt_version: int, text: str, day_start: datetime.datetime-) -> None:- """Queue one summary for the worker.-- `timestamp` is the partition day, not the emission time: the table replaces on a key that holds- `toDate(timestamp)`, so a re-run supersedes its own row instead of adding a second one."""- emit_embedding_request(- content=text,- team_id=team_id,- product=EMBEDDING_PRODUCT,- document_type=EMBEDDING_DOCUMENT_TYPE,- rendering=EMBEDDING_RENDERING,- document_id=document_id(user_id, partition_key),- models=[model.value for model in EmbeddingModelName],- timestamp=day_start,- metadata={- "team_id": team_id,- "user_id": user_id,- "partition": partition_key,- "prompt_version": prompt_version,- },- )---def flush_embedding_requests() -> None:- """Deliver every queued request before the asset waits for the vectors, so a broken producer- fails the run instead of looking like a slow worker."""- undelivered = get_producer(topic=KAFKA_DOCUMENT_EMBEDDINGS_INPUT_TOPIC).flush(KAFKA_FLUSH_TIMEOUT_SECONDS)- if undelivered:- raise RuntimeError(f"{undelivered} embedding requests were not delivered to Kafka")---def landed_vectors(- documents: Collection[SummaryDocument], day_start: datetime.datetime-) -> dict[SummaryDocument, LandedVector]:- """The latest vector and the text it embeds for each document that has one."""- if not documents:- return {}- rows = cast(- list[tuple[Any, ...]],- sync_execute(- SUMMARY_EMBEDDINGS_SQL,- {- "product": EMBEDDING_PRODUCT,- "document_type": EMBEDDING_DOCUMENT_TYPE,- "rendering": EMBEDDING_RENDERING,- "team_ids": sorted({document.team_id for document in documents}),- "document_ids": sorted({document.document_id for document in documents}),- "day_start": day_start.replace(tzinfo=None),- "day_end": (day_start + datetime.timedelta(days=1)).replace(tzinfo=None),- },- settings=SUMMARY_EMBEDDINGS_QUERY_SETTINGS,- workload=etl_workload(),- ch_user=ClickHouseUser.SIGNALS,- )- or [],- )- return {- SummaryDocument(team_id=int(team_id), document_id=str(doc_id)): LandedVector(- embedding=list(embedding), content=str(content)- )- for team_id, doc_id, embedding, content in rows- }---def pending_documents(- wanted: dict[SummaryDocument, str], landed: dict[SummaryDocument, LandedVector]-) -> list[SummaryDocument]:- """The documents with no vector yet, or with a vector of other text, such as one written before- the summaries partition was re-run."""- return sorted(- document- for document, text in wanted.items()- if (vector := landed.get(document)) is None or vector.content != text- )diff --git a/products/signals/dags/inbox_ranking/activity_summary/prompt.py b/products/signals/dags/inbox_ranking/activity_summary/prompt.pydeleted file mode 100644index a38baff65940..000000000000--- a/products/signals/dags/inbox_ranking/activity_summary/prompt.py+++ /dev/null@@ -1,166 +0,0 @@-"""The summary prompt, the output contract and the normalization of the model's answer.--`summary_text` is the text the embeddings asset embeds, so its format is part of the contract with-every vector already written. Bump `PROMPT_VERSION` when the prompt, the normalization or-`summary_text` changes.-"""--import json-import datetime-from typing import Any--PROMPT_VERSION = 1--SUMMARY_MAX_WORDS = 60-MAX_FOCUS_AREAS = 6-MAX_PRODUCTS = 10-STRENGTHS = ("high", "medium", "low")-DEFAULT_STRENGTH = "low"-# Strings a model writes in place of a JSON null.-_NULL_STRINGS = frozenset({"", "none", "null", "n/a", "na", "nil"})--SYSTEM_PROMPT = """You maintain a short summary of what one person has been working on in one PostHog project.-You get their previous summary, if any, and a sample of their recent activity in up to three sections: tasks they gave to PostHog AI or the Slack agent, sessions of their own coding agent over MCP, and things they created or changed in PostHog.-Update the summary to reflect what they are focused on now. Keep what is still true, drop what has gone quiet, and add what is new. Weigh themes that repeat across sections above one-off items. Treat bulk changes of the same kind as one theme, not a list.-Name concrete products, features, metrics, pipelines and workflows. Never write generic phrases like "data analysis".-Write about the work, not the person: no names, emails, IDs, secrets, or quotes from the input. Open with the work itself.-Return JSON: `summary` (at most 60 words), `focus_areas` (up to 6, each with `topic`, `strength` high, medium or low, and `last_seen` as YYYY-MM-DD), `products`, and `recent_shift` (one sentence, or null).-Set `recent_shift` only when there is a previous summary and the activity shows a real change from it. Otherwise it is null."""--# Strict structured output. The normalization below still checks every rule, because a provider-# behind the gateway may not enforce the schema.-RESPONSE_FORMAT: Any = {- "type": "json_schema",- "json_schema": {- "name": "activity_summary",- "strict": True,- "schema": {- "type": "object",- "properties": {- "summary": {"type": "string"},- "focus_areas": {- "type": "array",- "items": {- "type": "object",- "properties": {- "topic": {"type": "string"},- "strength": {"type": "string", "enum": list(STRENGTHS)},- "last_seen": {"type": "string"},- },- "required": ["topic", "strength", "last_seen"],- "additionalProperties": False,- },- },- "products": {"type": "array", "items": {"type": "string"}},- "recent_shift": {"type": ["string", "null"]},- },- "required": ["summary", "focus_areas", "products", "recent_shift"],- "additionalProperties": False,- },- },-}---class MalformedSummaryError(ValueError):- """The model's answer cannot be turned into a summary. The message never quotes the answer."""---def user_message(rendered_input: str, previous_summary: dict[str, Any] | None, as_of: datetime.date) -> str:- """The user turn: the day the activity ends, the previous summary and the rendered input.-- The rendered input comes from `activity_summary_inputs` as is, and the previous summary is this- asset's own output, so no other text reaches the model."""- previous = (- json.dumps(- {key: previous_summary.get(key) for key in ("summary", "focus_areas", "products")},- ensure_ascii=False,- sort_keys=True,- )- if previous_summary- else "None"- )- return f"Today is {as_of.isoformat()}.\n\nPrevious summary:\n{previous}\n\nRecent activity:\n{rendered_input}"---def _clean(value: Any) -> str | None:- if not isinstance(value, str):- return None- text = " ".join(value.split())- return None if text.lower() in _NULL_STRINGS else text---def cap_words(text: str, max_words: int = SUMMARY_MAX_WORDS) -> str:- words = text.split()- return text if len(words) <= max_words else " ".join(words[:max_words])---def _last_seen(value: Any) -> str | None:- text = _clean(value)- if text is None:- return None- try:- return datetime.date.fromisoformat(text).isoformat()- except ValueError:- return None---def _focus_areas(value: Any) -> list[dict[str, str | None]]:- if not isinstance(value, list):- return []- areas: list[dict[str, str | None]] = []- for item in value:- if not isinstance(item, dict) or (topic := _clean(item.get("topic"))) is None:- continue- strength = (_clean(item.get("strength")) or "").lower()- areas.append(- {- "topic": topic,- "strength": strength if strength in STRENGTHS else DEFAULT_STRENGTH,- "last_seen": _last_seen(item.get("last_seen")),- }- )- if len(areas) == MAX_FOCUS_AREAS:- break- return areas---def _products(value: Any) -> list[str]:- if not isinstance(value, list):- return []- products: list[str] = []- for item in value:- product = _clean(item)- if product is not None and product not in products:- products.append(product)- return products[:MAX_PRODUCTS]---def normalize_summary(content: str | None, *, had_previous_summary: bool) -> dict[str, Any]:- """The model's JSON answer under the output rules, or `MalformedSummaryError`.-- Only a missing or empty `summary` makes an answer malformed. Every other field degrades to an- empty or null value, so one bad focus area does not cost the row."""- if not content:- raise MalformedSummaryError("empty response")- try:- parsed = json.loads(content)- except json.JSONDecodeError as error:- raise MalformedSummaryError("response is not JSON") from error- if not isinstance(parsed, dict):- raise MalformedSummaryError("response is not a JSON object")- summary = _clean(parsed.get("summary"))- if summary is None:- raise MalformedSummaryError("response has no summary")- recent_shift = _clean(parsed.get("recent_shift")) if had_previous_summary else None- return {- "summary": cap_words(summary),- "focus_areas": _focus_areas(parsed.get("focus_areas")),- "products": _products(parsed.get("products")),- "recent_shift": recent_shift,- }---def summary_text(summary: dict[str, Any]) -> str:- """The text that gets embedded: the summary, then the focus area topics in the model's order."""- topics = [area["topic"] for area in summary.get("focus_areas", [])]- return f"{summary['summary']}\nFocus areas: {'; '.join(topics)}" if topics else summary["summary"]diff --git a/products/signals/dags/inbox_ranking/activity_summary/queries.py b/products/signals/dags/inbox_ranking/activity_summary/queries.pydeleted file mode 100644index f423b9bab35d..000000000000--- a/products/signals/dags/inbox_ranking/activity_summary/queries.py+++ /dev/null@@ -1,204 +0,0 @@-"""HogQL for the activity summary assets. Both read the dogfood project's events (`LABELS_TEAM_ID`).--`$group_2` is the project group, keyed by the team's uuid.--The inputs queries return free text for the internal cohort only. The projection queries cover every-(project, person) pair and return counts and lengths, so no free text leaves ClickHouse for them.-"""--import datetime-from collections.abc import Collection-from typing import Any--from posthog.hogql import ast-from posthog.hogql.constants import HogQLGlobalSettings, LimitContext-from posthog.hogql.query import execute_hogql_query--from posthog.clickhouse.client.connection import ClickHouseUser-from posthog.models import Team--from products.signals.dags.inbox_ranking.activity_summary.render import (- MAX_MCP_SESSIONS_PER_PERSON,- MAX_UNITS_PER_DAY,- MCP_LINE_PREFIX_CHARS,- MCP_STEP_CHARS,- MIN_WORK_CALLS,- PLUMBING_TOOL_NAMES,- UPKEEP_TOOL_PREFIXES,-)-from products.signals.dags.inbox_ranking.dataset.queries import etl_workload, utc_bound--# Each kept MCP source, with the scope presets that mark a person's own calls ("" is no preset).-# Scouts, research and implementation runs carry other presets and are not a person's intent. Slack-# and PostHog AI calls run inside tasks, and the inputs read those tasks' own summaries instead.-MCP_SOURCE_SCOPE_PRESETS: dict[str, tuple[str, ...]] = {- "mcp": ("user", ""),-}--# The wrapper row of the single `exec` tool. The real tool call it runs is captured as its own row.-MCP_WRAPPER_TOOL_NAME = "exec"--_MCP_FILTERS = f"""- event = '$mcp_tool_call'- AND timestamp >= toDateTime({{window_start}}) AND timestamp < toDateTime({{window_end}})- AND concat(ifNull(toString(properties.source), ''), ':', ifNull(toString(properties.$mcp_scope_preset), '')) IN {{kept_scopes}}- AND ifNull(toString(properties.$mcp_tool_name), '') != '{MCP_WRAPPER_TOOL_NAME}'- AND nullIf(toString(properties.$mcp_intent), '') IS NOT NULL- AND nullIf($group_2, '') IS NOT NULL-"""--_SESSION_KEY = """- coalesce(- nullIf(toString(properties.$mcp_conversation_id), ''),- nullIf(toString(properties.$mcp_session_id), ''),- nullIf(toString(properties.$session_id), ''),- concat(distinct_id, '-', toString(toDate(toTimeZone(timestamp, 'UTC'))))- )-"""--# One row per session with its calls as parallel arrays. groupArray walks the rows in one order for-# every array, so index i of each array is the same call. render.py sorts and filters them. Only the-# newest sessions of each person come back, so a very active person cannot make the read unbounded.-INPUT_MCP_SESSIONS_SQL = f"""-SELECT- distinct_id,- $group_2 AS project,- {_SESSION_KEY} AS session_key,- groupArray(toTimeZone(timestamp, 'UTC')) AS timestamps,- groupArray(ifNull(toString(properties.$mcp_tool_name), '')) AS tools,- groupArray(toString(properties.$mcp_intent)) AS intents-FROM events-WHERE {_MCP_FILTERS}- AND distinct_id IN {{distinct_ids}}- AND $group_2 IN {{projects}}-GROUP BY distinct_id, project, session_key-ORDER BY max(timestamp) DESC-LIMIT {MAX_MCP_SESSIONS_PER_PERSON} BY distinct_id-"""--_IS_WORK_CALL = """- (ifNull(toString(properties.$mcp_tool_name), '') NOT IN {plumbing_tools}- AND NOT arrayExists(prefix -> startsWith(ifNull(toString(properties.$mcp_tool_name), ''), prefix), {upkeep_prefixes}))-"""--# The projection approximates render.py in SQL, and these filters do not run here:-# - the repeated-job filter and the dedupe of identical lines, so units are overcounted for people-# with scheduled jobs;-# - secret stripping, whitespace folding and the cut at a word boundary, which change lengths very little.-# A line is estimated from the first and last work intents of the session, each capped like the render.-# Sessions per pair are capped at `MAX_UNITS_PER_DAY` per active day, like the burst cap.-# The pair is (project, distinct_id), not (project, person): a person with several distinct ids is-# counted more than once. The calibration ratio against the inputs asset absorbs the length error.-PROJECTION_MCP_SQL = f"""-SELECT- distinct_id,- project,- least(- countIf(work_calls >= {MIN_WORK_CALLS}),- {MAX_UNITS_PER_DAY} * uniqExactIf(day, work_calls >= {MIN_WORK_CALLS})- ) AS mcp_sessions,- avgIf(line_chars, work_calls >= {MIN_WORK_CALLS}) AS mcp_avg_line_chars-FROM (- SELECT- distinct_id,- $group_2 AS project,- {_SESSION_KEY} AS session_key,- toDate(toTimeZone(max(timestamp), 'UTC')) AS day,- countIf({_IS_WORK_CALL}) AS work_calls,- least(length(argMinIf(toString(properties.$mcp_intent), timestamp, {_IS_WORK_CALL})), {MCP_STEP_CHARS})- + least(length(argMaxIf(toString(properties.$mcp_intent), timestamp, {_IS_WORK_CALL})), {MCP_STEP_CHARS})- + {MCP_LINE_PREFIX_CHARS} AS line_chars- FROM events- WHERE {_MCP_FILTERS}- GROUP BY distinct_id, project, session_key-)-GROUP BY distinct_id, project-HAVING mcp_sessions > 0-"""--# The projection aggregates every pair over 28 days of MCP calls, so it gets the spill guard of the-# other wide reads in this dag on top of the time limit.-_PROJECTION_SETTINGS = HogQLGlobalSettings(- max_execution_time=600,- max_memory_usage=20 * 1024**3,- max_bytes_before_external_group_by=10 * 1024**3,-)-_INPUT_SETTINGS = HogQLGlobalSettings(max_execution_time=600)---def _strings(values: Collection[str]) -> ast.Tuple:- return ast.Tuple(exprs=[ast.Constant(value=value) for value in sorted(values)])---def kept_scopes() -> list[str]:- return [f"{source}:{preset}" for source, presets in MCP_SOURCE_SCOPE_PRESETS.items() for preset in presets]---def run_query(- sql: str,- *,- team: Team,- query_type: str,- window_start: datetime.datetime,- window_end: datetime.datetime,- placeholders: dict[str, ast.Expr],- settings: HogQLGlobalSettings,-) -> list[tuple[Any, ...]]:- response = execute_hogql_query(- query=sql,- team=team,- query_type=query_type,- placeholders={- "window_start": ast.Constant(value=utc_bound(window_start)),- "window_end": ast.Constant(value=utc_bound(window_end)),- "kept_scopes": _strings(kept_scopes()),- **placeholders,- },- limit_context=LimitContext.SAVED_QUERY,- workload=etl_workload(),- settings=settings,- # The dag runs without a user; the read is a trusted internal ETL over the dogfood project.- bypass_warehouse_access_control=True,- ch_user=ClickHouseUser.SIGNALS,- )- return [tuple(row) for row in response.results or []]---def input_rows(- sql: str,- *,- team: Team,- query_type: str,- window_start: datetime.datetime,- window_end: datetime.datetime,- distinct_ids: Collection[str],- projects: Collection[str],-) -> list[tuple[Any, ...]]:- if not distinct_ids or not projects:- return []- return run_query(- sql,- team=team,- query_type=query_type,- window_start=window_start,- window_end=window_end,- placeholders={"distinct_ids": _strings(distinct_ids), "projects": _strings(projects)},- settings=_INPUT_SETTINGS,- )---def projection_query_rows(- sql: str, *, team: Team, query_type: str, window_start: datetime.datetime, window_end: datetime.datetime-) -> list[tuple[Any, ...]]:- return run_query(- sql,- team=team,- query_type=query_type,- window_start=window_start,- window_end=window_end,- placeholders={- "plumbing_tools": _strings(PLUMBING_TOOL_NAMES),- "upkeep_prefixes": ast.Array(exprs=[ast.Constant(value=prefix) for prefix in UPKEEP_TOOL_PREFIXES]),- },- settings=_PROJECTION_SETTINGS,- )diff --git a/products/signals/dags/inbox_ranking/activity_summary/render.py b/products/signals/dags/inbox_ranking/activity_summary/render.pydeleted file mode 100644index 38ac999c5fa0..000000000000--- a/products/signals/dags/inbox_ranking/activity_summary/render.py+++ /dev/null@@ -1,588 +0,0 @@-"""Pure functions that turn one person's recent activity into the input of an activity summary.--Nothing here reads a database. `dag.py` fetches the rows, and this module filters, caps, samples and-renders them, and estimates what a summary update would cost.--Most activity recorded under a person is not theirs: scouts, agents and scheduled jobs act with the-person's identity. Each filter is a named rule in `Drop`, and `build_input` counts what each rule-removed, so a rule that starts dropping everything, or nothing, shows up in the output.-"""--import re-import math-import random-import hashlib-import datetime-from collections import Counter-from collections.abc import Iterable, Mapping, Sequence-from enum import StrEnum--import numpy as np--from posthog.dataclasses import frozen--# Bump on any change to the filters, the caps, the sampling or the rendered layout, so partitions stay comparable.-RENDER_VERSION = 2--WINDOW_DAYS = 28-RECENCY_HALF_LIFE_DAYS = 7-CHARS_PER_TOKEN = 4--# Rows read per person before any filter, newest first. They bound query and memory cost for very-# active people; the filters and caps below work on what is left.-MAX_TASKS_PER_PERSON = 200-MAX_MCP_SESSIONS_PER_PERSON = 300-MAX_ACTIVITY_ROWS_PER_PERSON = 1000--# Units one source may keep from a single UTC day, newest first, so one busy afternoon cannot fill a section.-MAX_UNITS_PER_DAY = 3-TASK_LINES = 12-MCP_LINES = 10-ACTIVITY_LINES = 15-# Per-line caps, so one runaway title or intent cannot take the budget. With them a full prompt-# section stays under SECTIONS_MAX_CHARS; the hard stop is the backstop if the caps change.-TASK_LINE_CHARS = 260-TASK_GOAL_CHARS = 180-TASK_TITLE_CHARS = 100-MAX_TAGS = 5-MAX_TAG_CHARS = 30-MCP_LINE_CHARS = 240-MCP_STEP_CHARS = 110-ACTIVITY_LINE_CHARS = 120-ACTIVITY_NAME_CHARS = 70-SECTIONS_MAX_CHARS = 8000--# Generated intents are a sentence or two. The cap bounds the regex work on a client-supplied value.-MAX_INTENT_CHARS = 1000-# Masking and whitespace folding shorten a text, so the patterns read this many times its cap.-RAW_TEXT_FACTOR = 4-# Task summaries run to a few hundred characters; the cap bounds the regex work on a long one.-MAX_SUMMARY_CHARS = 2000-# The same intent text more often than this on one day, for one person, is a scheduled job.-REPEATED_INTENT_DAILY_LIMIT = 5-# A session needs this many calls that do real work, after plumbing and upkeep, to say what the person tried.-MIN_WORK_CALLS = 2-# A task title with no summary needs this many words to say anything ("root cause" does not).-MIN_BARE_TITLE_WORDS = 8--STEP_SEPARATOR = " → "-TASKS_HEADER = "Tasks the person gave to PostHog AI or the Slack agent ([tags] goal):"-MCP_HEADER = "Sessions of the person's own coding agent over MCP (first step → last step):"-ACTIVITY_HEADER = "Things the person created or changed in PostHog:"--# Tools an agent calls to find its way around, not to do the work: tool discovery, schema reads,-# docs and knowledge lookups, and project lookup or switching. Their intents restate the task.-PLUMBING_TOOL_NAMES: frozenset[str] = frozenset(- {- "business-knowledge-document-window-retrieve",- "business-knowledge-documents-search",- "business-knowledge-repositories-search",- "docs-search",- "generate-app-url",- "metric-describe",- "metric-list",- "organization-details-get",- "project-get",- "projects-get",- "read-data-schema",- "switch-organization",- "switch-project",- }-)-# Agents keep their own notes in skills and request approval reviews on every PR. Heavy agent users-# make hundreds of these calls, and they say nothing about what the person works on.-UPKEEP_TOOL_PREFIXES: tuple[str, ...] = ("skill-", "stamphog-")--# Helper tasks that a task spawns for itself, such as repository selection, carry the parent's origin.-SANDBOX_PROMPT_TITLE = re.compile(r"^\[sandbox_prompt:")-# Titles a button or a workflow writes, with only an id changing. Each collapses to one line with a count.-TEMPLATED_TASK_TITLES: tuple[re.Pattern[str], ...] = (re.compile(r"^Fix failing CI checks in PR #\d+$"),)-_TEMPLATE_ID = re.compile(r"#\d+")-_SUMMARY_LABEL = re.compile(r"^(?:Goal|Asked|Question)\s*:\s*", re.IGNORECASE)-_SENTENCE_END = re.compile(r"(?<=[.!?])\s+")--# Scopes agents write in bulk, such as tags applied to every report they touch.-NOISY_ACTIVITY_SCOPES: frozenset[str] = frozenset({"TaggedItem", "Tag"})-# A scout run writes its changes with this client tag, which the caller cannot set.-SCOUT_CLIENT_PREFIX = "scout:"-# Names that a scheduled job stamps with a date, such as a notebook regenerated every day.-_AUTO_NAME = re.compile(r"\b\d{4}-\d{2}-\d{2}\b|model report", re.IGNORECASE)--_EMAIL = re.compile(r"[\w.+-]+@[\w-]+(?:\.[\w-]+)+")-_URL_QUERY = re.compile(r"(https?://[^\s?#]+)[?#]\S*")-# Whole links, dropped from task summaries: a URL carries no topic the summary text does not already name.-_URL = re.compile(r"<?https?://\S+>?")-# Long runs of word characters: a run that mixes letters and digits is an API key, session token,-# hash or UUID. The digit and letter checks run in `_mask_token`, because lookaheads in the pattern-# rescan the run at every word boundary inside it and take quadratic time on a long input.-_WORD_RUN = re.compile(r"[\w-]{24,}")-_DIGIT = re.compile(r"\d")-_LETTER = re.compile(r"[A-Za-z]")-_WHITESPACE = re.compile(r"\s+")---class Drop(StrEnum):- """Why a unit or call did not reach the prompt."""-- PLUMBING_TOOL = "plumbing_tool"- UPKEEP_TOOL = "upkeep_tool"- REPEATED_JOB = "repeated_job"- THIN_SESSION = "thin_session"- SANDBOX_PROMPT_TASK = "sandbox_prompt_task"- BARE_TITLE = "bare_title"- NOISY_SCOPE = "noisy_scope"- SCOUT_CLIENT = "scout_client"- AUTO_NAMED_OBJECT = "auto_named_object"- UNNAMED_OBJECT = "unnamed_object"- DUPLICATE = "duplicate"- BURST_CAP = "burst_cap"- SAMPLE_CAP = "sample_cap"- SECTIONS_BUDGET = "sections_budget"---@frozen-class McpCall:- at: datetime.datetime- tool: str- intent: str---@frozen-class McpSession:- calls: tuple[McpCall, ...]---@frozen-class TaskItem:- """A task the person started, with its latest run's summary and tags."""-- at: datetime.datetime- title: str- summary: str | None- tags: tuple[str, ...]---@frozen-class ActivityItem:- """One activity log row the person made."""-- at: datetime.datetime- scope: str- activity: str- name: str | None- client: str | None---@frozen-class Unit:- """One rendered line, dated by its latest activity."""-- at: datetime.datetime- text: str---@frozen-class RenderedInput:- text: str- n_tasks: int- n_mcp_sessions: int- n_activity_lines: int- task_chars: int- mcp_chars: int- activity_chars: int- input_chars: int- input_tokens_estimate: int- drops: dict[str, int]---def _mask_token(match: re.Match[str]) -> str:- run = match.group()- return "<token>" if _DIGIT.search(run) and _LETTER.search(run) else run---def bounded_raw(text: str, max_chars: int) -> str:- """At most `max_chars` of the text, without a word that the cut splits.-- A split secret is too short for the masking patterns to match, so the partial word goes too."""- if len(text) <= max_chars:- return text- end = max_chars- if not text[end].isspace():- while end > 0 and not text[end - 1].isspace():- end -= 1- return text[:end]---def clean_text(text: str, max_chars: int) -> str:- """Strip emails, URL query strings and token-like strings, fold the text onto one line, and cut- it to `max_chars`.-- The cut comes after the masking, so it cannot leave part of a secret unmasked. The patterns read- a bounded prefix, because event properties are client-supplied and have no length limit."""- text = bounded_raw(text, max_chars * RAW_TEXT_FACTOR)- text = _EMAIL.sub("<email>", text)- text = _URL_QUERY.sub(r"\1", text)- text = _WORD_RUN.sub(_mask_token, text)- return _WHITESPACE.sub(" ", text).strip()[:max_chars].strip()---def cap_line(text: str, max_chars: int) -> str:- """Cut at a word boundary and mark the cut."""- if len(text) <= max_chars:- return text- cut = text[: max_chars - 1]- return (cut.rsplit(" ", 1)[0] if " " in cut else cut) + "…"---def _day(at: datetime.datetime) -> datetime.date:- return at.astimezone(datetime.UTC).date()---def job_intents(sessions: Iterable[McpSession]) -> frozenset[tuple[datetime.date, str]]:- """The (UTC day, intent) pairs that one person sent more than `REPEATED_INTENT_DAILY_LIMIT` times.-- Pass every session of the person, across projects, because a job does not stop being a job when- it switches project."""- counts = Counter((_day(call.at), call.intent.strip()) for session in sessions for call in session.calls)- return frozenset(key for key, count in counts.items() if count > REPEATED_INTENT_DAILY_LIMIT)---def collapse_repeats(intents: Sequence[str]) -> list[str]:- return [intent for index, intent in enumerate(intents) if index == 0 or intent != intents[index - 1]]---def _call_drop(call: McpCall, jobs: frozenset[tuple[datetime.date, str]]) -> Drop | None:- if call.tool in PLUMBING_TOOL_NAMES:- return Drop.PLUMBING_TOOL- if call.tool.startswith(UPKEEP_TOOL_PREFIXES):- return Drop.UPKEEP_TOOL- if (_day(call.at), call.intent.strip()) in jobs:- return Drop.REPEATED_JOB- return None---def mcp_unit(session: McpSession, jobs: frozenset[tuple[datetime.date, str]], drops: Counter[str]) -> Unit | None:- """The session as `first step → last step`, or None when too little real work is left."""- calls = sorted(session.calls, key=lambda call: call.at)- work: list[str] = []- for call in calls:- if (reason := _call_drop(call, jobs)) is not None:- drops[reason] += 1- continue- if intent := clean_text(call.intent, MAX_INTENT_CHARS):- work.append(intent)- if len(work) < MIN_WORK_CALLS:- drops[Drop.THIN_SESSION] += 1- return None- steps = collapse_repeats(work)- path = cap_line(steps[0], MCP_STEP_CHARS)- if len(steps) > 1:- path += STEP_SEPARATOR + cap_line(steps[-1], MCP_STEP_CHARS)- line = f"- {_day(calls[-1].at).isoformat()} ({len(calls)} calls) {path}"- return Unit(at=calls[-1].at, text=cap_line(line, MCP_LINE_CHARS))---def summary_goal(summary: str) -> str:- """The opening sentences of a task summary, which state what was asked, without the label."""- text = _SUMMARY_LABEL.sub("", clean_text(_URL.sub("", bounded_raw(summary, MAX_SUMMARY_CHARS)), MAX_SUMMARY_CHARS))- goal = ""- for sentence in _SENTENCE_END.split(text):- if goal and len(goal) + len(sentence) + 1 > TASK_GOAL_CHARS:- break- goal = f"{goal} {sentence}".strip()- return cap_line(goal, TASK_GOAL_CHARS)---def _templated_label(title: str) -> str | None:- for pattern in TEMPLATED_TASK_TITLES:- if pattern.match(title):- return _TEMPLATE_ID.sub("#N", title)- return None---def task_units(tasks: Iterable[TaskItem], drops: Counter[str]) -> list[Unit]:- """One line per task, with templated titles collapsed to one line per template."""- units: list[Unit] = []- templated: dict[str, list[datetime.datetime]] = {}- for task in tasks:- title = task.title.strip()- if SANDBOX_PROMPT_TITLE.match(title):- drops[Drop.SANDBOX_PROMPT_TASK] += 1- continue- if (label := _templated_label(title)) is not None:- templated.setdefault(label, []).append(task.at)- continue- day = _day(task.at).isoformat()- if task.summary:- tags = [cap_line(clean_text(tag, MAX_TAG_CHARS * 2), MAX_TAG_CHARS) for tag in task.tags[:MAX_TAGS]]- tag_text = f"[{', '.join(tag for tag in tags if tag)}] " if any(tags) else ""- line = f"- {day} {tag_text}{summary_goal(task.summary)}"- elif len(title.split()) >= MIN_BARE_TITLE_WORDS:- line = f"- {day} {cap_line(clean_text(title, TASK_TITLE_CHARS * 2), TASK_TITLE_CHARS)}"- else:- drops[Drop.BARE_TITLE] += 1- continue- units.append(Unit(at=task.at, text=cap_line(line, TASK_LINE_CHARS)))- for label, times in templated.items():- latest = max(times)- units.append(- Unit(at=latest, text=cap_line(f"- {_day(latest).isoformat()} {label} (x{len(times)})", TASK_LINE_CHARS))- )- return units---def activity_units(items: Iterable[ActivityItem], drops: Counter[str]) -> list[Unit]:- """Changes collapsed to one line per object, creates first, then the most changed, then the newest.-- Lines are already collapsed per object, so they skip the per-day burst cap: a busy day of- changes to different objects lands on one date and the cap would keep only a few of them."""- collapsed: dict[tuple[str, str, str], list[datetime.datetime]] = {}- for item in items:- name = (item.name or "").strip()- if item.scope in NOISY_ACTIVITY_SCOPES:- drops[Drop.NOISY_SCOPE] += 1- elif (item.client or "").startswith(SCOUT_CLIENT_PREFIX):- drops[Drop.SCOUT_CLIENT] += 1- elif not name:- drops[Drop.UNNAMED_OBJECT] += 1- elif _AUTO_NAME.search(name):- drops[Drop.AUTO_NAMED_OBJECT] += 1- else:- collapsed.setdefault((item.scope, item.activity, name), []).append(item.at)- ranked = sorted(- collapsed.items(),- key=lambda entry: (entry[0][1] != "created", -len(entry[1]), -max(entry[1]).timestamp()),- )- if len(ranked) > ACTIVITY_LINES:- drops[Drop.SAMPLE_CAP] += len(ranked) - ACTIVITY_LINES- units: list[Unit] = []- for (scope, activity, name), times in ranked[:ACTIVITY_LINES]:- latest = max(times)- count = f" (x{len(times)})" if len(times) > 1 else ""- name_text = cap_line(clean_text(name, ACTIVITY_NAME_CHARS * 2), ACTIVITY_NAME_CHARS)- line = f'- {_day(latest).isoformat()} {activity} {scope} "{name_text}"{count}'- units.append(Unit(at=latest, text=cap_line(line, ACTIVITY_LINE_CHARS)))- return sorted(units, key=lambda unit: (unit.at, unit.text))---def dedupe_latest(units: Iterable[Unit], drops: Counter[str]) -> list[Unit]:- """One unit per distinct text, the latest one, in chronological order."""- latest: dict[str, Unit] = {}- for unit in units:- if unit.text in latest:- drops[Drop.DUPLICATE] += 1- if unit.text not in latest or unit.at > latest[unit.text].at:- latest[unit.text] = unit- return sorted(latest.values(), key=lambda unit: (unit.at, unit.text))---def burst_cap(units: Sequence[Unit], drops: Counter[str]) -> list[Unit]:- """At most `MAX_UNITS_PER_DAY` units per UTC day, newest first, in chronological order."""- kept: list[Unit] = []- per_day: Counter[datetime.date] = Counter()- for unit in sorted(units, key=lambda unit: (unit.at, unit.text), reverse=True):- if per_day[_day(unit.at)] >= MAX_UNITS_PER_DAY:- drops[Drop.BURST_CAP] += 1- continue- per_day[_day(unit.at)] += 1- kept.append(unit)- return sorted(kept, key=lambda unit: (unit.at, unit.text))---def sampling_seed(partition_key: str, team_id: int, user_id: int) -> int:- # Not hash(): Python salts it per process, and a re-run must pick the same units.- digest = hashlib.sha256(f"{partition_key}:{team_id}:{user_id}".encode()).digest()- return int.from_bytes(digest[:8], "big")---def sample_units(units: Sequence[Unit], *, limit: int, as_of: datetime.datetime, seed: int) -> list[Unit]:- """Up to `limit` units without replacement, weighted towards recent ones, in chronological order.-- Weighted reservoir keys (Efraimidis and Spirakis): each unit draws `u ** (1 / weight)` and the- largest keys win, so one busy afternoon cannot fill every slot."""- if len(units) <= limit:- return sorted(units, key=lambda unit: (unit.at, unit.text))- rng = random.Random(seed)- keyed: list[tuple[float, Unit]] = []- for unit in sorted(units, key=lambda unit: (unit.at, unit.text)):- age_days = max((as_of - unit.at).total_seconds(), 0.0) / 86400- weight = 0.5 ** (age_days / RECENCY_HALF_LIFE_DAYS)- keyed.append((rng.random() ** (1 / weight), unit))- chosen = sorted(keyed, key=lambda pair: pair[0], reverse=True)[:limit]- return sorted((unit for _, unit in chosen), key=lambda unit: (unit.at, unit.text))---def estimate_tokens(chars: int) -> int:- return math.ceil(chars / CHARS_PER_TOKEN)---def _section(header: str, units: Sequence[Unit]) -> str:- return "\n".join([header, *(unit.text for unit in units)]) if units else ""---def _sections_chars(sections: Sequence[tuple[str, list[Unit]]]) -> int:- return sum(len(_section(header, units)) + 2 for header, units in sections)---def enforce_budget(sections: list[tuple[str, list[Unit]]], drops: Counter[str]) -> None:- """Drop the oldest unit across sections until the rendered sections fit `SECTIONS_MAX_CHARS`."""- while _sections_chars(sections) > SECTIONS_MAX_CHARS:- oldest = min((units[0].at, index) for index, (_, units) in enumerate(sections) if units)- sections[oldest[1]][1].pop(0)- drops[Drop.SECTIONS_BUDGET] += 1---def build_input(- sessions: Iterable[McpSession],- tasks: Iterable[TaskItem],- activity: Iterable[ActivityItem],- *,- jobs: frozenset[tuple[datetime.date, str]],- as_of: datetime.datetime,- seed: int,-) -> RenderedInput | None:- """The rendered input of one (project, person) pair, or None when nothing survives the filters.-- Order: the named filters, then the burst cap, then the recency-weighted sample, then the hard- budget, so the sample is drawn from clean units only."""- drops: Counter[str] = Counter()- task_pool = burst_cap(dedupe_latest(task_units(tasks, drops), drops), drops)- mcp_pool = burst_cap(- dedupe_latest((unit for session in sessions if (unit := mcp_unit(session, jobs, drops)) is not None), drops),- drops,- )- activity_lines = activity_units(activity, drops)- if not task_pool and not mcp_pool and not activity_lines:- return None- # Separate seeds per source, so new activity in one source does not reshuffle another.- chosen_tasks = sample_units(task_pool, limit=TASK_LINES, as_of=as_of, seed=seed)- chosen_mcp = sample_units(mcp_pool, limit=MCP_LINES, as_of=as_of, seed=seed + 1)- drops[Drop.SAMPLE_CAP] += len(task_pool) - len(chosen_tasks) + len(mcp_pool) - len(chosen_mcp)- sections = [(TASKS_HEADER, chosen_tasks), (MCP_HEADER, chosen_mcp), (ACTIVITY_HEADER, activity_lines)]- enforce_budget(sections, drops)- rendered = [_section(header, units) for header, units in sections]- text = "\n\n".join(part for part in rendered if part)- return RenderedInput(- text=text,- n_tasks=len(sections[0][1]),- n_mcp_sessions=len(sections[1][1]),- n_activity_lines=len(sections[2][1]),- task_chars=len(rendered[0]),- mcp_chars=len(rendered[1]),- activity_chars=len(rendered[2]),- input_chars=len(text),- input_tokens_estimate=estimate_tokens(len(text)),- drops={str(reason): count for reason, count in sorted(drops.items()) if count},- )---# `- 2026-01-31 (12 calls) ` before the steps and ` → ` between them, for the projection's length estimate.-MCP_LINE_PREFIX_CHARS = 24 + len(STEP_SEPARATOR)---@frozen-class PairActivity:- """MCP counts and lengths for one (project, person) pair, as the projection SQL computes them.-- The projection covers only the MCP section: tasks and activity are capped per person, so their- share of a prompt has a known upper bound and goes in as a fixed overhead."""-- mcp_sessions: int- mcp_avg_line_chars: float-- @property- def units(self) -> int:- return self.mcp_sessions---def estimated_input_chars(activity: PairActivity) -> int:- """The length the MCP section would get: a sample of average-length lines under its header."""- if activity.mcp_sessions <= 0:- return 0- sampled = min(activity.mcp_sessions, MCP_LINES)- return round(len(MCP_HEADER) + sampled * (min(activity.mcp_avg_line_chars, MCP_LINE_CHARS) + 1))---def capped_sections_tokens() -> int:- """The most tokens the task and activity sections can take under their caps."""- tasks = len(TASKS_HEADER) + TASK_LINES * (TASK_LINE_CHARS + 1)- activity = len(ACTIVITY_HEADER) + ACTIVITY_LINES * (ACTIVITY_LINE_CHARS + 1)- return estimate_tokens(tasks + activity)---def calibration_ratio[K](actual_chars: Mapping[K, int], estimated_chars: Mapping[K, int]) -> float:- """How far the SQL estimate is from the real rendered length, over the pairs both sides have.-- 1.0 when no pair matches, so a missing inputs partition leaves the estimate as it is."""- keys = [key for key in actual_chars if estimated_chars.get(key, 0) > 0]- estimated = sum(estimated_chars[key] for key in keys)- if not estimated:- return 1.0- return sum(actual_chars[key] for key in keys) / estimated---def updates_per_window(units: int, *, min_units: int, min_days: int) -> int:- """Updates a pair earns in the window: one per `min_units` new units, at most one per `min_days`."""- return min(units // min_units, WINDOW_DAYS // min_days)---MIN_UNITS_OPTIONS = (3, 5, 10)-MIN_DAYS_OPTIONS = (7, 14)-DAYS_PER_MONTH = 30---@frozen-class ProjectionPrices:- prompt_overhead_tokens: int- output_tokens_per_update: int- input_price_per_million_tokens: float | None- output_price_per_million_tokens: float | None---def _percentile(values: Sequence[int], q: float) -> int | None:- return int(np.percentile(values, q)) if values else None---def projection_rows(activities: Sequence[PairActivity], *, ratio: float, prices: ProjectionPrices) -> list[dict]:- """One row per (`MIN_UNITS`, `MIN_DAYS`) option: eligible pairs, input token percentiles, and the- monthly volume and cost. Counts only: no pair is named."""- tokens = [estimate_tokens(round(estimated_input_chars(activity) * ratio)) for activity in activities]- rows: list[dict] = []- for min_units in MIN_UNITS_OPTIONS:- eligible = [index for index, activity in enumerate(activities) if activity.units >= min_units]- eligible_tokens = [tokens[index] for index in eligible]- for min_days in MIN_DAYS_OPTIONS:- monthly_updates = [- updates_per_window(activities[index].units, min_units=min_units, min_days=min_days)- * DAYS_PER_MONTH- / WINDOW_DAYS- for index in eligible- ]- updates = sum(monthly_updates)- input_tokens = sum(- count * (tokens[index] + prices.prompt_overhead_tokens)- for count, index in zip(monthly_updates, eligible, strict=True)- )- output_tokens = updates * prices.output_tokens_per_update- cost = None- if prices.input_price_per_million_tokens is not None and prices.output_price_per_million_tokens is not None:- cost = (- input_tokens * prices.input_price_per_million_tokens- + output_tokens * prices.output_price_per_million_tokens- ) / 1_000_000- rows.append(- {- "min_units": min_units,- "min_days": min_days,- "eligible_pairs": len(eligible),- "input_tokens_p50": _percentile(eligible_tokens, 50),- "input_tokens_p90": _percentile(eligible_tokens, 90),- "input_tokens_p99": _percentile(eligible_tokens, 99),- "updates_per_month": updates,- "input_tokens_per_month": input_tokens,- "output_tokens_per_month": output_tokens,- "projected_monthly_cost_usd": cost,- "calibration_ratio": ratio,- }- )- return rowsdiff --git a/products/signals/dags/inbox_ranking/activity_summary/summaries.py b/products/signals/dags/inbox_ranking/activity_summary/summaries.pydeleted file mode 100644index 4a573b959c01..000000000000--- a/products/signals/dags/inbox_ranking/activity_summary/summaries.py+++ /dev/null@@ -1,193 +0,0 @@-"""The LLM step behind `activity_summaries`: one call per inputs row, its cost, and its event.--The call goes through `build_openai_client`, so the deployment picks the gateway: the Go AI gateway-when its settings are set, else the Python gateway on the `signals` route.-"""--import json-import time-import datetime-from collections.abc import Callable, Mapping, Sequence-from concurrent.futures import ThreadPoolExecutor-from typing import Any--from openai import OpenAI, OpenAIError-from openai.types.chat import ChatCompletionMessageParam--from posthog.dataclasses import frozen-from posthog.llm.gateway_client import build_openai_client, trace_id_from_seed--from products.signals.dags.inbox_ranking.activity_summary.prompt import (- PROMPT_VERSION,- RESPONSE_FORMAT,- SYSTEM_PROMPT,- MalformedSummaryError,- normalize_summary,- summary_text,- user_message,-)-from products.signals.dags.inbox_ranking.training.telemetry import TrainingEvent--AI_PRODUCT = "activity_summary"-# Not a person: the gateway attributes every call to the pipeline, and the events use the same id.-PIPELINE_DISTINCT_ID = "inbox_ranking_activity_summary"-DEFAULT_MODEL = "gpt-6-luna"-# One retry on a malformed answer. The SDK already retries a failed request on its own.-MAX_ATTEMPTS = 2-REQUEST_TIMEOUT_SECONDS = 120.0--STATUS_OK = "ok"-STATUS_FAILED = "failed"--SUMMARY_GENERATED_EVENT = "activity summary generated"---@frozen-class SummaryPrices:- input_price_per_million_tokens: float | None- output_price_per_million_tokens: float | None---def cost_usd(tokens_in: int, tokens_out: int, prices: SummaryPrices) -> float | None:- """None without both prices, so a run that does not set them records no guess."""- if prices.input_price_per_million_tokens is None or prices.output_price_per_million_tokens is None:- return None- return (- tokens_in * prices.input_price_per_million_tokens + tokens_out * prices.output_price_per_million_tokens- ) / 1_000_000---@frozen-class SummaryRequest:- team_id: int- user_id: int- rendered_input: str- previous_summary: dict[str, Any] | None---@frozen-class SummaryOutcome:- summary: dict[str, Any] | None- tokens_in: int- tokens_out: int- latency_ms: int- error: str | None---def gateway_client(partition_key: str) -> OpenAI:- return build_openai_client(- "signals",- ai_product=AI_PRODUCT,- trace_id=trace_id_from_seed(f"{AI_PRODUCT}:{partition_key}"),- properties={"partition": partition_key},- distinct_id=PIPELINE_DISTINCT_ID,- )---def summarize(client: OpenAI, request: SummaryRequest, *, model: str, as_of: datetime.date) -> SummaryOutcome:- """One summary. Tokens and latency add up over the attempts, because the retry costs too."""- messages: list[ChatCompletionMessageParam] = [- {"role": "system", "content": SYSTEM_PROMPT},- {"role": "user", "content": user_message(request.rendered_input, request.previous_summary, as_of)},- ]- tokens_in = tokens_out = 0- error: str | None = None- started = time.monotonic()- for _ in range(MAX_ATTEMPTS):- try:- response = client.chat.completions.create(- model=model,- messages=messages,- user=PIPELINE_DISTINCT_ID,- response_format=RESPONSE_FORMAT,- timeout=REQUEST_TIMEOUT_SECONDS,- )- except OpenAIError as api_error:- error = f"api_error: {type(api_error).__name__}"- break- if response.usage is not None:- tokens_in += response.usage.prompt_tokens- tokens_out += response.usage.completion_tokens- content = response.choices[0].message.content if response.choices else None- try:- summary = normalize_summary(content, had_previous_summary=request.previous_summary is not None)- except MalformedSummaryError as malformed:- error = f"malformed_output: {malformed}"- continue- return SummaryOutcome(- summary=summary,- tokens_in=tokens_in,- tokens_out=tokens_out,- latency_ms=round((time.monotonic() - started) * 1000),- error=None,- )- return SummaryOutcome(- summary=None,- tokens_in=tokens_in,- tokens_out=tokens_out,- latency_ms=round((time.monotonic() - started) * 1000),- error=error,- )---def summarize_all(- requests: Sequence[SummaryRequest], summarize_one: Callable[[SummaryRequest], SummaryOutcome], max_concurrency: int-) -> list[SummaryOutcome]:- """The outcomes in request order, at most `max_concurrency` calls at once."""- with ThreadPoolExecutor(max_workers=max(1, max_concurrency)) as pool:- return list(pool.map(summarize_one, requests))---def summary_row(request: SummaryRequest, outcome: SummaryOutcome, *, model: str, prices: SummaryPrices) -> dict:- return {- "team_id": request.team_id,- "user_id": request.user_id,- "summary_json": json.dumps(outcome.summary, ensure_ascii=False, sort_keys=True) if outcome.summary else None,- "summary_text": summary_text(outcome.summary) if outcome.summary else None,- "used_previous_summary": request.previous_summary is not None,- "model": model,- "prompt_version": PROMPT_VERSION,- "tokens_in": outcome.tokens_in,- "tokens_out": outcome.tokens_out,- "latency_ms": outcome.latency_ms,- "cost_usd": cost_usd(outcome.tokens_in, outcome.tokens_out, prices),- "status": STATUS_OK if outcome.summary else STATUS_FAILED,- "error": outcome.error,- }---_EVENT_ROW_KEYS = (- "team_id",- "user_id",- "model",- "prompt_version",- "status",- "error",- "used_previous_summary",- "tokens_in",- "tokens_out",- "cost_usd",- "latency_ms",-)-_EVENT_INPUT_KEYS = ("n_tasks", "n_mcp_sessions", "n_activity_lines")---def summary_events(- rows: Sequence[Mapping[str, Any]], inputs: Sequence[Mapping[str, Any]], *, capture_summary_text: bool-) -> list[TrainingEvent]:- """One event per summary row, `rows[i]` made from `inputs[i]`.-- Counts only by default. The summary text goes on the event only when `capture_summary_text` is- set, because an event cannot be edited or deleted the way a partition can. The `error` value is- one of this module's own fixed strings, never model output."""- events: list[TrainingEvent] = []- for row, input_row in zip(rows, inputs, strict=True):- properties: dict[str, object] = {key: row[key] for key in _EVENT_ROW_KEYS}- properties |= {key: input_row[key] for key in _EVENT_INPUT_KEYS}- properties |= {f"drops_{rule}": count for rule, count in json.loads(input_row["drops"] or "{}").items()}- if capture_summary_text and row["summary_json"]:- summary = json.loads(row["summary_json"])- properties["summary_text"] = row["summary_text"]- properties["focus_topics"] = [area["topic"] for area in summary["focus_areas"]]- events.append(TrainingEvent(event=SUMMARY_GENERATED_EVENT, properties=properties))- return eventsdiff --git a/products/signals/dags/inbox_ranking/tests/test_activity_summary.py b/products/signals/dags/inbox_ranking/tests/test_activity_summary.pydeleted file mode 100644index e7d21111c1a4..000000000000--- a/products/signals/dags/inbox_ranking/tests/test_activity_summary.py+++ /dev/null@@ -1,833 +0,0 @@-import io-import json-import datetime-from collections import Counter-from types import SimpleNamespace-from typing import Any--import pytest-import time_machine--import dagster-import pyarrow as pa-from botocore.exceptions import ClientError-from openai import APIConnectionError-from parameterized import parameterized--from posthog import settings--from products.signals.dags.inbox_ranking.activity_summary import dag-from products.signals.dags.inbox_ranking.activity_summary.embeddings import (- LandedVector,- SummaryDocument,- within_source_ttl,-)-from products.signals.dags.inbox_ranking.activity_summary.prompt import (- MalformedSummaryError,- normalize_summary,- summary_text,- user_message,-)-from products.signals.dags.inbox_ranking.activity_summary.render import (- ACTIVITY_HEADER,- ACTIVITY_LINE_CHARS,- ACTIVITY_LINES,- MAX_UNITS_PER_DAY,- MCP_HEADER,- MCP_LINE_CHARS,- MCP_LINES,- REPEATED_INTENT_DAILY_LIMIT,- SECTIONS_MAX_CHARS,- TASK_LINE_CHARS,- TASK_LINES,- TASKS_HEADER,- ActivityItem,- Drop,- McpCall,- McpSession,- PairActivity,- ProjectionPrices,- TaskItem,- Unit,- activity_units,- build_input,- burst_cap,- calibration_ratio,- clean_text,- job_intents,- mcp_unit,- projection_rows,- sample_units,- sampling_seed,- task_units,-)-from products.signals.dags.inbox_ranking.activity_summary.summaries import (- PIPELINE_DISTINCT_ID,- SummaryPrices,- SummaryRequest,- cost_usd,- summarize,- summary_events,- summary_row,-)--AS_OF = datetime.datetime(2026, 10, 19, tzinfo=datetime.UTC)---def _at(days_ago: float, minutes: int = 0) -> datetime.datetime:- return AS_OF - datetime.timedelta(days=days_ago) + datetime.timedelta(minutes=minutes)---def _calls(*steps: tuple[str, str], days_ago: float = 1) -> McpSession:- return McpSession(- calls=tuple(- McpCall(at=_at(days_ago, minutes=index), tool=tool, intent=intent)- for index, (tool, intent) in enumerate(steps)- )- )---def _session(*intents: str, tool: str = "insight-create", days_ago: float = 1) -> McpSession:- return _calls(*((tool, intent) for intent in intents), days_ago=days_ago)---def _task(title: str, summary: str | None = None, tags: tuple[str, ...] = (), days_ago: float = 1) -> TaskItem:- return TaskItem(at=_at(days_ago), title=title, summary=summary, tags=tags)---def _change(- scope: str, activity: str, name: str | None, client: str | None = None, days_ago: float = 1-) -> ActivityItem:- return ActivityItem(at=_at(days_ago), scope=scope, activity=activity, name=name, client=client)--- [- (- "plumbing and upkeep calls drop out, the line is first step to last step",- _calls(- ("read-data-schema", "Check the event schema"),- ("insight-create", "Build a signup funnel"),- ("skill-update", "Save handover notes"),- ("stamphog-review-runs-create", "Request an approval review"),- ("dashboard-create", "Add the funnel to a dashboard"),- ),- "- 2026-10-18 (5 calls) Build a signup funnel → Add the funnel to a dashboard",- {Drop.PLUMBING_TOOL: 1, Drop.UPKEEP_TOOL: 2},- ),- (- "repeated steps collapse to one",- _session("Build a funnel", "Build a funnel"),- "- 2026-10-18 (2 calls) Build a funnel",- {},- ),- (- "a session with one work call left is thin",- _calls(("project-get", "Find the project"), ("insight-create", "Build a funnel")),- None,- {Drop.PLUMBING_TOOL: 1, Drop.THIN_SESSION: 1},- ),- ]-)-def test_mcp_unit(_name: str, session: McpSession, expected: str | None, expected_drops: dict[str, int]) -> None:- drops: Counter[str] = Counter()- unit = mcp_unit(session, frozenset(), drops)- assert (unit.text if unit else None) == expected- assert dict(drops) == expected_drops---def test_repeated_intent_on_one_day_is_treated_as_a_job() -> None:- polling = [- _session("Poll the dashboard", "Read the insight", days_ago=1) for _ in range(REPEATED_INTENT_DAILY_LIMIT + 1)- ]- occasional = [- _session("Poll the dashboard", "Read the insight", days_ago=3) for _ in range(REPEATED_INTENT_DAILY_LIMIT)- ]- jobs = job_intents([*polling, *occasional])- drops: Counter[str] = Counter()-- assert mcp_unit(polling[0], jobs, drops) is None- unit = mcp_unit(occasional[0], jobs, drops)- assert unit is not None and unit.text.endswith("Poll the dashboard → Read the insight")- assert drops[Drop.REPEATED_JOB] == 2--- [- (- "a summary gives the goal, without its label, and the tags",- _task("why", summary="Goal: find why weekly signups dropped. Done: found a broken form.", tags=("growth",)),- "- 2026-10-18 [growth] find why weekly signups dropped. Done: found a broken form.",- {},- ),- (- "a helper task is dropped",- _task("[sandbox_prompt:repo_selection] You are a selection agent"),- None,- {Drop.SANDBOX_PROMPT_TASK: 1},- ),- ("a short title with no summary says nothing", _task("root cause"), None, {Drop.BARE_TITLE: 1}),- (- "a long title with no summary is kept",- _task("compare the retention of the two pricing plans over the last quarter"),- "- 2026-10-18 compare the retention of the two pricing plans over the last quarter",- {},- ),- ]-)-def test_task_units(_name: str, task: TaskItem, expected: str | None, expected_drops: dict[str, int]) -> None:- drops: Counter[str] = Counter()- units = task_units([task], drops)- assert [unit.text for unit in units] == ([expected] if expected else [])- assert dict(drops) == expected_drops---def test_templated_task_titles_collapse_to_one_line_with_a_count() -> None:- tasks = [_task(f"Fix failing CI checks in PR #{number}", days_ago=number) for number in (1, 2, 3)]- units = task_units(tasks, Counter())- assert [unit.text for unit in units] == ["- 2026-10-18 Fix failing CI checks in PR #N (x3)"]---def test_activity_units_filter_collapse_and_rank_creates_first() -> None:- drops: Counter[str] = Counter()- items = [- _change("TaggedItem", "created", "growth"),- _change("Insight", "created", "Signup funnel", client="scout:feature-flags"),- _change("Notebook", "updated", "Daily report 2026-10-18"),- _change("AlertConfiguration", "created", None),- _change("Notebook", "updated", "Pricing notes", days_ago=1),- _change("Notebook", "updated", "Pricing notes", days_ago=2),- _change("Dashboard", "created", "Growth overview", days_ago=3),- ]-- units = activity_units(items, drops)-- assert [unit.text for unit in units] == [- '- 2026-10-16 created Dashboard "Growth overview"',- '- 2026-10-18 updated Notebook "Pricing notes" (x2)',- ]- assert dict(drops) == {- Drop.NOISY_SCOPE: 1,- Drop.SCOUT_CLIENT: 1,- Drop.AUTO_NAMED_OBJECT: 1,- Drop.UNNAMED_OBJECT: 1,- }---def test_burst_cap_keeps_the_newest_units_of_a_day() -> None:- drops: Counter[str] = Counter()- units = [Unit(at=_at(1, minutes=minute), text=f"unit {minute}") for minute in range(MAX_UNITS_PER_DAY + 2)]-- kept = burst_cap(units, drops)-- assert [unit.text for unit in kept] == [f"unit {minute}" for minute in range(2, MAX_UNITS_PER_DAY + 2)]- assert drops[Drop.BURST_CAP] == 2--- [- ("email", "Ask [email protected] about it", "Ask <email> about it"),- (- "url query string",- "Open https://example.com/insights/abc?token=xyz&a=1 now",- "Open https://example.com/insights/abc now",- ),- ("token-like string", "Use key phx_9f8e7d6c5b4a3f2e1d0c9b8a7f6e5d4c", "Use key <token>"),- ("multiline text folds onto one line", "First line\n\n second\tline ", "First line second line"),- (- "ordinary text is untouched",- "Why did weekly signups drop in October?",- "Why did weekly signups drop in October?",- ),- ("long run without a digit is kept", "a-" * 20, ("a-" * 20).strip()),- ("text is cut to the cap", "x" * 100, "x" * 60),- (- "secret across the cap is masked, not cut",- "a" * 50 + " sk-proj-abcdefghijklmnop1234567890",- "a" * 50 + " <token>",- ),- ("word split by the read bound is dropped", " " * 230 + "ab sk-proj-abcdefghijklmnop1234567890", "ab"),- ]-)-def test_clean_text(_name: str, text: str, expected: str) -> None:- assert clean_text(text, 60) == expected---def test_sampling_is_deterministic_and_favors_recent_units() -> None:- units = [Unit(at=_at(days_ago), text=f"unit {days_ago}") for days_ago in range(28)]- seed = sampling_seed("2026-10-18", 2, 7)-- first = sample_units(units, limit=MCP_LINES, as_of=AS_OF, seed=seed)- assert first == sample_units(list(reversed(units)), limit=MCP_LINES, as_of=AS_OF, seed=seed)- assert len(first) == MCP_LINES- assert first == sorted(first, key=lambda unit: unit.at)-- picked_ages = [- (AS_OF - unit.at).days- for partition in range(50)- for unit in sample_units(units, limit=MCP_LINES, as_of=AS_OF, seed=sampling_seed(str(partition), 2, 7))- ]- assert sum(1 for age in picked_ages if age < 7) > sum(1 for age in picked_ages if age >= 21)---def test_build_input_renders_the_three_sections_in_order() -> None:- rendered = build_input(- [_session("Build a signup funnel", "Add it to a dashboard", days_ago=2)],- [_task("why", summary="Asked: why did signups drop?", tags=("growth",), days_ago=3)],- [_change("Dashboard", "created", "Growth overview", days_ago=1)],- jobs=frozenset(),- as_of=AS_OF,- seed=1,- )-- assert rendered is not None- assert rendered.text == "\n".join(- [- TASKS_HEADER,- "- 2026-10-16 [growth] why did signups drop?",- "",- MCP_HEADER,- "- 2026-10-17 (2 calls) Build a signup funnel → Add it to a dashboard",- "",- ACTIVITY_HEADER,- '- 2026-10-18 created Dashboard "Growth overview"',- ]- )- assert (rendered.n_tasks, rendered.n_mcp_sessions, rendered.n_activity_lines) == (1, 1, 1)- assert rendered.input_chars == len(rendered.text)- assert build_input([], [_task("root cause")], [], jobs=frozenset(), as_of=AS_OF, seed=1) is None---def test_build_input_stays_inside_its_bounds_for_a_very_active_person() -> None:- runaway = "word " * 2000- sessions = [- _session(f"{day} {minute} {runaway}", f"{runaway} end", days_ago=day + minute / 100)- for day in range(28)- for minute in range(20)- ]- tasks = [- _task(runaway, summary=f"{day} {n} {runaway}", tags=(runaway,) * 20, days_ago=day + 0.01 * n)- for day in range(28)- for n in range(20)- ]- activity = [_change("Insight", "created", f"{runaway} {n}", days_ago=n % 28) for n in range(500)]-- rendered = build_input(sessions, tasks, activity, jobs=frozenset(), as_of=AS_OF, seed=1)-- assert rendered is not None- assert (rendered.n_tasks, rendered.n_mcp_sessions, rendered.n_activity_lines) == (- TASK_LINES,- MCP_LINES,- ACTIVITY_LINES,- )- tasks_text, mcp_text, activity_text = rendered.text.split("\n\n")- assert max(len(line) for line in tasks_text.split("\n")[1:]) <= TASK_LINE_CHARS- assert max(len(line) for line in mcp_text.split("\n")[1:]) <= MCP_LINE_CHARS- assert max(len(line) for line in activity_text.split("\n")[1:]) <= ACTIVITY_LINE_CHARS- assert rendered.input_chars <= SECTIONS_MAX_CHARS- assert rendered.drops[Drop.BURST_CAP] > 0 and rendered.drops[Drop.SAMPLE_CAP] > 0---def test_projection_counts_eligible_pairs_and_caps_updates_by_cadence() -> None:- activities = [- PairActivity(mcp_sessions=2, mcp_avg_line_chars=100.0),- PairActivity(mcp_sessions=40, mcp_avg_line_chars=150.0),- ]- prices = ProjectionPrices(- prompt_overhead_tokens=0,- output_tokens_per_update=0,- input_price_per_million_tokens=None,- output_price_per_million_tokens=None,- )-- rows = {(row["min_units"], row["min_days"]): row for row in projection_rows(activities, ratio=1.0, prices=prices)}-- assert rows[(3, 7)]["eligible_pairs"] == 1- # 40 units would earn 13 updates at 3 units each, but a 7-day cadence allows 4 in the 28-day window.- assert rows[(3, 7)]["updates_per_month"] == pytest.approx(4 * 30 / 28)- assert rows[(10, 14)]["updates_per_month"] == pytest.approx(2 * 30 / 28)- assert all(row["projected_monthly_cost_usd"] is None for row in rows.values())---def test_calibration_ratio_ignores_unmatched_pairs() -> None:- assert calibration_ratio({(1, "a"): 300, (1, "b"): 999}, {(1, "a"): 200, (2, "c"): 50}) == pytest.approx(1.5)- assert calibration_ratio({}, {(1, "a"): 200}) == 1.0---def test_asset_rows_match_their_schemas_and_skip_projects_outside_the_organization() -> None:- window = dag.activity_window("2026-10-18")- member_own = dag.MemberProject(team_id=2, user_id=7)- member_customer = dag.MemberProject(team_id=99, user_id=7)- sessions = {- dag.PairKey(project="own-project", distinct_id="member"): [- _session("Build a signup funnel", "Add it to a dashboard")- ],- dag.PairKey(project="customer-project", distinct_id="member"): [- _session("Debug a customer's flag", "Read its rollout")- ],- }-- rows = dag.input_rows(- sessions=sessions,- tasks={member_customer: [_task("compare the retention of the two pricing plans over the last quarter")]},- activity={member_own: [_change("Dashboard", "created", "Growth overview")]},- users={"member": 7},- projects={"own-project": 2},- window=window,- partition_key="2026-10-18",- )-- assert [row["team_id"] for row in rows] == [2]- assert set(rows[0]) == set(dag.INPUTS_SCHEMA.names)- assert json.loads(rows[0]["drops"]) == {}- projection = projection_rows(- [PairActivity(mcp_sessions=5, mcp_avg_line_chars=100.0)],- ratio=1.0,- prices=ProjectionPrices(- prompt_overhead_tokens=0,- output_tokens_per_update=0,- input_price_per_million_tokens=1.0,- output_price_per_million_tokens=1.0,- ),- )- assert set(projection[0]) | {"window_start", "window_end"} == set(dag.COST_PROJECTION_SCHEMA.names)---PARTITION = "2026-10-18"-PREFIX = "inbox_ranking"-PREVIOUS = {"summary": "Builds the signup funnel.", "focus_areas": [], "products": ["product_analytics"]}--[email protected]([("with a previous summary", PREVIOUS), ("cold start", None)])-def test_user_message_sends_only_the_rendered_input_and_the_previous_summary(- _name: str, previous: dict[str, Any] | None-) -> None:- message = user_message("- 2026-10-17 Build a funnel", previous, datetime.date(2026, 10, 18))- assert message.startswith("Today is 2026-10-18.")- assert message.endswith("Recent activity:\n- 2026-10-17 Build a funnel")- if previous:- assert json.dumps(PREVIOUS["summary"]) in message- else:- assert "Previous summary:\nNone" in message---def _answer(**overrides: Any) -> str:- return json.dumps(- {- "summary": "Builds the signup funnel.",- "focus_areas": [{"topic": "signup funnel", "strength": "high", "last_seen": "2026-10-17"}],- "products": ["product_analytics"],- "recent_shift": "Moved from flags to funnels.",- **overrides,- }- )--- [- ("recent_shift is dropped without a previous summary", _answer(), False, {"recent_shift": None}),- (- "recent_shift is kept with a previous summary",- _answer(),- True,- {"recent_shift": "Moved from flags to funnels."},- ),- ("a 'None' recent_shift becomes null", _answer(recent_shift="None"), True, {"recent_shift": None}),- ("a 'null' recent_shift becomes null", _answer(recent_shift=" null "), True, {"recent_shift": None}),- (- "the summary is cut to the word cap",- _answer(summary="word " * 80),- False,- {"summary": " ".join(["word"] * 60)},- ),- (- "at most six focus areas, an unknown strength is low and a bad date is null",- _answer(focus_areas=[{"topic": f"t{i}", "strength": "huge", "last_seen": "soon"} for i in range(8)]),- False,- {"focus_areas": [{"topic": f"t{i}", "strength": "low", "last_seen": None} for i in range(6)]},- ),- (- "a focus area without a topic is dropped",- _answer(focus_areas=[{"strength": "high"}]),- False,- {"focus_areas": []},- ),- ]-)-def test_normalize_summary(_name: str, content: str, had_previous: bool, expected: dict[str, Any]) -> None:- assert normalize_summary(content, had_previous_summary=had_previous).items() >= expected.items()--- [- ("empty", None),- ("not JSON", "Here is the summary: ..."),- ("not an object", "[1, 2]"),- ("no summary", _answer(summary=" ")),- ]-)-def test_normalize_summary_rejects_a_malformed_answer(_name: str, content: str | None) -> None:- with pytest.raises(MalformedSummaryError):- normalize_summary(content, had_previous_summary=False)---def test_summary_text_is_deterministic_and_keeps_the_focus_order() -> None:- summary = normalize_summary(- _answer(- focus_areas=[- {"topic": "b", "strength": "high", "last_seen": ""},- {"topic": "a", "strength": "low", "last_seen": ""},- ]- ),- had_previous_summary=False,- )- reloaded = json.loads(json.dumps(summary, sort_keys=True))- assert summary_text(summary) == summary_text(reloaded) == "Builds the signup funnel.\nFocus areas: b; a"- assert summary_text({**summary, "focus_areas": []}) == "Builds the signup funnel."--- [- (- "both prices",- 1_000_000,- 200_000,- SummaryPrices(input_price_per_million_tokens=0.1, output_price_per_million_tokens=0.5),- 0.2,- ),- (- "no output price",- 1_000,- 100,- SummaryPrices(input_price_per_million_tokens=0.1, output_price_per_million_tokens=None),- None,- ),- (- "no prices",- 1_000,- 100,- SummaryPrices(input_price_per_million_tokens=None, output_price_per_million_tokens=None),- None,- ),- ]-)-def test_cost_usd(_name: str, tokens_in: int, tokens_out: int, prices: SummaryPrices, expected: float | None) -> None:- assert cost_usd(tokens_in, tokens_out, prices) == (pytest.approx(expected) if expected is not None else None)---class _FakeLLM:- def __init__(self, answers: list[str | Exception]):- self.answers = list(answers)- self.requests: list[dict[str, Any]] = []- self.chat = SimpleNamespace(completions=SimpleNamespace(create=self.create))-- def create(self, **kwargs: Any) -> SimpleNamespace:- self.requests.append(kwargs)- answer = self.answers.pop(0) if len(self.answers) > 1 else self.answers[0]- if isinstance(answer, Exception):- raise answer- return SimpleNamespace(- usage=SimpleNamespace(prompt_tokens=100, completion_tokens=20),- choices=[SimpleNamespace(message=SimpleNamespace(content=answer))],- )---_CONNECTION_ERROR = APIConnectionError(request=SimpleNamespace()) # type: ignore[arg-type]--- [- ("a malformed answer is retried once", ["not json", _answer()], "ok", 2, None),- (- "two malformed answers fail the row",- ["not json", "still not json"],- "failed",- 2,- "malformed_output: response is not JSON",- ),- (- "a failed request fails the row without a retry",- [_CONNECTION_ERROR],- "failed",- 1,- "api_error: APIConnectionError",- ),- ]-)-def test_summarize_retries_a_malformed_answer_once(- _name: str, answers: list[str | Exception], status: str, attempts: int, error: str | None-) -> None:- llm = _FakeLLM(answers)- request = SummaryRequest(team_id=2, user_id=7, rendered_input="- a line", previous_summary=None)- outcome = summarize(llm, request, model="gpt-6-luna", as_of=datetime.date(2026, 10, 18)) # type: ignore[arg-type]- row = summary_row(- request,- outcome,- model="gpt-6-luna",- prices=SummaryPrices(input_price_per_million_tokens=None, output_price_per_million_tokens=None),- )-- assert len(llm.requests) == attempts- assert {req["user"] for req in llm.requests} == {PIPELINE_DISTINCT_ID}- assert (row["status"], row["error"]) == (status, error)- successful_calls = attempts if not isinstance(answers[0], Exception) else 0- assert (row["tokens_in"], row["tokens_out"]) == (100 * successful_calls, 20 * successful_calls)- assert set(row) == set(dag.SUMMARIES_SCHEMA.names)--[email protected]([("text off by default", False), ("text on when asked", True)])-def test_summary_events_carry_free_text_only_when_asked(_name: str, capture_text: bool) -> None:- request = SummaryRequest(team_id=2, user_id=7, rendered_input="- a line", previous_summary=None)- outcome = summarize(_FakeLLM([_answer()]), request, model="gpt-6-luna", as_of=datetime.date(2026, 10, 18)) # type: ignore[arg-type]- row = summary_row(- request,- outcome,- model="gpt-6-luna",- prices=SummaryPrices(input_price_per_million_tokens=None, output_price_per_million_tokens=None),- )- input_row = {"n_tasks": 1, "n_mcp_sessions": 0, "n_activity_lines": 3, "drops": json.dumps({"burst_cap": 4})}-- (event,) = summary_events([row], [input_row], capture_summary_text=capture_text)-- assert {- "team_id": 2,- "user_id": 7,- "status": "ok",- "n_activity_lines": 3,- "drops_burst_cap": 4,- }.items() <= event.properties.items()- free_text = {"summary_text", "focus_topics"} & event.properties.keys()- assert free_text == ({"summary_text", "focus_topics"} if capture_text else set())- if not capture_text:- assert not any(isinstance(value, str) and "signup" in value for value in event.properties.values())---class _MemoryS3:- def __init__(self) -> None:- self.objects: dict[str, tuple[bytes, dict[str, str]]] = {}-- def upload_fileobj(self, fileobj: io.IOBase, bucket: str, key: str, ExtraArgs: dict[str, Any]) -> None:- self.objects[key] = (fileobj.read(), ExtraArgs["Metadata"])-- def _get(self, key: str, operation: str) -> tuple[bytes, dict[str, str]]:- if key not in self.objects:- raise ClientError({"Error": {"Code": "NoSuchKey"}}, operation)- return self.objects[key]-- def get_object(self, *, Bucket: str, Key: str) -> dict[str, Any]:- return {"Body": io.BytesIO(self._get(Key, "GetObject")[0])}-- def head_object(self, *, Bucket: str, Key: str) -> dict[str, Any]:- return {"Metadata": self._get(Key, "HeadObject")[1]}-- def get_paginator(self, _name: str) -> "_MemoryS3":- return self-- def paginate(self, *, Bucket: str, Prefix: str): # noqa: ANN201- yield {"Contents": [{"Key": key} for key in sorted(self.objects) if key.startswith(Prefix)]}-- def write(self, table: str, partition: str, rows: list[dict[str, Any]], schema: pa.Schema) -> None:- key = dag.partition_object_key(PREFIX, table, partition)- dag.write_parquet(self, "bucket", key, pa.Table.from_pylist(rows, schema=schema))-- def read(self, table: str, partition: str) -> list[dict[str, Any]]:- table_ = dag.read_parquet_if_exists(self, "bucket", dag.partition_object_key(PREFIX, table, partition))- return [] if table_ is None else table_.to_pylist()---class _FailingCapture:- def capture(self, **kwargs: Any) -> None:- raise RuntimeError("capture is down")-- def shutdown(self) -> None:- pass---def memory_s3(monkeypatch: pytest.MonkeyPatch) -> _MemoryS3:- s3 = _MemoryS3()- monkeypatch.setattr(dag, "skip_unconfigured", lambda context: False)- monkeypatch.setattr(dag, "s3_client", lambda: s3)- monkeypatch.setattr(dag, "dataset_bucket", lambda: "bucket")- monkeypatch.setattr(settings, "INBOX_RANKING_DATASET_S3_PREFIX", PREFIX)- return s3---def _input_row(team_id: int, user_id: int, text: str) -> dict[str, Any]:- window = dag.activity_window(PARTITION)- return {- "team_id": team_id,- "user_id": user_id,- "window_start": window.start,- "window_end": window.end,- "n_tasks": 0,- "n_mcp_sessions": 1,- "n_activity_lines": 0,- "rendered_input": text,- "task_chars": 0,- "mcp_chars": len(text),- "activity_chars": 0,- "input_chars": len(text),- "input_tokens_estimate": 1,- "drops": "{}",- "render_version": 2,- }---def _summary_row(team_id: int, user_id: int, status: str, summary: dict[str, Any] | None) -> dict[str, Any]:- return {- "team_id": team_id,- "user_id": user_id,- "summary_json": json.dumps(summary) if summary else None,- "summary_text": summary_text(summary) if summary else None,- "used_previous_summary": False,- "model": "gpt-6-luna",- "prompt_version": 1,- "tokens_in": 1,- "tokens_out": 1,- "latency_ms": 1,- "cost_usd": None,- "status": status,- "error": None if summary else "malformed_output: empty response",- }---def test_summaries_asset_reads_earlier_partitions_only_and_survives_a_failed_capture(- memory_s3: _MemoryS3, monkeypatch: pytest.MonkeyPatch-) -> None:- memory_s3.write(- dag.INPUTS_TABLE,- PARTITION,- [_input_row(2, 7, "- input seven"), _input_row(2, 8, "- input eight")],- dag.INPUTS_SCHEMA,- )- older = {"summary": "Older work on flags.", "focus_areas": [], "products": []}- memory_s3.write(dag.SUMMARIES_TABLE, "2026-10-10", [_summary_row(2, 7, "ok", older)], dag.SUMMARIES_SCHEMA)- memory_s3.write(- dag.SUMMARIES_TABLE,- "2026-10-17",- [_summary_row(2, 7, "ok", PREVIOUS), _summary_row(2, 8, "failed", None)],- dag.SUMMARIES_SCHEMA,- )- memory_s3.write(dag.SUMMARIES_TABLE, "2026-10-19", [_summary_row(2, 8, "ok", older)], dag.SUMMARIES_SCHEMA)- llm = _FakeLLM([_answer()])- monkeypatch.setattr(dag, "gateway_client", lambda partition_key: llm)- monkeypatch.setattr("products.signals.dags.inbox_ranking.training.telemetry.is_cloud", lambda: True)- monkeypatch.setattr(- "products.signals.dags.inbox_ranking.training.telemetry.get_client", lambda region, **kwargs: _FailingCapture()- )-- for _ in range(2):- dag.activity_summaries(dagster.build_asset_context(partition_key=PARTITION), dag.SummariesConfig())-- rows = memory_s3.read(dag.SUMMARIES_TABLE, PARTITION)- assert [(row["user_id"], row["status"], row["used_previous_summary"]) for row in rows] == [- (7, "ok", True),- (8, "ok", False),- ]- messages = {- req["messages"][1]["content"].rsplit("\n", 1)[-1]: req["messages"][1]["content"] for req in llm.requests- }- assert "Builds the signup funnel." in messages["- input seven"]- assert "Previous summary:\nNone" in messages["- input eight"]---class _FailingForLLM(_FakeLLM):- def __init__(self, failing_input: str):- super().__init__([_answer()])- self.failing_input = failing_input-- def create(self, **kwargs: Any) -> SimpleNamespace:- if kwargs["messages"][1]["content"].endswith(self.failing_input):- self.requests.append(kwargs)- raise _CONNECTION_ERROR- return super().create(**kwargs)--[email protected]("users", [[7], [7, 8]], ids=["every row fails", "one row fails"])-def test_summaries_asset_keeps_earlier_successes_when_a_rerun_fails(- users: list[int], memory_s3: _MemoryS3, monkeypatch: pytest.MonkeyPatch-) -> None:- names = {7: "seven", 8: "eight"}- memory_s3.write(- dag.INPUTS_TABLE,- PARTITION,- [_input_row(2, user, f"- input {names[user]}") for user in users],- dag.INPUTS_SCHEMA,- )- memory_s3.write(dag.SUMMARIES_TABLE, PARTITION, [_summary_row(2, 7, "ok", PREVIOUS)], dag.SUMMARIES_SCHEMA)- monkeypatch.setattr(dag, "gateway_client", lambda partition_key: _FailingForLLM("- input seven"))- context = dagster.build_asset_context(partition_key=PARTITION)-- if len(users) == 1:- with pytest.raises(dagster.Failure):- dag.activity_summaries(context, dag.SummariesConfig())- else:- dag.activity_summaries(context, dag.SummariesConfig())-- rows = memory_s3.read(dag.SUMMARIES_TABLE, PARTITION)- assert [(row["user_id"], row["status"]) for row in rows] == [(user, "ok") for user in users]- assert rows[0]["summary_json"] == json.dumps(PREVIOUS)--- [- ("recent partition", "2026-10-18", True),- ("at the cutoff", "2026-07-29", True),- ("past the cutoff", "2026-07-28", False),- ]-)-def test_within_source_ttl(_name: str, partition_key: str, expected: bool) -> None:- assert within_source_ttl(partition_key, datetime.date(2026, 10, 19)) is expected---@time_machine.travel(datetime.datetime(2026, 10, 19, 12, tzinfo=datetime.UTC), tick=False)-def test_embeddings_asset_embeds_only_new_text_and_never_shrinks_a_written_partition(- memory_s3: _MemoryS3, monkeypatch: pytest.MonkeyPatch-) -> None:- current = {"summary": "Builds the signup funnel.", "focus_areas": [], "products": []}- memory_s3.write(- dag.SUMMARIES_TABLE,- PARTITION,- [_summary_row(2, 7, "ok", current), _summary_row(2, 8, "ok", current), _summary_row(2, 9, "failed", None)],- dag.SUMMARIES_SCHEMA,- )- vector = [0.5] * 4- store = {- SummaryDocument(team_id=2, document_id=f"7:{PARTITION}"): LandedVector(- embedding=vector, content=summary_text(current)- ),- SummaryDocument(team_id=2, document_id=f"8:{PARTITION}"): LandedVector(- embedding=vector, content="text before a re-run"- ),- }- emitted: list[tuple[int, str]] = []-- def emit(- *, team_id: int, user_id: int, partition_key: str, prompt_version: int, text: str, day_start: datetime.datetime- ) -> None:- emitted.append((user_id, text))- store[SummaryDocument(team_id=team_id, document_id=f"{user_id}:{partition_key}")] = LandedVector(- embedding=vector, content=text- )-- monkeypatch.setattr(dag, "emit_summary_embedding", emit)- monkeypatch.setattr(dag, "flush_embedding_requests", lambda: None)- monkeypatch.setattr(dag.time, "sleep", lambda seconds: None)- monkeypatch.setattr(- dag, "landed_vectors", lambda documents, day_start: {d: store[d] for d in documents if d in store}- )-- dag.activity_summary_embeddings(dagster.build_asset_context(partition_key=PARTITION), dag.SummaryEmbeddingsConfig())-- assert emitted == [(8, summary_text(current))]- rows = memory_s3.read(dag.SUMMARY_EMBEDDINGS_TABLE, PARTITION)- assert [row["user_id"] for row in rows] == [7, 8]- assert set(rows[0]) == set(dag.SUMMARY_EMBEDDINGS_SCHEMA.names)-- store.clear()- monkeypatch.setattr(dag, "emit_summary_embedding", lambda **kwargs: None)- dag.activity_summary_embeddings(- dagster.build_asset_context(partition_key=PARTITION), dag.SummaryEmbeddingsConfig(wait_seconds=0)- )- assert [row["user_id"] for row in memory_s3.read(dag.SUMMARY_EMBEDDINGS_TABLE, PARTITION)] == [7, 8]