:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(signals): archive daily activity summary vectors for ranking training (#124484)
posthog[bot]297a240a4646
diff --git a/posthog/dags/slack_alerts.py b/posthog/dags/slack_alerts.pyindex d1025e6f42c7..60729894b628 100644--- a/posthog/dags/slack_alerts.py+++ b/posthog/dags/slack_alerts.py@@ -115,6 +115,7 @@ ASSET_NAME_PREFIX_OWNERS: tuple[tuple[str, JobOwners], ...] = (("inbox_signal_", JobOwners.TEAM_SELF_DRIVING),("inbox_ranking_", JobOwners.TEAM_SELF_DRIVING),("inbox_user_", JobOwners.TEAM_SELF_DRIVING),+ ("inbox_activity_", JobOwners.TEAM_SELF_DRIVING),)diff --git a/posthog/dags/tests/test_slack_alerts.py b/posthog/dags/tests/test_slack_alerts.pyindex e3fd16958358..58db22c7f9f5 100644--- a/posthog/dags/tests/test_slack_alerts.py+++ b/posthog/dags/tests/test_slack_alerts.py@@ -92,6 +92,7 @@ class TestSlackAlertsRouting:("inbox_signal_embeddings",),("inbox_ranking_model_candidate",),("inbox_user_report_interactions",),+ ("inbox_activity_summary_vectors",),])def test_asset_job_with_inbox_steps_routes_to_self_driving(self, step):diff --git a/products/signals/dags/inbox_ranking/AGENTS.md b/products/signals/dags/inbox_ranking/AGENTS.mdindex e88cedefc602..d819483fe12d 100644--- a/products/signals/dags/inbox_ranking/AGENTS.md+++ b/products/signals/dags/inbox_ranking/AGENTS.md@@ -10,6 +10,7 @@ Read `README.md` first for what the dataset is and how partitions behave. This f- `inbox_project_vectors` is a **leaf** with its own `PROJECT_VECTORS_SCHEMA_VERSION`: keep it out of `inbox_report_model_data`'s `deps`. Its `deps=[EMBEDDINGS_TABLE, LABELS_TABLE]` keep it after the embeddings snapshot, so the two vector-heavy steps do not share the run pod's memory. `project_vectors` takes a report into a vector only when its vector landed and its UI engagement happened strictly before `as_of`; never widen that to the latest snapshot value. A report born on day D reads the dt=D−1 vectors (see the README).- `inbox_engagement_export` is a **leaf** with its own `ENGAGEMENT_EXPORT_SCHEMA_VERSION`, read by another region. It writes to the mirror store only under `serving_engagement_key`, because the mirror's write grant covers only `<prefix>/serving/`. Keep its columns exactly `LABEL_COLUMNS` and its rows limited to reports with no `SignalReport` in this region: no team ids, counts, text or vectors leave the region. A mirror failure logs and must not fail the asset. Keep the local-share guard before the write: when the local report lookup finds no local report (`no_local_reports`) or a share below `ENGAGEMENT_EXPORT_MIN_LOCAL_SHARE` (`local_share_collapsed`), the asset writes nothing and still succeeds. Its `inbox_ranking_engagement_export_published` event carries counts only, never a report id or a team id.- `inbox_mirror_project_vectors` (`mirror/dag.py`) is the only inbox-ranking asset that registers on EU (`is_inbox_ranking_mirror_registered()`). It reads this region's report vectors through `report_embeddings_table` and the engagement export through `serving_engagement_key`, and writes only to this region's object store at `serving_project_vectors_key`. Keep it on the same `project_vectors()` and the same consent rule as `inbox_project_vectors`, so the two regions build their vectors the same way. A missing, too old or unknown-schema export fails the asset rather than writing vectors without engagement.+- `inbox_activity_summary_vectors` is a **leaf** with its own `ACTIVITY_SUMMARY_VECTORS_SCHEMA_VERSION`: keep it out of `inbox_report_model_data`'s `deps`, and never select the summary text (`content`). It is the only durable copy of the activity summary vectors once the source TTL passes, so a re-run must keep the rows the source no longer returns (`merge_activity_summary_vectors`). Training reads a user's vector at a moment through `activity_vector_as_of`, never the newest vector of the user.- Register new jobs/schedules in `posthog/dags/locations/signals.py` (or `signals_ml.py` for a torch-dependent dag), keep the EU gate (`is_inbox_ranking_registered()`), and remember the ci-dagster paths filter must cover every module a dag file imports (`uv run .github/scripts/check-dagster-paths.py`).## Training dag specifics@@ -46,7 +47,7 @@ Read `README.md` first for what the dataset is and how partitions behave. This f## Invariants — do not break-- `dt=` partitions are **immutable snapshots** with deterministic object keys; the only mutation ever applied is an idempotent re-run of the same partition. The exception is `inbox_signal_embeddings`, an emission log whose partition holds only that day's inserts — see the README's signal-grain section before touching it. Its re-run must stay **additive** (union with the existing object): the source drops rows it already archived, so a plain overwrite destroys history that exists nowhere else.+- `dt=` partitions are **immutable snapshots** with deterministic object keys; the only mutation ever applied is an idempotent re-run of the same partition. The exception is `inbox_signal_embeddings`, an emission log whose partition holds only that day's inserts — see the README's signal-grain section before touching it. Its re-run must stay **additive** (union with the existing object): the source drops rows it already archived, so a plain overwrite destroys history that exists nowhere else. `inbox_activity_summary_vectors` is the same kind of log (dt=D holds the vectors generated during D), and its re-run keeps every row the source no longer returns.- Label columns are **cumulative from `LABELS_EPOCH`**; never bake a maturity window or a rolling time bound into the SQL (the saved `inbox_ranking_*` views roll 90 days — that is exactly why their SQL is inlined here with explicit bounds instead of reused).- Every cumulative outcome count ships with the column holding the moment its first counted event arrived, declared as a pair in `OUTCOME_FIRST_EVENT_COLUMNS` (a test fails on an unpaired count). A count alone dates an outcome only to the whole cumulative window, so a horizon label or a time-to-outcome read has nothing but a proxy without that moment. The timestamp must use the same predicate as its count, or the pair describes different events.- Every `STATUS_SQL` aggregate reads only the latest transition's tenant (`event_team_id = latest_event_team_id`).diff --git a/products/signals/dags/inbox_ranking/README.md b/products/signals/dags/inbox_ranking/README.mdindex f346cb738489..db8dddd608ef 100644--- a/products/signals/dags/inbox_ranking/README.md+++ b/products/signals/dags/inbox_ranking/README.md@@ -27,7 +27,7 @@ Registered via `posthog/dags/locations/signals.py` (US only) and loaded locally## The dataset dag-`inbox_ranking_dataset_job` runs daily at 02:30 UTC (schedule default: running on prod US, stopped everywhere else — including hosted DEV and E2E, which have no dogfood project to read labels from) and builds nine assets on one daily partition, each a Parquet object in S3:+`inbox_ranking_dataset_job` runs daily at 02:30 UTC (schedule default: running on prod US, stopped everywhere else — including hosted DEV and E2E, which have no dogfood project to read labels from) and builds ten assets on one daily partition, each a Parquet object in S3:```texts3://<bucket>/<prefix>/@@ -41,7 +41,8 @@ s3://<bucket>/<prefix>/├── inbox_report_title_embeddings/v1/dt=YYYY-MM-DD/ # the same shape, for the title-only rendering├── inbox_report_reviewers/v1/dt=YYYY-MM-DD/ # one row per suggested reviewer of each spine report├── inbox_user_report_interactions/v1/dt=YYYY-MM-DD/ # cumulative interactions per (report, person)-└── inbox_project_vectors/v1/dt=YYYY-MM-DD/ # per-team mean report vectors of past engagement, by kind+├── inbox_project_vectors/v1/dt=YYYY-MM-DD/ # per-team mean report vectors of past engagement, by kind+└── inbox_activity_summary_vectors/v1/dt=YYYY-MM-DD/ # one row per activity summary vector generated during the day```The first four are report grain and land in one table. `inbox_signal_embeddings` is signal grain, feeds the group-level model, and is read on its own — training joins it to `inbox_report_model_data` by `report_id`.@@ -164,6 +165,20 @@ Point-in-time caveats, per source:- The count check remains as a backstop: a union can only grow, so a smaller result means the merge itself is broken and the write is refused. Row counts are stamped in S3 object metadata at write time to make that check a `head_object`. A deliberate shrink still means deleting the object by hand.- Signal text never leaves ClickHouse: the query selects the vector and the structured metadata (weight, source, match graph), not `content` or the free-text metadata fields.+### Activity summary vectors++`inbox_activity_summary_vectors` is a (team, user, day) leaf with its own `ACTIVITY_SUMMARY_VECTORS_SCHEMA_VERSION`.+It is the durable history of the per-user activity summary vectors, for a ranking model that reads a user vector next to the report vector.+Each successful activity summary build writes one vector per (user, day) to the document embeddings table (`document_id` = `"{user_id}:{date}"`, `timestamp` = the summary's `generated_at`).+That table drops a row 3 months after its `timestamp`, and `UserActivitySummaryVersion` keeps only the newest versions of a pair, so a vector that this asset does not copy in time cannot be rebuilt.++- `dt=D` holds only the vectors whose `timestamp` falls in D, one row per `(team_id, document_id)`, the newest `inserted_at` winning. It is not a cumulative snapshot.+- The columns are `team_id`, `user_id`, `document_id`, `generated_date`, `generated_at`, `embedding_small` (small-1536, the report vectors' space), `embedding_inserted_at`, `prompt_version`, `render_version`, `model` and `source` (`live`, or `backfill` for a summary built as of a past date). No summary text leaves ClickHouse.+- Training reads the union of partitions and calls `activity_vector_as_of` (`training/examples.py`) for the vector a user had at a moment: the newest one generated strictly before it, or nothing when that one is older than a staleness cap.+- A re-run replaces the rows the source still returns and keeps the rows it no longer returns, so a re-run after the TTL does not erase history. It drops the rows of teams without training consent.+- **It is a leaf.** Nothing joins it, so a failure here does not fail the training table. A day of vectors is small, so it has no ordering dependency.+- Backfill: materialize every partition from the first activity summary build that is still inside the source TTL. Repeat for the matching partitions after a backfill of past-dated summaries, before those vectors expire.+## The training dag`inbox_ranking_training_job` runs daily at 05:00 UTC on the same partition definition (gated like the dataset job).@@ -474,7 +489,7 @@ The dags train only on reports from organizations whose `Organization.is_ai_trai`consent.training_consent_team_ids()` returns the consenting teams that hold an inbox report, and every asset reads it once per run.- `inbox_report_state` keeps only the spine reports of those teams. A label event that names another team's report does not pull it back in. That report's labels still land in `inbox_report_model_data` as a label-only row, which training skips.-- `inbox_report_embeddings`, `inbox_report_title_embeddings` and `inbox_signal_embeddings` read only those teams, through `team_id IN (...)` in the ClickHouse query.+- `inbox_report_embeddings`, `inbox_report_title_embeddings`, `inbox_signal_embeddings` and `inbox_activity_summary_vectors` read only those teams, through `team_id IN (...)` in the ClickHouse query.- The `nn` examples asset drops every state row whose `report_team_id` is not in the current set, before it builds examples. The lookback window reaches partitions written before an organization opted out, so this filter is what makes an opt-out take effect on the next training run.The filter follows the current setting, so an organization that opts back in is trained on again.diff --git a/products/signals/dags/inbox_ranking/dataset/dag.py b/products/signals/dags/inbox_ranking/dataset/dag.pyindex da94186989b0..e4c88bbc2e22 100644--- a/products/signals/dags/inbox_ranking/dataset/dag.py+++ b/products/signals/dags/inbox_ranking/dataset/dag.py@@ -1,6 +1,6 @@"""Daily modeling dataset for the Self-driving Inbox report-ranking model.-Nine assets on one daily partition, each writing Parquet under the configured S3 prefix:+Ten assets on one daily partition, each writing Parquet under the configured S3 prefix:inbox_report_state/v1/dt=D/ Postgres spine + report-state columnsinbox_report_embeddings/v1/dt=D/ report_id -> small-1536 vector as of snapshot end@@ -11,6 +11,7 @@ Nine assets on one daily partition, each writing Parquet under the configured S3inbox_report_reviewers/v1/dt=D/ one row per suggested reviewer of each spine reportinbox_user_report_interactions/v1/dt=D/ cumulative interactions at (report, person) graininbox_project_vectors/v1/dt=D/ per-team mean report vectors of past engagement, by kind+ inbox_activity_summary_vectors/v1/dt=D/ one row per activity summary vector generated during Dinbox_engagement_export writes no dataset object. It sends the dt=D engagement moments of the otherregion's reports to the serving mirror store, for that region's project vectors.@@ -22,6 +23,8 @@ is a report-grain leaf: nothing joins it, and no model family trains on it. inboit to the report tables by report_id. inbox_user_report_interactions is a (report, person) grainleaf with its own schema version, so nothing joins it either. inbox_project_vectors is a(team, kind) leaf with its own schema version, built from the embeddings and labels snapshots.+inbox_activity_summary_vectors is a (team, user, day) leaf with its own schema version: the durable+copy of the per-user activity summary vectors, which the source table drops after its 3-month TTL.Partition dt=D is a full snapshot of the eligible report inventory (promoted or ever-labeled),with every label aggregate bounded `event_time < D+1 00:00 UTC`. Label columns are cumulative,@@ -81,6 +84,10 @@ from posthog.dags.common import dagster_tagsfrom posthog.dataclasses import frozenfrom posthog.models import OrganizationMembership, Team, User+from products.signals.backend.activity_summary.tick import (+ EMBEDDING_DOCUMENT_TYPE as ACTIVITY_SUMMARY_DOCUMENT_TYPE,+ EMBEDDING_RENDERING as ACTIVITY_SUMMARY_RENDERING,+)from products.signals.backend.models import (SignalReport,SignalReportAction,@@ -139,6 +146,8 @@ from products.signals.dags.inbox_ranking.dataset.project_vectors import (teams_with_history,)from products.signals.dags.inbox_ranking.dataset.queries import (+ ACTIVITY_SUMMARY_VECTORS_QUERY_SETTINGS,+ ACTIVITY_SUMMARY_VECTORS_SQL,LABEL_DEFAULTS,LABEL_STREAMS,LABELED_REPORT_IDS_SQL,@@ -179,6 +188,7 @@ REVIEWERS_TABLE = "inbox_report_reviewers"USER_INTERACTIONS_TABLE = "inbox_user_report_interactions"PROJECT_VECTORS_TABLE = "inbox_project_vectors"ENGAGEMENT_EXPORT = "inbox_engagement_export"+ACTIVITY_SUMMARY_VECTORS_TABLE = "inbox_activity_summary_vectors"# The reviewers asset is a leaf with its own columns, so its schema moves apart from# FEATURE_SCHEMA_VERSION. The same rule applies: additive nullable columns bump it.@@ -186,6 +196,11 @@ REVIEWERS_SCHEMA_VERSION = 1# The engagement export is read in another region, so its shape moves apart from every local snapshot.ENGAGEMENT_EXPORT_SCHEMA_VERSION = 1+# The activity summary vectors are a (team, user, day) leaf, so their shape moves apart too.+ACTIVITY_SUMMARY_VECTORS_SCHEMA_VERSION = 1+# Every document type that carries an activity summary vector. A backfilled type joins this tuple,+# and its rows say so through `source`.+ACTIVITY_SUMMARY_DOCUMENT_TYPES = (ACTIVITY_SUMMARY_DOCUMENT_TYPE,)# Most labels rows belong to this region's reports. A share below this floor means the local report# lookup lost rows, and the export would send this region's report ids to the other region.ENGAGEMENT_EXPORT_MIN_LOCAL_SHARE = 0.25@@ -289,6 +304,21 @@ _REVIEWER_FIELDS: list[tuple[str, pa.DataType]] = []REVIEWERS_SCHEMA = pa.schema(_REVIEWER_FIELDS)+_ACTIVITY_SUMMARY_VECTOR_FIELDS: list[tuple[str, pa.DataType]] = [+ ("team_id", pa.int64()),+ ("user_id", pa.int64()),+ ("document_id", pa.string()),+ ("generated_date", pa.date32()),+ ("generated_at", _TIMESTAMP),+ ("embedding_small", pa.list_(pa.float32())),+ ("embedding_inserted_at", _TIMESTAMP),+ ("prompt_version", pa.int32()),+ ("render_version", pa.int32()),+ ("model", pa.string()),+ ("source", pa.string()),+]+ACTIVITY_SUMMARY_VECTORS_SCHEMA = pa.schema(_ACTIVITY_SUMMARY_VECTOR_FIELDS)+# What makes one emission distinct from another, so a re-run can tell a row it already archived from# a genuinely new one. embedding_inserted_at is the version: the source keys a signal by# (team_id, document_id) and distinguishes its versions by inserted_at.@@ -1481,6 +1511,135 @@ def inbox_project_vectors(context: dagster.AssetExecutionContext) -> None:)+def activity_summary_vectors_table(partition_key: str, consent_team_ids: Collection[int]) -> tuple[pa.Table, int]:+ """The activity summary vectors generated during dt=D for the teams in `consent_team_ids`, one row+ per document, and the number of rows skipped because their document_id names no user."""+ window_start, window_end = snapshot_bounds(partition_key)+ results = (+ cast(+ list[tuple[Any, ...]],+ sync_execute(+ ACTIVITY_SUMMARY_VECTORS_SQL,+ {+ "product": EMBEDDING_PRODUCT,+ "document_types": list(ACTIVITY_SUMMARY_DOCUMENT_TYPES),+ "rendering": ACTIVITY_SUMMARY_RENDERING,+ "window_start": window_start.replace(tzinfo=None),+ "window_end": window_end.replace(tzinfo=None),+ "team_ids": sorted(consent_team_ids),+ },+ settings=ACTIVITY_SUMMARY_VECTORS_QUERY_SETTINGS,+ workload=etl_workload(),+ ch_user=ClickHouseUser.SIGNALS,+ )+ or [],+ )+ if consent_team_ids+ else []+ )++ columns: dict[str, list[Any]] = {name: [] for name, _type in _ACTIVITY_SUMMARY_VECTOR_FIELDS}+ malformed = 0+ for (+ team_id,+ document_id,+ generated_at,+ embedding,+ inserted_at,+ prompt_version,+ render_version,+ model,+ source,+ ) in results:+ # A malformed id is skipped and counted rather than raised: failing the asset would lose+ # the whole day's vectors, and the source drops them after its TTL.+ user_part, _, _ = str(document_id).partition(":")+ try:+ user_id = int(user_part)+ except ValueError:+ malformed += 1+ continue+ generated_at_utc = cast(datetime.datetime, ensure_utc(generated_at))+ columns["team_id"].append(int(team_id))+ columns["user_id"].append(user_id)+ columns["document_id"].append(str(document_id))+ columns["generated_date"].append(generated_at_utc.date())+ columns["generated_at"].append(generated_at_utc)+ columns["embedding_small"].append(list(embedding))+ columns["embedding_inserted_at"].append(ensure_utc(inserted_at))+ columns["prompt_version"].append(prompt_version)+ columns["render_version"].append(render_version)+ columns["model"].append(model)+ columns["source"].append(source)+ return pa.Table.from_pydict(columns, schema=ACTIVITY_SUMMARY_VECTORS_SCHEMA), malformed+++def merge_activity_summary_vectors(existing: pa.Table, fresh: pa.Table, consent_team_ids: Collection[int]) -> pa.Table:+ """The rows a re-run writes over a partition it already holds.++ A fresh row replaces the existing row of its document, so a re-embedded vector is updated and+ each document keeps one row. An existing row whose document the fresh scan no longer returns+ stays, because the source TTL or a merge removed it and this partition is the only copy left. A+ team without training consent now loses its rows in both cases."""+ fresh_documents = set(+ zip(fresh.column("team_id").to_pylist(), fresh.column("document_id").to_pylist(), strict=True)+ )+ keep = pa.array(+ [+ team_id in consent_team_ids and (team_id, document_id) not in fresh_documents+ for team_id, document_id in zip(+ existing.column("team_id").to_pylist(), existing.column("document_id").to_pylist(), strict=True+ )+ ],+ type=pa.bool_(),+ )+ return pa.concat_tables([fresh, existing.filter(keep)])++[email protected](name=ACTIVITY_SUMMARY_VECTORS_TABLE, **COMMON_ASSET_KWARGS)+def inbox_activity_summary_vectors(context: dagster.AssetExecutionContext) -> None:+ """One row per activity summary vector generated during dt=D, for user vectors in ranking training.++ The source table drops a vector 3 months after its timestamp, and `UserActivitySummaryVersion`+ keeps only the newest versions of a pair, so this partition is the durable history: a vector+ that is not copied before it expires cannot be rebuilt. Partition D holds only the vectors+ generated during D, not a cumulative snapshot. Training reads the union of partitions and takes+ the vector a user had at a moment through `activity_vector_as_of`.++ A leaf: nothing joins it, and it carries no summary text. A re-run replaces the rows the source+ still returns and keeps the rest (`merge_activity_summary_vectors`).+ """+ if skip_unconfigured(context):+ return+ tag_dagster_queries(context, query_type="inbox_ranking_activity_summary_vectors")+ partition_key = context.partition_key+ consent_team_ids = training_consent_team_ids()+ fresh, malformed = activity_summary_vectors_table(partition_key, consent_team_ids)++ bucket = dataset_bucket()+ key = partition_object_key(settings.INBOX_RANKING_DATASET_S3_PREFIX, ACTIVITY_SUMMARY_VECTORS_TABLE, partition_key)+ client = s3_client()+ existing = read_parquet_if_exists(client, bucket, key)+ table = fresh if existing is None else merge_activity_summary_vectors(existing, fresh, consent_team_ids)+ write_parquet(client, bucket, key, table, schema_version=ACTIVITY_SUMMARY_VECTORS_SCHEMA_VERSION)++ team_ids = table.column("team_id").to_pylist()+ user_ids = table.column("user_id").to_pylist()+ sources = table.column("source").to_pylist()+ context.add_output_metadata(+ {+ "rows": dagster.MetadataValue.int(table.num_rows),+ "teams": dagster.MetadataValue.int(len(set(team_ids))),+ "users": dagster.MetadataValue.int(len(set(zip(team_ids, user_ids, strict=True)))),+ **{f"{source}_rows": dagster.MetadataValue.int(sources.count(source)) for source in ("live", "backfill")},+ "carried_over": dagster.MetadataValue.int(table.num_rows - fresh.num_rows),+ "malformed_document_ids": dagster.MetadataValue.int(malformed),+ "training_consent_teams": dagster.MetadataValue.int(len(consent_team_ids)),+ "s3_key": dagster.MetadataValue.text(f"s3://{bucket}/{key}"),+ }+ )++def remote_engagement_rows(labels: pa.Table, local_report_ids: Collection[str]) -> pa.Table:"""The label rows of reports that do not exist in this region's Postgres.@@ -1772,6 +1931,7 @@ inbox_ranking_dataset_job = dagster.define_asset_job(USER_INTERACTIONS_TABLE,PROJECT_VECTORS_TABLE,ENGAGEMENT_EXPORT,+ ACTIVITY_SUMMARY_VECTORS_TABLE,],# The seven label streams run sequentially and each may take its full 600s query timeout, so an# hour left a slow-but-valid pass no room for the join, the S3 writes, or an asset retry — anddiff --git a/products/signals/dags/inbox_ranking/dataset/queries.py b/products/signals/dags/inbox_ranking/dataset/queries.pyindex 79de627344ed..d534df3c10c6 100644--- a/products/signals/dags/inbox_ranking/dataset/queries.py+++ b/products/signals/dags/inbox_ranking/dataset/queries.py@@ -275,6 +275,38 @@ WHERE product = %(product)sAND inserted_at < %(window_end)s"""+# Activity summary documents: one vector per (team, user, generated day), with document_id+# "{user_id}:{date}" and timestamp = the summary's generated_at. The window is on timestamp, not on+# inserted_at, so a partition holds the vectors generated during D whatever the embedding lag. The+# sort key leads with (team_id, toDate(timestamp)), so this reads one day of the consenting teams.+# A rebuild of the same day re-emits under the same document_id, and argMax keeps the newest version+# when a merge has not removed the older one yet. The summary text (`content`) is never selected.+ACTIVITY_SUMMARY_VECTORS_QUERY_SETTINGS: dict[str, int] = {+ "max_execution_time": 600,+ "max_memory_usage": 10 * 1024**3,+}++ACTIVITY_SUMMARY_VECTORS_SQL = f"""+SELECT+ team_id,+ document_id,+ argMax(timestamp, inserted_at) AS generated_at,+ argMax(embedding, inserted_at) AS embedding,+ max(inserted_at) AS embedding_inserted_at,+ nullIf(argMax(JSONExtractInt(metadata, 'prompt_version'), inserted_at), 0) AS prompt_version,+ nullIf(argMax(JSONExtractInt(metadata, 'render_version'), inserted_at), 0) AS render_version,+ nullIf(argMax(JSONExtractString(metadata, 'model'), inserted_at), '') AS model,+ argMax(if(JSONExtractString(metadata, 'source') = 'backfill', 'backfill', 'live'), inserted_at) AS source+FROM {EMBEDDINGS_TABLE}+WHERE product = %(product)s+ AND document_type IN %(document_types)s+ AND rendering = %(rendering)s+ AND team_id IN %(team_ids)s+ AND timestamp >= %(window_start)s+ AND timestamp < %(window_end)s+GROUP BY team_id, document_id+"""+IMPRESSIONS_COLUMNS = ("first_impressed_at","impression_unit_count",diff --git a/products/signals/dags/inbox_ranking/tests/test_dataset.py b/products/signals/dags/inbox_ranking/tests/test_dataset.pyindex 64a75f5977db..973bd7aa5f58 100644--- a/products/signals/dags/inbox_ranking/tests/test_dataset.py+++ b/products/signals/dags/inbox_ranking/tests/test_dataset.py@@ -25,6 +25,7 @@ from products.signals.backend.report_embeddings import EMBEDDING_RENDERING_TITLEfrom products.signals.dags.inbox_ranking import commonfrom products.signals.dags.inbox_ranking.dataset import dag, queriesfrom products.signals.dags.inbox_ranking.dataset.dag import (+ ACTIVITY_SUMMARY_VECTORS_SCHEMA,EMBEDDINGS_SCHEMA,LABELS_SCHEMA,MODEL_DATA_SCHEMA,@@ -33,6 +34,7 @@ from products.signals.dags.inbox_ranking.dataset.dag import (assemble_model_rows,assemble_user_interaction_rows,label_provenance_ok,+ merge_activity_summary_vectors,)from products.signals.dags.inbox_ranking.dataset.queries import (IMPRESSIONS_SQL,@@ -1250,7 +1252,13 @@ def test_an_empty_result_still_writes_the_full_schema(monkeypatch, asset):@pytest.mark.parametrize(- "asset", [dag.inbox_report_embeddings, dag.inbox_report_title_embeddings, dag.inbox_signal_embeddings]+ "asset",+ [+ dag.inbox_report_embeddings,+ dag.inbox_report_title_embeddings,+ dag.inbox_signal_embeddings,+ dag.inbox_activity_summary_vectors,+ ],)@pytest.mark.parametrize("consent_team_ids", [frozenset({2, 7}), frozenset()])def test_embeddings_are_read_only_for_teams_opted_in_to_ai_training(monkeypatch, asset, consent_team_ids):@@ -1263,6 +1271,71 @@ def test_embeddings_are_read_only_for_teams_opted_in_to_ai_training(monkeypatch,assert written["table"].num_rows == 0+def test_activity_summary_vectors_read_the_day_and_parse_the_user_from_the_document_id(monkeypatch):+ generated_at = datetime.datetime(2026, 7, 29, 23, 50)+ written = _run_embeddings_asset(+ monkeypatch,+ dag.inbox_activity_summary_vectors,+ [+ (2, "41:2026-07-29", generated_at, [0.5, 0.25], datetime.datetime(2026, 7, 30, 0, 5), 6, 7, "m", "live"),+ (2, "not-a-user:2026-07-29", generated_at, [0.1], generated_at, 6, 7, "m", "backfill"),+ ],+ )++ assert (written["params"]["window_start"], written["params"]["window_end"]) == (+ datetime.datetime(2026, 7, 29),+ datetime.datetime(2026, 7, 30),+ )+ assert written["params"]["document_types"] == ["activity_summary"]+ assert written["key"] == common.partition_object_key(+ "inbox_ranking", "inbox_activity_summary_vectors", SNAPSHOT_DATE.isoformat()+ )+ assert written["table"].schema == ACTIVITY_SUMMARY_VECTORS_SCHEMA+ assert written["table"].to_pylist() == [+ {+ "team_id": 2,+ "user_id": 41,+ "document_id": "41:2026-07-29",+ "generated_date": SNAPSHOT_DATE,+ "generated_at": generated_at.replace(tzinfo=datetime.UTC),+ "embedding_small": [0.5, 0.25],+ "embedding_inserted_at": datetime.datetime(2026, 7, 30, 0, 5, tzinfo=datetime.UTC),+ "prompt_version": 6,+ "render_version": 7,+ "model": "m",+ "source": "live",+ }+ ]+++def test_an_activity_vectors_re_run_keeps_expired_rows_and_drops_teams_without_consent():+ def vectors(rows: list[tuple[int, str, float]]) -> pa.Table:+ return pa.Table.from_pylist(+ [+ {+ "team_id": team_id,+ "user_id": int(document_id.split(":")[0]),+ "document_id": document_id,+ "generated_date": SNAPSHOT_DATE,+ "embedding_small": [value],+ }+ for team_id, document_id, value in rows+ ],+ schema=ACTIVITY_SUMMARY_VECTORS_SCHEMA,+ )++ existing = vectors([(2, "1:2026-07-29", 0.1), (2, "2:2026-07-29", 0.2), (7, "3:2026-07-29", 0.3)])+ fresh = vectors([(2, "1:2026-07-29", 0.9), (2, "4:2026-07-29", 0.4)])++ merged = merge_activity_summary_vectors(existing, fresh, frozenset({2}))++ assert sorted(zip(merged.column("document_id").to_pylist(), merged.column("embedding_small").to_pylist())) == [+ ("1:2026-07-29", [pytest.approx(0.9)]),+ ("2:2026-07-29", [pytest.approx(0.2)]),+ ("4:2026-07-29", [pytest.approx(0.4)]),+ ]++def test_the_leaf_snapshots_run_in_the_job_without_feeding_the_join():selection = dag.inbox_ranking_dataset_job.selection.resolve([@@ -1276,6 +1349,7 @@ def test_the_leaf_snapshots_run_in_the_job_without_feeding_the_join():dag.inbox_user_report_interactions,dag.inbox_project_vectors,dag.inbox_engagement_export,+ dag.inbox_activity_summary_vectors,])model_data_deps = {key.path[-1] for key in dag.inbox_report_model_data.keys_by_input_name.values()}@@ -1285,6 +1359,7 @@ def test_the_leaf_snapshots_run_in_the_job_without_feeding_the_join():dag.USER_INTERACTIONS_TABLE,dag.PROJECT_VECTORS_TABLE,dag.ENGAGEMENT_EXPORT,+ dag.ACTIVITY_SUMMARY_VECTORS_TABLE,):assert dagster.AssetKey(leaf) in selectionassert leaf not in model_data_depsdiff --git a/products/signals/dags/inbox_ranking/tests/test_training.py b/products/signals/dags/inbox_ranking/tests/test_training.pyindex f30b3bcc0b35..23ec41e7ac83 100644--- a/products/signals/dags/inbox_ranking/tests/test_training.py+++ b/products/signals/dags/inbox_ranking/tests/test_training.py@@ -87,6 +87,7 @@ from products.signals.dags.inbox_ranking.training.dag import (from products.signals.dags.inbox_ranking.training.examples import (STATE_LAG_LIMIT,Snapshot,+ activity_vector_as_of,assemble_snapshot,birth_day_positives,build_examples,@@ -474,6 +475,47 @@ def test_holdout_mask_cuts_by_report_not_by_row():assert holdout_mask(examples, holdout_days=7).tolist() == [False, False, True, True]+ACTIVITY_AS_OF = datetime.datetime(2026, 8, 10, 12, 0, tzinfo=datetime.UTC)+++ "generated_hours_before,expected_hours_before",+ [+ ([], None),+ ([72, 24, 48], 24),+ # A summary built after the scoring moment did not exist yet.+ ([24, -1], 24),+ ([24 * 8], None),+ ],+ ids=["no_vector_yet", "newest_of_several", "vector_after_the_moment", "stale_vector"],+)+def test_activity_vector_as_of_reads_the_newest_vector_before_the_moment(generated_hours_before, expected_hours_before):+ generated_ats = [ACTIVITY_AS_OF - datetime.timedelta(hours=hours) for hours in generated_hours_before]+ vectors = pd.DataFrame(+ {+ "team_id": [1] * len(generated_ats) + [2, 1],+ "user_id": [7] * len(generated_ats) + [7, 8],+ # Another team's user 7 and another user of team 1, both fresher than any of user 7's vectors.+ "generated_at": pd.to_datetime(+ generated_ats + [ACTIVITY_AS_OF - datetime.timedelta(hours=1)] * 2, utc=True+ ),+ "embedding_small": [[float(hours)] for hours in generated_hours_before] + [[-1.0], [-2.0]],+ }+ )++ found = activity_vector_as_of(+ vectors, team_id=1, user_id=7, as_of=ACTIVITY_AS_OF, max_age=datetime.timedelta(days=7)+ )++ if expected_hours_before is None:+ assert found is None+ else:+ assert found is not None+ assert found.embedding == [float(expected_hours_before)]+ assert found.age == datetime.timedelta(hours=expected_hours_before)+ assert found.generated_at == ACTIVITY_AS_OF - found.age++class _FakeClient:def __init__(self) -> None:self.calls: list[dict[str, Any]] = []diff --git a/products/signals/dags/inbox_ranking/training/examples.py b/products/signals/dags/inbox_ranking/training/examples.pyindex 4dde0ccb2b7f..3613749033ef 100644--- a/products/signals/dags/inbox_ranking/training/examples.py+++ b/products/signals/dags/inbox_ranking/training/examples.py@@ -419,6 +419,35 @@ def _with_features(return pd.concat(frames, ignore_index=True)[columns]+@frozen+class ActivityVector:+ embedding: list[float]+ generated_at: datetime.datetime+ age: datetime.timedelta+++def activity_vector_as_of(+ vectors: pd.DataFrame, *, team_id: int, user_id: int, as_of: datetime.datetime, max_age: datetime.timedelta+) -> ActivityVector | None:+ """The activity summary vector user `user_id` of team `team_id` had at `as_of`, read from the union+ of `inbox_activity_summary_vectors` partitions.++ That is the newest vector generated strictly before `as_of`, so a summary built after the moment+ never leaks into it. None when the user has no such vector, or when the newest one is older than+ `max_age`: a summary that stale no longer describes what the user does."""+ known = vectors.loc[+ (vectors["team_id"] == team_id) & (vectors["user_id"] == user_id) & (vectors["generated_at"] < as_of)+ ]+ if known.empty:+ return None+ latest = known.loc[known["generated_at"].idxmax()]+ generated_at = pd.Timestamp(latest["generated_at"]).to_pydatetime()+ age = as_of - generated_at+ if age > max_age:+ return None+ return ActivityVector(embedding=list(latest["embedding_small"]), generated_at=generated_at, age=age)++def holdout_mask(examples: pd.DataFrame, holdout_days: int) -> pd.Series:"""True for every row of the reports created in the last `holdout_days` of the example set.Cut by report so a report's snapshot rows never straddle train and holdout."""