Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

feat(signals): archive daily activity summary vectors for ranking training (#124484)

posthog[bot]297a240a4646
diff --git a/posthog/dags/slack_alerts.py b/posthog/dags/slack_alerts.py
index d1025e6f42c7..60729894b628 100644
--- a/posthog/dags/slack_alerts.py
+++ b/posthog/dags/slack_alerts.py
@@ -115,6 +115,7 @@ ASSET_NAME_PREFIX_OWNERS: tuple[tuple[str, JobOwners], ...] = (
("inbox_signal_", JobOwners.TEAM_SELF_DRIVING),
("inbox_ranking_", JobOwners.TEAM_SELF_DRIVING),
("inbox_user_", JobOwners.TEAM_SELF_DRIVING),
+ ("inbox_activity_", JobOwners.TEAM_SELF_DRIVING),
)
diff --git a/posthog/dags/tests/test_slack_alerts.py b/posthog/dags/tests/test_slack_alerts.py
index e3fd16958358..58db22c7f9f5 100644
--- a/posthog/dags/tests/test_slack_alerts.py
+++ b/posthog/dags/tests/test_slack_alerts.py
@@ -92,6 +92,7 @@ class TestSlackAlertsRouting:
("inbox_signal_embeddings",),
("inbox_ranking_model_candidate",),
("inbox_user_report_interactions",),
+ ("inbox_activity_summary_vectors",),
]
)
def test_asset_job_with_inbox_steps_routes_to_self_driving(self, step):
diff --git a/products/signals/dags/inbox_ranking/AGENTS.md b/products/signals/dags/inbox_ranking/AGENTS.md
index e88cedefc602..d819483fe12d 100644
--- a/products/signals/dags/inbox_ranking/AGENTS.md
+++ b/products/signals/dags/inbox_ranking/AGENTS.md
@@ -10,6 +10,7 @@ Read `README.md` first for what the dataset is and how partitions behave. This f
- `inbox_project_vectors` is a **leaf** with its own `PROJECT_VECTORS_SCHEMA_VERSION`: keep it out of `inbox_report_model_data`'s `deps`. Its `deps=[EMBEDDINGS_TABLE, LABELS_TABLE]` keep it after the embeddings snapshot, so the two vector-heavy steps do not share the run pod's memory. `project_vectors` takes a report into a vector only when its vector landed and its UI engagement happened strictly before `as_of`; never widen that to the latest snapshot value. A report born on day D reads the dt=D−1 vectors (see the README).
- `inbox_engagement_export` is a **leaf** with its own `ENGAGEMENT_EXPORT_SCHEMA_VERSION`, read by another region. It writes to the mirror store only under `serving_engagement_key`, because the mirror's write grant covers only `<prefix>/serving/`. Keep its columns exactly `LABEL_COLUMNS` and its rows limited to reports with no `SignalReport` in this region: no team ids, counts, text or vectors leave the region. A mirror failure logs and must not fail the asset. Keep the local-share guard before the write: when the local report lookup finds no local report (`no_local_reports`) or a share below `ENGAGEMENT_EXPORT_MIN_LOCAL_SHARE` (`local_share_collapsed`), the asset writes nothing and still succeeds. Its `inbox_ranking_engagement_export_published` event carries counts only, never a report id or a team id.
- `inbox_mirror_project_vectors` (`mirror/dag.py`) is the only inbox-ranking asset that registers on EU (`is_inbox_ranking_mirror_registered()`). It reads this region's report vectors through `report_embeddings_table` and the engagement export through `serving_engagement_key`, and writes only to this region's object store at `serving_project_vectors_key`. Keep it on the same `project_vectors()` and the same consent rule as `inbox_project_vectors`, so the two regions build their vectors the same way. A missing, too old or unknown-schema export fails the asset rather than writing vectors without engagement.
+- `inbox_activity_summary_vectors` is a **leaf** with its own `ACTIVITY_SUMMARY_VECTORS_SCHEMA_VERSION`: keep it out of `inbox_report_model_data`'s `deps`, and never select the summary text (`content`). It is the only durable copy of the activity summary vectors once the source TTL passes, so a re-run must keep the rows the source no longer returns (`merge_activity_summary_vectors`). Training reads a user's vector at a moment through `activity_vector_as_of`, never the newest vector of the user.
- Register new jobs/schedules in `posthog/dags/locations/signals.py` (or `signals_ml.py` for a torch-dependent dag), keep the EU gate (`is_inbox_ranking_registered()`), and remember the ci-dagster paths filter must cover every module a dag file imports (`uv run .github/scripts/check-dagster-paths.py`).
## Training dag specifics
@@ -46,7 +47,7 @@ Read `README.md` first for what the dataset is and how partitions behave. This f
## Invariants — do not break
-- `dt=` partitions are **immutable snapshots** with deterministic object keys; the only mutation ever applied is an idempotent re-run of the same partition. The exception is `inbox_signal_embeddings`, an emission log whose partition holds only that day's inserts — see the README's signal-grain section before touching it. Its re-run must stay **additive** (union with the existing object): the source drops rows it already archived, so a plain overwrite destroys history that exists nowhere else.
+- `dt=` partitions are **immutable snapshots** with deterministic object keys; the only mutation ever applied is an idempotent re-run of the same partition. The exception is `inbox_signal_embeddings`, an emission log whose partition holds only that day's inserts — see the README's signal-grain section before touching it. Its re-run must stay **additive** (union with the existing object): the source drops rows it already archived, so a plain overwrite destroys history that exists nowhere else. `inbox_activity_summary_vectors` is the same kind of log (dt=D holds the vectors generated during D), and its re-run keeps every row the source no longer returns.
- Label columns are **cumulative from `LABELS_EPOCH`**; never bake a maturity window or a rolling time bound into the SQL (the saved `inbox_ranking_*` views roll 90 days — that is exactly why their SQL is inlined here with explicit bounds instead of reused).
- Every cumulative outcome count ships with the column holding the moment its first counted event arrived, declared as a pair in `OUTCOME_FIRST_EVENT_COLUMNS` (a test fails on an unpaired count). A count alone dates an outcome only to the whole cumulative window, so a horizon label or a time-to-outcome read has nothing but a proxy without that moment. The timestamp must use the same predicate as its count, or the pair describes different events.
- Every `STATUS_SQL` aggregate reads only the latest transition's tenant (`event_team_id = latest_event_team_id`).
diff --git a/products/signals/dags/inbox_ranking/README.md b/products/signals/dags/inbox_ranking/README.md
index f346cb738489..db8dddd608ef 100644
--- a/products/signals/dags/inbox_ranking/README.md
+++ b/products/signals/dags/inbox_ranking/README.md
@@ -27,7 +27,7 @@ Registered via `posthog/dags/locations/signals.py` (US only) and loaded locally
## The dataset dag
-`inbox_ranking_dataset_job` runs daily at 02:30 UTC (schedule default: running on prod US, stopped everywhere else — including hosted DEV and E2E, which have no dogfood project to read labels from) and builds nine assets on one daily partition, each a Parquet object in S3:
+`inbox_ranking_dataset_job` runs daily at 02:30 UTC (schedule default: running on prod US, stopped everywhere else — including hosted DEV and E2E, which have no dogfood project to read labels from) and builds ten assets on one daily partition, each a Parquet object in S3:
```text
s3://<bucket>/<prefix>/
@@ -41,7 +41,8 @@ s3://<bucket>/<prefix>/
├── inbox_report_title_embeddings/v1/dt=YYYY-MM-DD/ # the same shape, for the title-only rendering
├── inbox_report_reviewers/v1/dt=YYYY-MM-DD/ # one row per suggested reviewer of each spine report
├── inbox_user_report_interactions/v1/dt=YYYY-MM-DD/ # cumulative interactions per (report, person)
-└── inbox_project_vectors/v1/dt=YYYY-MM-DD/ # per-team mean report vectors of past engagement, by kind
+├── inbox_project_vectors/v1/dt=YYYY-MM-DD/ # per-team mean report vectors of past engagement, by kind
+└── inbox_activity_summary_vectors/v1/dt=YYYY-MM-DD/ # one row per activity summary vector generated during the day
```
The first four are report grain and land in one table. `inbox_signal_embeddings` is signal grain, feeds the group-level model, and is read on its own — training joins it to `inbox_report_model_data` by `report_id`.
@@ -164,6 +165,20 @@ Point-in-time caveats, per source:
- The count check remains as a backstop: a union can only grow, so a smaller result means the merge itself is broken and the write is refused. Row counts are stamped in S3 object metadata at write time to make that check a `head_object`. A deliberate shrink still means deleting the object by hand.
- Signal text never leaves ClickHouse: the query selects the vector and the structured metadata (weight, source, match graph), not `content` or the free-text metadata fields.
+### Activity summary vectors
+
+`inbox_activity_summary_vectors` is a (team, user, day) leaf with its own `ACTIVITY_SUMMARY_VECTORS_SCHEMA_VERSION`.
+It is the durable history of the per-user activity summary vectors, for a ranking model that reads a user vector next to the report vector.
+Each successful activity summary build writes one vector per (user, day) to the document embeddings table (`document_id` = `"{user_id}:{date}"`, `timestamp` = the summary's `generated_at`).
+That table drops a row 3 months after its `timestamp`, and `UserActivitySummaryVersion` keeps only the newest versions of a pair, so a vector that this asset does not copy in time cannot be rebuilt.
+
+- `dt=D` holds only the vectors whose `timestamp` falls in D, one row per `(team_id, document_id)`, the newest `inserted_at` winning. It is not a cumulative snapshot.
+- The columns are `team_id`, `user_id`, `document_id`, `generated_date`, `generated_at`, `embedding_small` (small-1536, the report vectors' space), `embedding_inserted_at`, `prompt_version`, `render_version`, `model` and `source` (`live`, or `backfill` for a summary built as of a past date). No summary text leaves ClickHouse.
+- Training reads the union of partitions and calls `activity_vector_as_of` (`training/examples.py`) for the vector a user had at a moment: the newest one generated strictly before it, or nothing when that one is older than a staleness cap.
+- A re-run replaces the rows the source still returns and keeps the rows it no longer returns, so a re-run after the TTL does not erase history. It drops the rows of teams without training consent.
+- **It is a leaf.** Nothing joins it, so a failure here does not fail the training table. A day of vectors is small, so it has no ordering dependency.
+- Backfill: materialize every partition from the first activity summary build that is still inside the source TTL. Repeat for the matching partitions after a backfill of past-dated summaries, before those vectors expire.
+
## The training dag
`inbox_ranking_training_job` runs daily at 05:00 UTC on the same partition definition (gated like the dataset job).
@@ -474,7 +489,7 @@ The dags train only on reports from organizations whose `Organization.is_ai_trai
`consent.training_consent_team_ids()` returns the consenting teams that hold an inbox report, and every asset reads it once per run.
- `inbox_report_state` keeps only the spine reports of those teams. A label event that names another team's report does not pull it back in. That report's labels still land in `inbox_report_model_data` as a label-only row, which training skips.
-- `inbox_report_embeddings`, `inbox_report_title_embeddings` and `inbox_signal_embeddings` read only those teams, through `team_id IN (...)` in the ClickHouse query.
+- `inbox_report_embeddings`, `inbox_report_title_embeddings`, `inbox_signal_embeddings` and `inbox_activity_summary_vectors` read only those teams, through `team_id IN (...)` in the ClickHouse query.
- The `nn` examples asset drops every state row whose `report_team_id` is not in the current set, before it builds examples. The lookback window reaches partitions written before an organization opted out, so this filter is what makes an opt-out take effect on the next training run.
The filter follows the current setting, so an organization that opts back in is trained on again.
diff --git a/products/signals/dags/inbox_ranking/dataset/dag.py b/products/signals/dags/inbox_ranking/dataset/dag.py
index da94186989b0..e4c88bbc2e22 100644
--- a/products/signals/dags/inbox_ranking/dataset/dag.py
+++ b/products/signals/dags/inbox_ranking/dataset/dag.py
@@ -1,6 +1,6 @@
"""Daily modeling dataset for the Self-driving Inbox report-ranking model.
-Nine assets on one daily partition, each writing Parquet under the configured S3 prefix:
+Ten assets on one daily partition, each writing Parquet under the configured S3 prefix:
inbox_report_state/v1/dt=D/ Postgres spine + report-state columns
inbox_report_embeddings/v1/dt=D/ report_id -> small-1536 vector as of snapshot end
@@ -11,6 +11,7 @@ Nine assets on one daily partition, each writing Parquet under the configured S3
inbox_report_reviewers/v1/dt=D/ one row per suggested reviewer of each spine report
inbox_user_report_interactions/v1/dt=D/ cumulative interactions at (report, person) grain
inbox_project_vectors/v1/dt=D/ per-team mean report vectors of past engagement, by kind
+ inbox_activity_summary_vectors/v1/dt=D/ one row per activity summary vector generated during D
inbox_engagement_export writes no dataset object. It sends the dt=D engagement moments of the other
region's reports to the serving mirror store, for that region's project vectors.
@@ -22,6 +23,8 @@ is a report-grain leaf: nothing joins it, and no model family trains on it. inbo
it to the report tables by report_id. inbox_user_report_interactions is a (report, person) grain
leaf with its own schema version, so nothing joins it either. inbox_project_vectors is a
(team, kind) leaf with its own schema version, built from the embeddings and labels snapshots.
+inbox_activity_summary_vectors is a (team, user, day) leaf with its own schema version: the durable
+copy of the per-user activity summary vectors, which the source table drops after its 3-month TTL.
Partition dt=D is a full snapshot of the eligible report inventory (promoted or ever-labeled),
with every label aggregate bounded `event_time < D+1 00:00 UTC`. Label columns are cumulative,
@@ -81,6 +84,10 @@ from posthog.dags.common import dagster_tags
from posthog.dataclasses import frozen
from posthog.models import OrganizationMembership, Team, User
+from products.signals.backend.activity_summary.tick import (
+ EMBEDDING_DOCUMENT_TYPE as ACTIVITY_SUMMARY_DOCUMENT_TYPE,
+ EMBEDDING_RENDERING as ACTIVITY_SUMMARY_RENDERING,
+)
from products.signals.backend.models import (
SignalReport,
SignalReportAction,
@@ -139,6 +146,8 @@ from products.signals.dags.inbox_ranking.dataset.project_vectors import (
teams_with_history,
)
from products.signals.dags.inbox_ranking.dataset.queries import (
+ ACTIVITY_SUMMARY_VECTORS_QUERY_SETTINGS,
+ ACTIVITY_SUMMARY_VECTORS_SQL,
LABEL_DEFAULTS,
LABEL_STREAMS,
LABELED_REPORT_IDS_SQL,
@@ -179,6 +188,7 @@ REVIEWERS_TABLE = "inbox_report_reviewers"
USER_INTERACTIONS_TABLE = "inbox_user_report_interactions"
PROJECT_VECTORS_TABLE = "inbox_project_vectors"
ENGAGEMENT_EXPORT = "inbox_engagement_export"
+ACTIVITY_SUMMARY_VECTORS_TABLE = "inbox_activity_summary_vectors"
# The reviewers asset is a leaf with its own columns, so its schema moves apart from
# FEATURE_SCHEMA_VERSION. The same rule applies: additive nullable columns bump it.
@@ -186,6 +196,11 @@ REVIEWERS_SCHEMA_VERSION = 1
# The engagement export is read in another region, so its shape moves apart from every local snapshot.
ENGAGEMENT_EXPORT_SCHEMA_VERSION = 1
+# The activity summary vectors are a (team, user, day) leaf, so their shape moves apart too.
+ACTIVITY_SUMMARY_VECTORS_SCHEMA_VERSION = 1
+# Every document type that carries an activity summary vector. A backfilled type joins this tuple,
+# and its rows say so through `source`.
+ACTIVITY_SUMMARY_DOCUMENT_TYPES = (ACTIVITY_SUMMARY_DOCUMENT_TYPE,)
# Most labels rows belong to this region's reports. A share below this floor means the local report
# lookup lost rows, and the export would send this region's report ids to the other region.
ENGAGEMENT_EXPORT_MIN_LOCAL_SHARE = 0.25
@@ -289,6 +304,21 @@ _REVIEWER_FIELDS: list[tuple[str, pa.DataType]] = [
]
REVIEWERS_SCHEMA = pa.schema(_REVIEWER_FIELDS)
+_ACTIVITY_SUMMARY_VECTOR_FIELDS: list[tuple[str, pa.DataType]] = [
+ ("team_id", pa.int64()),
+ ("user_id", pa.int64()),
+ ("document_id", pa.string()),
+ ("generated_date", pa.date32()),
+ ("generated_at", _TIMESTAMP),
+ ("embedding_small", pa.list_(pa.float32())),
+ ("embedding_inserted_at", _TIMESTAMP),
+ ("prompt_version", pa.int32()),
+ ("render_version", pa.int32()),
+ ("model", pa.string()),
+ ("source", pa.string()),
+]
+ACTIVITY_SUMMARY_VECTORS_SCHEMA = pa.schema(_ACTIVITY_SUMMARY_VECTOR_FIELDS)
+
# What makes one emission distinct from another, so a re-run can tell a row it already archived from
# a genuinely new one. embedding_inserted_at is the version: the source keys a signal by
# (team_id, document_id) and distinguishes its versions by inserted_at.
@@ -1481,6 +1511,135 @@ def inbox_project_vectors(context: dagster.AssetExecutionContext) -> None:
)
+def activity_summary_vectors_table(partition_key: str, consent_team_ids: Collection[int]) -> tuple[pa.Table, int]:
+ """The activity summary vectors generated during dt=D for the teams in `consent_team_ids`, one row
+ per document, and the number of rows skipped because their document_id names no user."""
+ window_start, window_end = snapshot_bounds(partition_key)
+ results = (
+ cast(
+ list[tuple[Any, ...]],
+ sync_execute(
+ ACTIVITY_SUMMARY_VECTORS_SQL,
+ {
+ "product": EMBEDDING_PRODUCT,
+ "document_types": list(ACTIVITY_SUMMARY_DOCUMENT_TYPES),
+ "rendering": ACTIVITY_SUMMARY_RENDERING,
+ "window_start": window_start.replace(tzinfo=None),
+ "window_end": window_end.replace(tzinfo=None),
+ "team_ids": sorted(consent_team_ids),
+ },
+ settings=ACTIVITY_SUMMARY_VECTORS_QUERY_SETTINGS,
+ workload=etl_workload(),
+ ch_user=ClickHouseUser.SIGNALS,
+ )
+ or [],
+ )
+ if consent_team_ids
+ else []
+ )
+
+ columns: dict[str, list[Any]] = {name: [] for name, _type in _ACTIVITY_SUMMARY_VECTOR_FIELDS}
+ malformed = 0
+ for (
+ team_id,
+ document_id,
+ generated_at,
+ embedding,
+ inserted_at,
+ prompt_version,
+ render_version,
+ model,
+ source,
+ ) in results:
+ # A malformed id is skipped and counted rather than raised: failing the asset would lose
+ # the whole day's vectors, and the source drops them after its TTL.
+ user_part, _, _ = str(document_id).partition(":")
+ try:
+ user_id = int(user_part)
+ except ValueError:
+ malformed += 1
+ continue
+ generated_at_utc = cast(datetime.datetime, ensure_utc(generated_at))
+ columns["team_id"].append(int(team_id))
+ columns["user_id"].append(user_id)
+ columns["document_id"].append(str(document_id))
+ columns["generated_date"].append(generated_at_utc.date())
+ columns["generated_at"].append(generated_at_utc)
+ columns["embedding_small"].append(list(embedding))
+ columns["embedding_inserted_at"].append(ensure_utc(inserted_at))
+ columns["prompt_version"].append(prompt_version)
+ columns["render_version"].append(render_version)
+ columns["model"].append(model)
+ columns["source"].append(source)
+ return pa.Table.from_pydict(columns, schema=ACTIVITY_SUMMARY_VECTORS_SCHEMA), malformed
+
+
+def merge_activity_summary_vectors(existing: pa.Table, fresh: pa.Table, consent_team_ids: Collection[int]) -> pa.Table:
+ """The rows a re-run writes over a partition it already holds.
+
+ A fresh row replaces the existing row of its document, so a re-embedded vector is updated and
+ each document keeps one row. An existing row whose document the fresh scan no longer returns
+ stays, because the source TTL or a merge removed it and this partition is the only copy left. A
+ team without training consent now loses its rows in both cases."""
+ fresh_documents = set(
+ zip(fresh.column("team_id").to_pylist(), fresh.column("document_id").to_pylist(), strict=True)
+ )
+ keep = pa.array(
+ [
+ team_id in consent_team_ids and (team_id, document_id) not in fresh_documents
+ for team_id, document_id in zip(
+ existing.column("team_id").to_pylist(), existing.column("document_id").to_pylist(), strict=True
+ )
+ ],
+ type=pa.bool_(),
+ )
+ return pa.concat_tables([fresh, existing.filter(keep)])
+
+
[email protected](name=ACTIVITY_SUMMARY_VECTORS_TABLE, **COMMON_ASSET_KWARGS)
+def inbox_activity_summary_vectors(context: dagster.AssetExecutionContext) -> None:
+ """One row per activity summary vector generated during dt=D, for user vectors in ranking training.
+
+ The source table drops a vector 3 months after its timestamp, and `UserActivitySummaryVersion`
+ keeps only the newest versions of a pair, so this partition is the durable history: a vector
+ that is not copied before it expires cannot be rebuilt. Partition D holds only the vectors
+ generated during D, not a cumulative snapshot. Training reads the union of partitions and takes
+ the vector a user had at a moment through `activity_vector_as_of`.
+
+ A leaf: nothing joins it, and it carries no summary text. A re-run replaces the rows the source
+ still returns and keeps the rest (`merge_activity_summary_vectors`).
+ """
+ if skip_unconfigured(context):
+ return
+ tag_dagster_queries(context, query_type="inbox_ranking_activity_summary_vectors")
+ partition_key = context.partition_key
+ consent_team_ids = training_consent_team_ids()
+ fresh, malformed = activity_summary_vectors_table(partition_key, consent_team_ids)
+
+ bucket = dataset_bucket()
+ key = partition_object_key(settings.INBOX_RANKING_DATASET_S3_PREFIX, ACTIVITY_SUMMARY_VECTORS_TABLE, partition_key)
+ client = s3_client()
+ existing = read_parquet_if_exists(client, bucket, key)
+ table = fresh if existing is None else merge_activity_summary_vectors(existing, fresh, consent_team_ids)
+ write_parquet(client, bucket, key, table, schema_version=ACTIVITY_SUMMARY_VECTORS_SCHEMA_VERSION)
+
+ team_ids = table.column("team_id").to_pylist()
+ user_ids = table.column("user_id").to_pylist()
+ sources = table.column("source").to_pylist()
+ context.add_output_metadata(
+ {
+ "rows": dagster.MetadataValue.int(table.num_rows),
+ "teams": dagster.MetadataValue.int(len(set(team_ids))),
+ "users": dagster.MetadataValue.int(len(set(zip(team_ids, user_ids, strict=True)))),
+ **{f"{source}_rows": dagster.MetadataValue.int(sources.count(source)) for source in ("live", "backfill")},
+ "carried_over": dagster.MetadataValue.int(table.num_rows - fresh.num_rows),
+ "malformed_document_ids": dagster.MetadataValue.int(malformed),
+ "training_consent_teams": dagster.MetadataValue.int(len(consent_team_ids)),
+ "s3_key": dagster.MetadataValue.text(f"s3://{bucket}/{key}"),
+ }
+ )
+
+
def remote_engagement_rows(labels: pa.Table, local_report_ids: Collection[str]) -> pa.Table:
"""The label rows of reports that do not exist in this region's Postgres.
@@ -1772,6 +1931,7 @@ inbox_ranking_dataset_job = dagster.define_asset_job(
USER_INTERACTIONS_TABLE,
PROJECT_VECTORS_TABLE,
ENGAGEMENT_EXPORT,
+ ACTIVITY_SUMMARY_VECTORS_TABLE,
],
# The seven label streams run sequentially and each may take its full 600s query timeout, so an
# hour left a slow-but-valid pass no room for the join, the S3 writes, or an asset retry — and
diff --git a/products/signals/dags/inbox_ranking/dataset/queries.py b/products/signals/dags/inbox_ranking/dataset/queries.py
index 79de627344ed..d534df3c10c6 100644
--- a/products/signals/dags/inbox_ranking/dataset/queries.py
+++ b/products/signals/dags/inbox_ranking/dataset/queries.py
@@ -275,6 +275,38 @@ WHERE product = %(product)s
AND inserted_at < %(window_end)s
"""
+# Activity summary documents: one vector per (team, user, generated day), with document_id
+# "{user_id}:{date}" and timestamp = the summary's generated_at. The window is on timestamp, not on
+# inserted_at, so a partition holds the vectors generated during D whatever the embedding lag. The
+# sort key leads with (team_id, toDate(timestamp)), so this reads one day of the consenting teams.
+# A rebuild of the same day re-emits under the same document_id, and argMax keeps the newest version
+# when a merge has not removed the older one yet. The summary text (`content`) is never selected.
+ACTIVITY_SUMMARY_VECTORS_QUERY_SETTINGS: dict[str, int] = {
+ "max_execution_time": 600,
+ "max_memory_usage": 10 * 1024**3,
+}
+
+ACTIVITY_SUMMARY_VECTORS_SQL = f"""
+SELECT
+ team_id,
+ document_id,
+ argMax(timestamp, inserted_at) AS generated_at,
+ argMax(embedding, inserted_at) AS embedding,
+ max(inserted_at) AS embedding_inserted_at,
+ nullIf(argMax(JSONExtractInt(metadata, 'prompt_version'), inserted_at), 0) AS prompt_version,
+ nullIf(argMax(JSONExtractInt(metadata, 'render_version'), inserted_at), 0) AS render_version,
+ nullIf(argMax(JSONExtractString(metadata, 'model'), inserted_at), '') AS model,
+ argMax(if(JSONExtractString(metadata, 'source') = 'backfill', 'backfill', 'live'), inserted_at) AS source
+FROM {EMBEDDINGS_TABLE}
+WHERE product = %(product)s
+ AND document_type IN %(document_types)s
+ AND rendering = %(rendering)s
+ AND team_id IN %(team_ids)s
+ AND timestamp >= %(window_start)s
+ AND timestamp < %(window_end)s
+GROUP BY team_id, document_id
+"""
+
IMPRESSIONS_COLUMNS = (
"first_impressed_at",
"impression_unit_count",
diff --git a/products/signals/dags/inbox_ranking/tests/test_dataset.py b/products/signals/dags/inbox_ranking/tests/test_dataset.py
index 64a75f5977db..973bd7aa5f58 100644
--- a/products/signals/dags/inbox_ranking/tests/test_dataset.py
+++ b/products/signals/dags/inbox_ranking/tests/test_dataset.py
@@ -25,6 +25,7 @@ from products.signals.backend.report_embeddings import EMBEDDING_RENDERING_TITLE
from products.signals.dags.inbox_ranking import common
from products.signals.dags.inbox_ranking.dataset import dag, queries
from products.signals.dags.inbox_ranking.dataset.dag import (
+ ACTIVITY_SUMMARY_VECTORS_SCHEMA,
EMBEDDINGS_SCHEMA,
LABELS_SCHEMA,
MODEL_DATA_SCHEMA,
@@ -33,6 +34,7 @@ from products.signals.dags.inbox_ranking.dataset.dag import (
assemble_model_rows,
assemble_user_interaction_rows,
label_provenance_ok,
+ merge_activity_summary_vectors,
)
from products.signals.dags.inbox_ranking.dataset.queries import (
IMPRESSIONS_SQL,
@@ -1250,7 +1252,13 @@ def test_an_empty_result_still_writes_the_full_schema(monkeypatch, asset):
@pytest.mark.parametrize(
- "asset", [dag.inbox_report_embeddings, dag.inbox_report_title_embeddings, dag.inbox_signal_embeddings]
+ "asset",
+ [
+ dag.inbox_report_embeddings,
+ dag.inbox_report_title_embeddings,
+ dag.inbox_signal_embeddings,
+ dag.inbox_activity_summary_vectors,
+ ],
)
@pytest.mark.parametrize("consent_team_ids", [frozenset({2, 7}), frozenset()])
def test_embeddings_are_read_only_for_teams_opted_in_to_ai_training(monkeypatch, asset, consent_team_ids):
@@ -1263,6 +1271,71 @@ def test_embeddings_are_read_only_for_teams_opted_in_to_ai_training(monkeypatch,
assert written["table"].num_rows == 0
+def test_activity_summary_vectors_read_the_day_and_parse_the_user_from_the_document_id(monkeypatch):
+ generated_at = datetime.datetime(2026, 7, 29, 23, 50)
+ written = _run_embeddings_asset(
+ monkeypatch,
+ dag.inbox_activity_summary_vectors,
+ [
+ (2, "41:2026-07-29", generated_at, [0.5, 0.25], datetime.datetime(2026, 7, 30, 0, 5), 6, 7, "m", "live"),
+ (2, "not-a-user:2026-07-29", generated_at, [0.1], generated_at, 6, 7, "m", "backfill"),
+ ],
+ )
+
+ assert (written["params"]["window_start"], written["params"]["window_end"]) == (
+ datetime.datetime(2026, 7, 29),
+ datetime.datetime(2026, 7, 30),
+ )
+ assert written["params"]["document_types"] == ["activity_summary"]
+ assert written["key"] == common.partition_object_key(
+ "inbox_ranking", "inbox_activity_summary_vectors", SNAPSHOT_DATE.isoformat()
+ )
+ assert written["table"].schema == ACTIVITY_SUMMARY_VECTORS_SCHEMA
+ assert written["table"].to_pylist() == [
+ {
+ "team_id": 2,
+ "user_id": 41,
+ "document_id": "41:2026-07-29",
+ "generated_date": SNAPSHOT_DATE,
+ "generated_at": generated_at.replace(tzinfo=datetime.UTC),
+ "embedding_small": [0.5, 0.25],
+ "embedding_inserted_at": datetime.datetime(2026, 7, 30, 0, 5, tzinfo=datetime.UTC),
+ "prompt_version": 6,
+ "render_version": 7,
+ "model": "m",
+ "source": "live",
+ }
+ ]
+
+
+def test_an_activity_vectors_re_run_keeps_expired_rows_and_drops_teams_without_consent():
+ def vectors(rows: list[tuple[int, str, float]]) -> pa.Table:
+ return pa.Table.from_pylist(
+ [
+ {
+ "team_id": team_id,
+ "user_id": int(document_id.split(":")[0]),
+ "document_id": document_id,
+ "generated_date": SNAPSHOT_DATE,
+ "embedding_small": [value],
+ }
+ for team_id, document_id, value in rows
+ ],
+ schema=ACTIVITY_SUMMARY_VECTORS_SCHEMA,
+ )
+
+ existing = vectors([(2, "1:2026-07-29", 0.1), (2, "2:2026-07-29", 0.2), (7, "3:2026-07-29", 0.3)])
+ fresh = vectors([(2, "1:2026-07-29", 0.9), (2, "4:2026-07-29", 0.4)])
+
+ merged = merge_activity_summary_vectors(existing, fresh, frozenset({2}))
+
+ assert sorted(zip(merged.column("document_id").to_pylist(), merged.column("embedding_small").to_pylist())) == [
+ ("1:2026-07-29", [pytest.approx(0.9)]),
+ ("2:2026-07-29", [pytest.approx(0.2)]),
+ ("4:2026-07-29", [pytest.approx(0.4)]),
+ ]
+
+
def test_the_leaf_snapshots_run_in_the_job_without_feeding_the_join():
selection = dag.inbox_ranking_dataset_job.selection.resolve(
[
@@ -1276,6 +1349,7 @@ def test_the_leaf_snapshots_run_in_the_job_without_feeding_the_join():
dag.inbox_user_report_interactions,
dag.inbox_project_vectors,
dag.inbox_engagement_export,
+ dag.inbox_activity_summary_vectors,
]
)
model_data_deps = {key.path[-1] for key in dag.inbox_report_model_data.keys_by_input_name.values()}
@@ -1285,6 +1359,7 @@ def test_the_leaf_snapshots_run_in_the_job_without_feeding_the_join():
dag.USER_INTERACTIONS_TABLE,
dag.PROJECT_VECTORS_TABLE,
dag.ENGAGEMENT_EXPORT,
+ dag.ACTIVITY_SUMMARY_VECTORS_TABLE,
):
assert dagster.AssetKey(leaf) in selection
assert leaf not in model_data_deps
diff --git a/products/signals/dags/inbox_ranking/tests/test_training.py b/products/signals/dags/inbox_ranking/tests/test_training.py
index f30b3bcc0b35..23ec41e7ac83 100644
--- a/products/signals/dags/inbox_ranking/tests/test_training.py
+++ b/products/signals/dags/inbox_ranking/tests/test_training.py
@@ -87,6 +87,7 @@ from products.signals.dags.inbox_ranking.training.dag import (
from products.signals.dags.inbox_ranking.training.examples import (
STATE_LAG_LIMIT,
Snapshot,
+ activity_vector_as_of,
assemble_snapshot,
birth_day_positives,
build_examples,
@@ -474,6 +475,47 @@ def test_holdout_mask_cuts_by_report_not_by_row():
assert holdout_mask(examples, holdout_days=7).tolist() == [False, False, True, True]
+ACTIVITY_AS_OF = datetime.datetime(2026, 8, 10, 12, 0, tzinfo=datetime.UTC)
+
+
+ "generated_hours_before,expected_hours_before",
+ [
+ ([], None),
+ ([72, 24, 48], 24),
+ # A summary built after the scoring moment did not exist yet.
+ ([24, -1], 24),
+ ([24 * 8], None),
+ ],
+ ids=["no_vector_yet", "newest_of_several", "vector_after_the_moment", "stale_vector"],
+)
+def test_activity_vector_as_of_reads_the_newest_vector_before_the_moment(generated_hours_before, expected_hours_before):
+ generated_ats = [ACTIVITY_AS_OF - datetime.timedelta(hours=hours) for hours in generated_hours_before]
+ vectors = pd.DataFrame(
+ {
+ "team_id": [1] * len(generated_ats) + [2, 1],
+ "user_id": [7] * len(generated_ats) + [7, 8],
+ # Another team's user 7 and another user of team 1, both fresher than any of user 7's vectors.
+ "generated_at": pd.to_datetime(
+ generated_ats + [ACTIVITY_AS_OF - datetime.timedelta(hours=1)] * 2, utc=True
+ ),
+ "embedding_small": [[float(hours)] for hours in generated_hours_before] + [[-1.0], [-2.0]],
+ }
+ )
+
+ found = activity_vector_as_of(
+ vectors, team_id=1, user_id=7, as_of=ACTIVITY_AS_OF, max_age=datetime.timedelta(days=7)
+ )
+
+ if expected_hours_before is None:
+ assert found is None
+ else:
+ assert found is not None
+ assert found.embedding == [float(expected_hours_before)]
+ assert found.age == datetime.timedelta(hours=expected_hours_before)
+ assert found.generated_at == ACTIVITY_AS_OF - found.age
+
+
class _FakeClient:
def __init__(self) -> None:
self.calls: list[dict[str, Any]] = []
diff --git a/products/signals/dags/inbox_ranking/training/examples.py b/products/signals/dags/inbox_ranking/training/examples.py
index 4dde0ccb2b7f..3613749033ef 100644
--- a/products/signals/dags/inbox_ranking/training/examples.py
+++ b/products/signals/dags/inbox_ranking/training/examples.py
@@ -419,6 +419,35 @@ def _with_features(
return pd.concat(frames, ignore_index=True)[columns]
+@frozen
+class ActivityVector:
+ embedding: list[float]
+ generated_at: datetime.datetime
+ age: datetime.timedelta
+
+
+def activity_vector_as_of(
+ vectors: pd.DataFrame, *, team_id: int, user_id: int, as_of: datetime.datetime, max_age: datetime.timedelta
+) -> ActivityVector | None:
+ """The activity summary vector user `user_id` of team `team_id` had at `as_of`, read from the union
+ of `inbox_activity_summary_vectors` partitions.
+
+ That is the newest vector generated strictly before `as_of`, so a summary built after the moment
+ never leaks into it. None when the user has no such vector, or when the newest one is older than
+ `max_age`: a summary that stale no longer describes what the user does."""
+ known = vectors.loc[
+ (vectors["team_id"] == team_id) & (vectors["user_id"] == user_id) & (vectors["generated_at"] < as_of)
+ ]
+ if known.empty:
+ return None
+ latest = known.loc[known["generated_at"].idxmax()]
+ generated_at = pd.Timestamp(latest["generated_at"]).to_pydatetime()
+ age = as_of - generated_at
+ if age > max_age:
+ return None
+ return ActivityVector(embedding=list(latest["embedding_small"]), generated_at=generated_at, age=age)
+
+
def holdout_mask(examples: pd.DataFrame, holdout_days: int) -> pd.Series:
"""True for every row of the reports created in the last `holdout_days` of the example set.
Cut by report so a report's snapshot rows never straddle train and holdout."""