:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(signals): add user vectors and pair examples for inbox ranking (#124494)
posthog[bot]86ba8976681a
diff --git a/posthog/dags/locations/signals_ml.py b/posthog/dags/locations/signals_ml.pyindex 2fa3d58d9f68..22a2336dd711 100644--- a/posthog/dags/locations/signals_ml.py+++ b/posthog/dags/locations/signals_ml.py@@ -4,6 +4,7 @@ from products.signals.dags.inbox_ranking.common import is_inbox_ranking_registerfrom products.signals.dags.inbox_ranking.nn import (dag as inbox_ranking_nn,experiments as inbox_ranking_nn_experiments,+ pairs as inbox_ranking_nn_pairs,)from . import loggers, resources@@ -20,10 +21,12 @@ if is_inbox_ranking_registered():inbox_ranking_nn.inbox_ranking_nn_with_prio_model_candidate,inbox_ranking_nn.inbox_ranking_nn_with_prio_model_champion,inbox_ranking_nn_experiments.inbox_ranking_nn_experiment_candidates,+ inbox_ranking_nn_pairs.inbox_ranking_nn_pair_examples,],jobs=[inbox_ranking_nn.inbox_ranking_nn_training_job,inbox_ranking_nn_experiments.inbox_ranking_nn_experiments_job,+ inbox_ranking_nn_pairs.inbox_ranking_nn_pair_examples_job,],schedules=[inbox_ranking_nn.inbox_ranking_nn_training_schedule,diff --git a/products/signals/dags/inbox_ranking/AGENTS.md b/products/signals/dags/inbox_ranking/AGENTS.mdindex d819483fe12d..bcb0fab88d70 100644--- a/products/signals/dags/inbox_ranking/AGENTS.md+++ b/products/signals/dags/inbox_ranking/AGENTS.md@@ -44,6 +44,7 @@ Read `README.md` first for what the dataset is and how partitions behave. This f- `nn_v3` and `nn_with_prio_v2` are trained by the `signals_ml` location (`nn/dag.py`), never by the main one. They share one `inbox_ranking_nn_examples` object, and each has its own candidate and champion asset from `family_assets`. The `nn_v3` assets keep the names `inbox_ranking_nn_model_{candidate,champion}`, so their Dagster history continues. A later family's candidate depends on the previous family's champion, for memory and not for data: two candidates at once can exceed the job's memory limit. A family whose set reads a state column the examples object lacks skips with a warning rather than training on zeros. torch is imported only inside the candidate asset's body (through `nn/train.py` and `multihead/network.py`). `signals_ml` must import without torch, and `test_multihead_network.py` pins that. Its decision goes through `decide_family_champion` in `training/dag.py`, graded through the incumbent's `model.holdout.onnx`. Keep `signals_ml` commented out in `.dagster_home/workspace.yaml`, and keep its schedule (04:30 UTC) running by default on prod US only.- An experiment is a `MODEL_FAMILIES` entry with `experiment=True`, a `baseline` serving family and an `expires` day. Only `inbox_ranking_nn_experiments_job` (`nn/experiments.py`) trains it, with its own schedule and resource tags, so a modeling idea never adds to the `nn` job's runtime or memory budget. It reads the day's `inbox_ranking_nn_examples` object and never rebuilds it. An experiment never enters the serving manifest, and `_pin_metadata` refuses its key, so the promote job, the served pointer and the pins cannot name it. To serve its recipe, register it as a serving family under a new `model_name`. Its `metadata.json` carries a `vs_baseline` block: the experiment and the baseline fit with each of `EXPERIMENT_SEEDS` on the same train rows and graded on the same holdout rows. An extra side input for an experiment goes through the registry in `training/side_inputs.py`, which refuses a frame without a timezone-aware `as_of_column` and drops rows known after the partition end. A feature set reads it through `rows_as_of`.+- `inbox_ranking_nn_pair_examples` (`nn/pairs.py`) has one row per (report example, impressed user), with its own `PAIR_EXAMPLES_SCHEMA_VERSION` and its own job without a schedule. It takes its report rows and holdout from the `open` head's rows of the day's `inbox_ranking_nn_examples` object and never rebuilds them. Its impressions and labels come from the dt=B+`PAIR_HORIZON_DAYS` interactions partition, and its user inputs are as of the start of day B, so the target report is never in the user's history. The object holds no vector: a reader rebuilds the user vectors with `pair_user_vectors`. `user_vectors()` lives next to `project_vectors()` and shares `known_before` and `_usable_embeddings` with it; never give it its own as-of rule or embedding gate.## Invariants — do not breakdiff --git a/products/signals/dags/inbox_ranking/README.md b/products/signals/dags/inbox_ranking/README.mdindex db8dddd608ef..5e6aeb70bcfe 100644--- a/products/signals/dags/inbox_ranking/README.md+++ b/products/signals/dags/inbox_ranking/README.md@@ -8,7 +8,7 @@ inbox_ranking/├── dataset/│ ├── dag.py # the dataset assets + job + schedule│ ├── queries.py # HogQL label SQL, embeddings SQL, stream merging-│ └── project_vectors.py # per-team context vectors (pure function)+│ └── project_vectors.py # per-team and per-user context vectors (pure functions)├── training/│ ├── dag.py # serving manifest → served scores → unseen grade assets + job + schedule, and the plumbing nn shares│ ├── promote.py # the manual promote job that writes served.json@@ -17,7 +17,7 @@ inbox_ranking/│ ├── train.py # per-head holdout/null metrics that every learner fills│ └── promotion.py # the candidate grading rule├── multihead/ # the multi-head network, its ONNX export, and the offline prototype-├── nn/ # the `nn` family: examples → candidate → champion assets, on the `signals_ml` location+├── nn/ # the `nn` family: examples → candidate → champion assets, and the pair examples, on the `signals_ml` location└── tests/```@@ -455,6 +455,25 @@ dagster job launch -w .dagster_home/workspace.yaml --location posthog.dags.locatThe newest day you can launch is the newest day the `nn` job finished. Adjacent days share most of their train and holdout rows, so their reads are not independent. Judge a `vs_baseline` gap on spaced days, for example every 7th day.+### Pair examples++`inbox_ranking_nn_pair_examples` (`nn/pairs.py`) holds the report-user pairs for a personal ranking lane: a model that reads a vector of the reports one user viewed before, next to the report.+It writes `inbox_ranking_nn_pair_examples/v1/dt=D/` with its own `PAIR_EXAMPLES_SCHEMA_VERSION`, and runs in its own job, `inbox_ranking_nn_pair_examples_job`, which has no schedule. Launch it for a partition after the `nn` job finished that day.++- **Rows**: one per (report example, user), for each user impressed with the report within `PAIR_HORIZON_DAYS` (7) of birth. The report rows are the `open` head's rows of the dt=D `inbox_ranking_nn_examples` object, so the report inputs, the holdout split (`holdout` is the row's `open_holdout`) and the day windows are the report lane's. A report born on day B reads its impressions from the dt=B+7 `inbox_user_report_interactions` partition, the partition the report's `open` label reads.+- **Labels**, from the same partition and before its end: `open_label` is the user's `first_opened_at`, and `action_label` is any of the user's UI action `first_*` columns from `UI_ACTION_COLUMNS`, the set the `action` project vector reads. There is no `fixed` label, because a fix belongs to the report and not to a user.+- **User inputs**, as of the start of day B, from the dt=D interactions and embeddings snapshots: `user_open_reports` and `user_dismiss_reports` (the reports in each `user_vectors()` mean), `prior_views` (every view before B, without the limit or the embedding gate), `days_since_last_view` (capped at `DAYS_SINCE_VIEW_CAP`) and `has_history`. The report and the reports born with it are outside the history by construction.+- **`is_suggested_reviewer`** comes from the dt=B `inbox_report_reviewers` partition when it exists, and from `impressed_as_suggested_reviewer_count` otherwise. `suggested_reviewer_source` says which (`reviewers` or `impressions`).+- **No vectors.** The object holds the user inputs above and the keys `report_id`, `report_team_id` and `user_distinct_id`. A reader rebuilds the `open` and `dismiss` user vectors with `pair_user_vectors`, which calls `user_vectors()` once per birth day.+- **Consent**: only reports and history of teams in `training_consent_team_ids()`, read at run time.+- **Metadata**: `rows`, `users`, `reports`, positives per label, rows and users per prior-view bucket (`0`, `1_9`, `10_49`, `50_plus`), and `suggested_reviewer_share`.+- **A missing input is a skip.** Without the dt=D examples, interactions or embeddings object, the asset writes nothing. A birth day without its dt=B+7 interactions partition gives no pairs and is counted in `birth_days_without_impressions`.++`user_vectors(interactions, embeddings, as_of, organizations=...)` in `dataset/project_vectors.py` builds one row per (organization, user, kind) with `vector`, `n_reports`, `last_at` and `as_of`.+The `open` kind reads `first_viewed_at`, the durable VIEW row, and the `dismiss` kind reads `first_ui_dismissed_at`.+The vector is the plain mean of the user's last `USER_HISTORY_LIMIT` (20) reports of the kind, through the as-of rule and the embedding gate of `project_vectors()`.+A report counts only in the organization of its own team, and a user without usable history gets no row.+## Configuration| Setting | Default | Meaning |diff --git a/products/signals/dags/inbox_ranking/dataset/project_vectors.py b/products/signals/dags/inbox_ranking/dataset/project_vectors.pyindex 86abcc68644a..14520cda263f 100644--- a/products/signals/dags/inbox_ranking/dataset/project_vectors.py+++ b/products/signals/dags/inbox_ranking/dataset/project_vectors.py@@ -12,11 +12,21 @@ gets one vector per kind, as of a moment:Only human UI engagement counts: the open, action and dismiss moments come from the inbox appevents, so MCP reads and agent actions never reach a vector. A project is a team, not anorganization.++`user_vectors` builds the same kind of vector for one person in one organization, from the+`inbox_user_report_interactions` rows of the reports that person viewed or dismissed:++ open the mean of the last `USER_HISTORY_LIMIT` reports the user viewed (`first_viewed_at`)+ dismiss the same, over the reports the user dismissed from the UI++Both functions take a report into a vector through the same as-of rule (`known_before`) and the same+embedding gate (`_usable_embeddings`)."""import datetimefrom collections import defaultdict-from collections.abc import Sequence+from collections.abc import Mapping, Sequence+from typing import TypeGuardimport numpy as npimport pyarrow as pa@@ -54,6 +64,25 @@ _PROJECT_VECTORS_FIELDS: list[tuple[str, pa.DataType]] = []PROJECT_VECTORS_SCHEMA = pa.schema(_PROJECT_VECTORS_FIELDS)+USER_HISTORY_LIMIT = 20+USER_OPEN, USER_DISMISS = "open", "dismiss"+# The durable VIEW action row dates an open, because the `Inbox report opened` event misses the+# surfaces that send no client event.+USER_HISTORY_COLUMNS: Mapping[str, str] = {USER_OPEN: "first_viewed_at", USER_DISMISS: DISMISS_COLUMN}+USER_VECTOR_KINDS = tuple(USER_HISTORY_COLUMNS)+USER_INTERACTION_COLUMNS = ("report_id", "report_team_id", "user_distinct_id", *USER_HISTORY_COLUMNS.values())++_USER_VECTORS_FIELDS: list[tuple[str, pa.DataType]] = [+ ("organization_id", pa.string()),+ ("user_distinct_id", pa.string()),+ ("kind", pa.string()),+ ("vector", pa.list_(pa.float32())),+ ("n_reports", pa.int32()),+ ("last_at", pa.timestamp("us", tz="UTC")),+ ("as_of", pa.timestamp("us", tz="UTC")),+]+USER_VECTORS_SCHEMA = pa.schema(_USER_VECTORS_FIELDS)+def project_vectors(labels: pa.Table, embeddings: pa.Table, as_of: datetime.datetime) -> pa.Table:"""One row per (team, kind) for every team with a report vector before `as_of`.@@ -68,11 +97,7 @@ def project_vectors(labels: pa.Table, embeddings: pa.Table, as_of: datetime.dateand an all-zero vector, so a reader can tell a thin history from a missing team."""usable = _usable_embeddings(embeddings, as_of)- matrix = (- pc.list_flatten(usable.column("embedding_small").combine_chunks())- .to_numpy(zero_copy_only=False)- .reshape(-1, EMBEDDING_DIMENSIONS)- )+ matrix = _embedding_matrix(usable)report_ids = [str(report_id) for report_id in usable.column("report_id").to_pylist()]rows_by_team: dict[int, list[int]] = defaultdict(list)for index, team_id in enumerate(usable.column("report_team_id").to_pylist()):@@ -87,15 +112,77 @@ def project_vectors(labels: pa.Table, embeddings: pa.Table, as_of: datetime.datefor kind in (OPEN, ACTION, DISMISS):moments = engaged_at[kind]engaged = [- (moment, index)- for index in team_rows- if (moment := moments.get(report_ids[index])) is not None and moment < as_of+ (moment, index) for index in team_rows if known_before(moment := moments.get(report_ids[index]), as_of)]- latest = [index for _, index in sorted(engaged, reverse=True)[:HISTORY_LIMIT]]+ latest = [index for _, index in _latest(engaged, HISTORY_LIMIT)]rows.append(_vector_row(team_id, kind, matrix, latest, as_of))return pa.Table.from_pylist(rows, schema=PROJECT_VECTORS_SCHEMA)+def user_vectors(+ interactions: pa.Table,+ embeddings: pa.Table,+ as_of: datetime.datetime,+ *,+ organizations: Mapping[int, str],+) -> pa.Table:+ """One row per (organization, user, kind) for every user with a usable report of that kind.++ `interactions` is an `inbox_user_report_interactions` snapshot and `embeddings` an+ `inbox_report_embeddings` snapshot, both at or after `as_of`. `organizations` maps a report's+ team to its organization. A report enters the user's row of its own team's organization, so+ history from another organization never reaches the row. A report whose team has no entry in+ `organizations` is left out.++ `vector` is the plain mean of the user's last `USER_HISTORY_LIMIT` reports of the kind, by the+ kind's moment, that pass the `project_vectors` rules: the moment is strictly before `as_of`, and+ the report vector landed before `as_of` and is not a tombstone. `last_at` is the newest of those+ moments. A user with no such report gets no row for the kind.+ """+ histories: dict[tuple[str, str, str], list[tuple[datetime.datetime, str]]] = defaultdict(list)+ for row in interactions.select(list(USER_INTERACTION_COLUMNS)).to_pylist():+ organization_id = organizations.get(row["report_team_id"])+ if organization_id is None or row["user_distinct_id"] is None:+ continue+ for kind, column in USER_HISTORY_COLUMNS.items():+ if known_before(moment := row[column], as_of):+ histories[(organization_id, row["user_distinct_id"], kind)].append((moment, str(row["report_id"])))++ # Only the history's reports are gated and flattened, so a call costs the user set's history+ # rather than a copy of the whole embeddings snapshot.+ needed = pa.array(sorted({report_id for history in histories.values() for _, report_id in history}), pa.string())+ usable = _usable_embeddings(embeddings.filter(pc.is_in(embeddings.column("report_id"), value_set=needed)), as_of)+ matrix = _embedding_matrix(usable)+ index_of = {str(report_id): index for index, report_id in enumerate(usable.column("report_id").to_pylist())}++ rows: list[dict[str, object]] = []+ for (organization_id, user_distinct_id, kind), history in sorted(histories.items()):+ latest = _latest(+ [(moment, index_of[report_id]) for moment, report_id in history if report_id in index_of],+ USER_HISTORY_LIMIT,+ )+ if not latest:+ continue+ indices = [index for _, index in latest]+ rows.append(+ {+ "organization_id": organization_id,+ "user_distinct_id": user_distinct_id,+ "kind": kind,+ "vector": matrix[indices].mean(axis=0, dtype=np.float64).astype(np.float32),+ "n_reports": len(indices),+ "last_at": latest[0][0],+ "as_of": as_of,+ }+ )+ return pa.Table.from_pylist(rows, schema=USER_VECTORS_SCHEMA)+++def known_before(moment: datetime.datetime | None, as_of: datetime.datetime) -> TypeGuard[datetime.datetime]:+ """The as-of rule of every context vector: a moment counts only when it is strictly before `as_of`."""+ return moment is not None and moment < as_of++def teams_with_history(table: pa.Table) -> dict[str, int]:"""How many teams of a `project_vectors` table have enough history for a real vector, per kind."""counts = table.select(["kind", "n_reports"]).to_pylist()@@ -130,6 +217,19 @@ def _usable_embeddings(embeddings: pa.Table, as_of: datetime.datetime) -> pa.Tabreturn embeddings.filter(mask).select(["report_id", "report_team_id", "embedding_small"])+def _embedding_matrix(usable: pa.Table) -> np.ndarray:+ return (+ pc.list_flatten(usable.column("embedding_small").combine_chunks())+ .to_numpy(zero_copy_only=False)+ .reshape(-1, EMBEDDING_DIMENSIONS)+ )+++def _latest(engaged: list[tuple[datetime.datetime, int]], limit: int) -> list[tuple[datetime.datetime, int]]:+ """The `limit` newest (moment, row) pairs, newest first."""+ return sorted(engaged, reverse=True)[:limit]++def _engaged_at(labels: pa.Table) -> dict[str, dict[str, datetime.datetime]]:"""The first UI engagement moment of each kind per report."""engaged_at: dict[str, dict[str, datetime.datetime]] = {OPEN: {}, ACTION: {}, DISMISS: {}}diff --git a/products/signals/dags/inbox_ranking/nn/pairs.py b/products/signals/dags/inbox_ranking/nn/pairs.pynew file mode 100644index 000000000000..b0b903161ca7--- /dev/null+++ b/products/signals/dags/inbox_ranking/nn/pairs.py@@ -0,0 +1,417 @@+"""Report-user pair examples for the personal ranking lane, without torch.++One asset on the `signals_ml` location, on the dataset dag's daily partition:++ inbox_ranking_nn_pair_examples/v1/dt=D/ one row per (report example, impressed user)++The report rows are the `open` head's rows of the day's `inbox_ranking_nn_examples` object, so the+report inputs, the holdout split and the day windows are the report-grain lane's. A report born on+day B pairs with each user who was impressed with it within `PAIR_HORIZON_DAYS` of birth, read from+the dt=B+`PAIR_HORIZON_DAYS` `inbox_user_report_interactions` partition, the same later partition the+report's `open` label reads. The pair labels come from the same partition.++The user inputs are as of the start of day B, the moment the report examples' project vectors use.+So the target report and the reports born with it are outside the user's history by construction.+Like the report examples object, the pair object holds no vector: `pair_user_vectors` rebuilds the+user vectors from the dt=D interactions and embeddings snapshots through `user_vectors()`.+"""++import datetime+from collections import Counter+from collections.abc import Mapping++import numpy as np+import pandas as pd+import dagster+import pyarrow as pa+import pyarrow.compute as pc++from posthog import settings+from posthog.models import Team++from products.signals.backend.ranking.features import EMBEDDING_DIMENSIONS+from products.signals.dags.inbox_ranking.common import (+ dataset_bucket,+ owner_tags,+ partition_object_key,+ read_parquet_if_exists,+ s3_client,+ skip_unconfigured,+ snapshot_bounds,+ write_parquet,+)+from products.signals.dags.inbox_ranking.consent import training_consent_team_ids+from products.signals.dags.inbox_ranking.dataset.dag import EMBEDDINGS_TABLE, REVIEWERS_TABLE, USER_INTERACTIONS_TABLE+from products.signals.dags.inbox_ranking.dataset.project_vectors import (+ EMBEDDING_COLUMNS,+ UI_ACTION_FIRST_COLUMNS,+ USER_DISMISS,+ USER_INTERACTION_COLUMNS,+ USER_OPEN,+ USER_VECTOR_KINDS,+ known_before,+ user_vectors,+)+from products.signals.dags.inbox_ranking.nn.dag import NN_ASSET_KWARGS, NN_EXAMPLES_TABLE+from products.signals.dags.inbox_ranking.training.dag import _LOOKBACK_MAPPING+from products.signals.dags.inbox_ranking.training.heads import HEADS_BY_NAME++NN_PAIR_EXAMPLES_TABLE = "inbox_ranking_nn_pair_examples"+PAIR_EXAMPLES_SCHEMA_VERSION = 1++# The report head whose rows, holdout and later label partition the pairs share.+_REPORT_HEAD = HEADS_BY_NAME["open"]+PAIR_HORIZON_DAYS = _REPORT_HEAD.horizon_days+PAIR_HEADS = ("open", "action")+DAYS_SINCE_VIEW_CAP = 90.0+# (name, lowest, highest) prior views per bucket. The within-user grade needs enough rows in the+# buckets with real history to be readable.+HISTORY_BUCKETS: tuple[tuple[str, int, int | None], ...] = (+ ("0", 0, 0),+ ("1_9", 1, 9),+ ("10_49", 10, 49),+ ("50_plus", 50, None),+)+REVIEWERS_SOURCE = "reviewers"+IMPRESSIONS_SOURCE = "impressions"++LABEL_INTERACTION_COLUMNS = (+ "report_id",+ "user_distinct_id",+ "first_impressed_at",+ "impressed_as_suggested_reviewer_count",+ "first_opened_at",+ *UI_ACTION_FIRST_COLUMNS,+)+REVIEWER_COLUMNS = ("report_id", "user_distinct_id")+_VIEW_COLUMN = "first_viewed_at"++_PAIR_EXAMPLES_FIELDS: list[tuple[str, pa.DataType]] = [+ ("snapshot_date", pa.date32()),+ ("report_id", pa.string()),+ ("report_team_id", pa.int64()),+ ("user_distinct_id", pa.string()),+ ("holdout", pa.bool_()),+ ("open_label", pa.float32()),+ ("action_label", pa.float32()),+ ("user_open_reports", pa.int32()),+ ("user_dismiss_reports", pa.int32()),+ ("prior_views", pa.int32()),+ ("days_since_last_view", pa.float32()),+ ("has_history", pa.bool_()),+ ("is_suggested_reviewer", pa.bool_()),+ ("suggested_reviewer_source", pa.string()),+]+PAIR_EXAMPLES_SCHEMA = pa.schema(_PAIR_EXAMPLES_FIELDS)+++def _users_history(history: pa.Table, users: list[str]) -> pa.Table:+ return history.filter(pc.is_in(history.column("user_distinct_id"), value_set=pa.array(users, pa.string())))+++def pair_rows(+ reports: pd.DataFrame,+ impressions: pa.Table,+ history: pa.Table,+ embeddings: pa.Table,+ *,+ birth_day: datetime.date,+ reviewers: pa.Table | None,+ organizations: Mapping[int, str],+) -> list[dict[str, object]]:+ """The pair rows of the reports born on `birth_day`.++ `reports` holds the day's report example rows (`report_id`, `report_team_id`, `holdout`).+ `impressions` is the dt=`birth_day`+`PAIR_HORIZON_DAYS` interactions partition, and `history`+ and `embeddings` are the dt=D snapshots the user vectors read. `reviewers` is the dt=`birth_day`+ reviewers partition, or None when that day has none, and then a suggested-reviewer impression+ stands in for it.+ """+ as_of, _ = snapshot_bounds(birth_day.isoformat())+ _, horizon_end = snapshot_bounds((birth_day + datetime.timedelta(days=PAIR_HORIZON_DAYS)).isoformat())+ report_ids = [str(report_id) for report_id in reports["report_id"]]+ report_teams = dict(zip(report_ids, (int(team_id) for team_id in reports["report_team_id"]), strict=True))+ holdout = dict(zip(report_ids, (bool(flag) for flag in reports["holdout"]), strict=True))++ impressed = [+ row+ for row in impressions.filter(+ pc.is_in(impressions.column("report_id"), value_set=pa.array(report_ids, pa.string()))+ )+ .select(list(LABEL_INTERACTION_COLUMNS))+ .to_pylist()+ if row["user_distinct_id"] is not None and known_before(row["first_impressed_at"], horizon_end)+ ]+ if not impressed:+ return []++ users_history = _users_history(history, sorted({row["user_distinct_id"] for row in impressed}))+ vectors = {+ (row["organization_id"], row["user_distinct_id"], row["kind"]): row+ for row in user_vectors(users_history, embeddings, as_of, organizations=organizations)+ .drop_columns(["vector"])+ .to_pylist()+ }+ prior_views: Counter[tuple[str, str]] = Counter()+ for row in users_history.select(["report_team_id", "user_distinct_id", _VIEW_COLUMN]).to_pylist():+ organization_id = organizations.get(row["report_team_id"])+ if organization_id is not None and known_before(row[_VIEW_COLUMN], as_of):+ prior_views[(organization_id, row["user_distinct_id"])] += 1+ suggested = (+ {(row["report_id"], row["user_distinct_id"]) for row in reviewers.select(list(REVIEWER_COLUMNS)).to_pylist()}+ if reviewers is not None+ else set()+ )++ rows: list[dict[str, object]] = []+ for row in impressed:+ report_id, user = str(row["report_id"]), row["user_distinct_id"]+ organization_id = organizations.get(report_teams[report_id])+ open_vector = vectors.get((organization_id, user, USER_OPEN))+ dismiss_vector = vectors.get((organization_id, user, USER_DISMISS))+ days_since_last_view = (+ min((as_of - open_vector["last_at"]).total_seconds() / 86400, DAYS_SINCE_VIEW_CAP)+ if open_vector is not None+ else DAYS_SINCE_VIEW_CAP+ )+ rows.append(+ {+ "snapshot_date": birth_day,+ "report_id": report_id,+ "report_team_id": report_teams[report_id],+ "user_distinct_id": user,+ "holdout": holdout[report_id],+ "open_label": float(known_before(row["first_opened_at"], horizon_end)),+ "action_label": float(+ any(known_before(row[column], horizon_end) for column in UI_ACTION_FIRST_COLUMNS)+ ),+ "user_open_reports": open_vector["n_reports"] if open_vector is not None else 0,+ "user_dismiss_reports": dismiss_vector["n_reports"] if dismiss_vector is not None else 0,+ "prior_views": prior_views[(organization_id, user)] if organization_id is not None else 0,+ "days_since_last_view": days_since_last_view,+ "has_history": open_vector is not None or dismiss_vector is not None,+ "is_suggested_reviewer": (+ (report_id, user) in suggested+ if reviewers is not None+ else (row["impressed_as_suggested_reviewer_count"] or 0) > 0+ ),+ "suggested_reviewer_source": REVIEWERS_SOURCE if reviewers is not None else IMPRESSIONS_SOURCE,+ }+ )+ return rows+++def pair_user_vectors(+ pairs: pd.DataFrame, history: pa.Table, embeddings: pa.Table, *, organizations: Mapping[int, str]+) -> np.ndarray:+ """The user vectors of each pair row, one per kind of `USER_VECTOR_KINDS`, as of the start of+ its birth day. A kind without history is all zeros, and the row's `user_*_reports` tells it apart."""+ pairs = pairs.reset_index(drop=True)+ vectors = np.zeros((len(pairs), len(USER_VECTOR_KINDS), EMBEDDING_DIMENSIONS), dtype=np.float32)+ for birth_day, group in pairs.groupby("snapshot_date", sort=True):+ as_of, _ = snapshot_bounds(birth_day.isoformat())+ day = user_vectors(+ _users_history(history, sorted(set(group["user_distinct_id"]))),+ embeddings,+ as_of,+ organizations=organizations,+ )+ by_key = {+ (organization_id, user, kind): vector+ for organization_id, user, kind, vector in zip(+ day.column("organization_id").to_pylist(),+ day.column("user_distinct_id").to_pylist(),+ day.column("kind").to_pylist(),+ day.column("vector").to_numpy(zero_copy_only=False),+ strict=True,+ )+ }+ for position, team_id, user in zip(+ group.index, group["report_team_id"], group["user_distinct_id"], strict=True+ ):+ organization_id = organizations.get(int(team_id))+ for kind_index, kind in enumerate(USER_VECTOR_KINDS):+ vector = by_key.get((organization_id, user, kind))+ if vector is not None:+ vectors[position, kind_index] = vector+ return vectors+++def pair_metadata(table: pa.Table) -> dict[str, dagster.MetadataValue]:+ frame = table.select(+ [+ "report_id",+ "user_distinct_id",+ "prior_views",+ "is_suggested_reviewer",+ *(f"{head}_label" for head in PAIR_HEADS),+ ]+ ).to_pandas()+ metadata: dict[str, dagster.MetadataValue] = {+ "rows": dagster.MetadataValue.int(len(frame)),+ "users": dagster.MetadataValue.int(int(frame["user_distinct_id"].nunique())),+ "reports": dagster.MetadataValue.int(int(frame["report_id"].nunique())),+ **{f"{head}_positives": dagster.MetadataValue.int(int(frame[f"{head}_label"].sum())) for head in PAIR_HEADS},+ "suggested_reviewer_share": dagster.MetadataValue.float(+ float(frame["is_suggested_reviewer"].mean()) if len(frame) else 0.0+ ),+ }+ for name, lowest, highest in HISTORY_BUCKETS:+ in_bucket = frame["prior_views"] >= lowest+ if highest is not None:+ in_bucket &= frame["prior_views"] <= highest+ metadata[f"history_{name}_rows"] = dagster.MetadataValue.int(int(in_bucket.sum()))+ metadata[f"history_{name}_users"] = dagster.MetadataValue.int(+ int(frame.loc[in_bucket, "user_distinct_id"].nunique())+ )+ return metadata+++def build_pair_examples(+ context: dagster.AssetExecutionContext,+ client,+ bucket: str,+ partition_key: str,+ *,+ consent_team_ids: frozenset[int],+ organizations: Mapping[int, str],+) -> dict[str, dagster.MetadataValue]:+ """Write the dt=`partition_key` pair examples and return the asset metadata. A partition without+ the report examples or the dt=D snapshots keeps the object it already holds."""+ prefix = settings.INBOX_RANKING_DATASET_S3_PREFIX+ examples = read_parquet_if_exists(client, bucket, partition_object_key(prefix, NN_EXAMPLES_TABLE, partition_key))+ history = read_parquet_if_exists(+ client,+ bucket,+ partition_object_key(prefix, USER_INTERACTIONS_TABLE, partition_key),+ columns=list(USER_INTERACTION_COLUMNS),+ )+ embeddings = read_parquet_if_exists(+ client, bucket, partition_object_key(prefix, EMBEDDINGS_TABLE, partition_key), columns=list(EMBEDDING_COLUMNS)+ )+ missing = [+ name+ for name, table in (+ (NN_EXAMPLES_TABLE, examples),+ (USER_INTERACTIONS_TABLE, history),+ (EMBEDDINGS_TABLE, embeddings),+ )+ if table is None+ ]+ if examples is None or history is None or embeddings is None:+ context.log.warning(f"no {', '.join(missing)} for dt={partition_key}; pair examples skip this partition")+ return {"skipped": dagster.MetadataValue.bool(True)}++ consented = pa.array(sorted(consent_team_ids), pa.int64())+ history = history.filter(pc.is_in(history.column("report_team_id"), value_set=consented))+ reports = examples.select(+ ["report_id", "snapshot_date", "report_team_id", f"{_REPORT_HEAD.name}_label", f"{_REPORT_HEAD.name}_holdout"]+ ).to_pandas()+ reports = reports[reports[f"{_REPORT_HEAD.name}_label"].notna()]+ with_consent = reports["report_team_id"].isin(consent_team_ids)+ excluded_reports = int((~with_consent).sum())+ reports = reports[with_consent].rename(columns={f"{_REPORT_HEAD.name}_holdout": "holdout"})++ rows: list[dict[str, object]] = []+ days_without_impressions = 0+ days_with_reviewers = 0+ for birth_day, group in reports.groupby("snapshot_date", sort=True):+ label_day = (birth_day + datetime.timedelta(days=PAIR_HORIZON_DAYS)).isoformat()+ impressions = read_parquet_if_exists(+ client,+ bucket,+ partition_object_key(prefix, USER_INTERACTIONS_TABLE, label_day),+ columns=list(LABEL_INTERACTION_COLUMNS),+ )+ if impressions is None:+ days_without_impressions += 1+ continue+ reviewers = read_parquet_if_exists(+ client,+ bucket,+ partition_object_key(prefix, REVIEWERS_TABLE, birth_day.isoformat()),+ columns=list(REVIEWER_COLUMNS),+ )+ days_with_reviewers += reviewers is not None+ rows.extend(+ pair_rows(+ group,+ impressions,+ history,+ embeddings,+ birth_day=birth_day,+ reviewers=reviewers,+ organizations=organizations,+ )+ )+ if days_without_impressions:+ context.log.warning(+ f"{days_without_impressions} birth days have no {USER_INTERACTIONS_TABLE} label partition; their reports have no pairs"+ )++ table = pa.Table.from_pylist(rows, schema=PAIR_EXAMPLES_SCHEMA)+ key = partition_object_key(prefix, NN_PAIR_EXAMPLES_TABLE, partition_key)+ write_parquet(client, bucket, key, table, snapshot_date=partition_key, schema_version=PAIR_EXAMPLES_SCHEMA_VERSION)+ return {+ **pair_metadata(table),+ "birth_days": dagster.MetadataValue.int(int(reports["snapshot_date"].nunique())),+ "birth_days_without_impressions": dagster.MetadataValue.int(days_without_impressions),+ "birth_days_with_reviewers": dagster.MetadataValue.int(days_with_reviewers),+ "excluded_no_training_consent_reports": dagster.MetadataValue.int(excluded_reports),+ "s3_key": dagster.MetadataValue.text(f"s3://{bucket}/{key}"),+ }+++def team_organizations(team_ids: frozenset[int]) -> dict[int, str]:+ return {+ team_id: str(organization_id)+ for team_id, organization_id in Team.objects.filter(id__in=team_ids).values_list("id", "organization_id")+ }+++ name=NN_PAIR_EXAMPLES_TABLE,+ deps=[+ NN_EXAMPLES_TABLE,+ dagster.AssetDep(USER_INTERACTIONS_TABLE, partition_mapping=_LOOKBACK_MAPPING),+ dagster.AssetDep(REVIEWERS_TABLE, partition_mapping=_LOOKBACK_MAPPING),+ EMBEDDINGS_TABLE,+ ],+ **NN_ASSET_KWARGS,+)+def inbox_ranking_nn_pair_examples(context: dagster.AssetExecutionContext) -> None:+ if skip_unconfigured(context):+ return+ consent_team_ids = training_consent_team_ids()+ context.add_output_metadata(+ build_pair_examples(+ context,+ s3_client(),+ dataset_bucket(),+ context.partition_key,+ consent_team_ids=consent_team_ids,+ organizations=team_organizations(consent_team_ids),+ )+ )+++# No schedule: the pair lane is an offline experiment, and the object of any past day can be rebuilt+# from the cumulative interactions partitions. Its own job keeps its memory apart from the nn job's.+inbox_ranking_nn_pair_examples_job = dagster.define_asset_job(+ name="inbox_ranking_nn_pair_examples_job",+ selection=[NN_PAIR_EXAMPLES_TABLE],+ tags={+ **owner_tags,+ "dagster/max_runtime": str(2 * 60 * 60),+ # One embeddings snapshot as Arrow, the dt=D interactions snapshot, and one label partition+ # at a time.+ "dagster-k8s/config": {+ "container_config": {+ "resources": {+ "requests": {"memory": "4Gi"},+ "limits": {"memory": "16Gi"},+ }+ }+ },+ },+)diff --git a/products/signals/dags/inbox_ranking/tests/test_pairs.py b/products/signals/dags/inbox_ranking/tests/test_pairs.pynew file mode 100644index 000000000000..d39b819f7c7b--- /dev/null+++ b/products/signals/dags/inbox_ranking/tests/test_pairs.py@@ -0,0 +1,199 @@+import io+import datetime+from typing import Any++import pytest++import numpy as np+import pandas as pd+import dagster+import pyarrow as pa+import pyarrow.parquet as pq+from parameterized import parameterized++from posthog import settings++from products.signals.backend.ranking.features import EMBEDDING_DIMENSIONS+from products.signals.dags.inbox_ranking.common import partition_object_key+from products.signals.dags.inbox_ranking.dataset.dag import (+ EMBEDDINGS_SCHEMA,+ EMBEDDINGS_TABLE,+ REVIEWERS_SCHEMA,+ USER_INTERACTIONS_SCHEMA,+ USER_INTERACTIONS_TABLE,+)+from products.signals.dags.inbox_ranking.dataset.project_vectors import USER_VECTOR_KINDS+from products.signals.dags.inbox_ranking.nn.dag import NN_EXAMPLES_TABLE+from products.signals.dags.inbox_ranking.nn.pairs import (+ IMPRESSIONS_SOURCE,+ NN_PAIR_EXAMPLES_TABLE,+ PAIR_EXAMPLES_SCHEMA,+ REVIEWERS_SOURCE,+ build_pair_examples,+ pair_rows,+ pair_user_vectors,+)+from products.signals.dags.inbox_ranking.tests.test_nn import _S3, _parquet++PREFIX = settings.INBOX_RANKING_DATASET_S3_PREFIX+BIRTH = datetime.date(2026, 8, 1)+BIRTH_START = datetime.datetime(2026, 8, 1, tzinfo=datetime.UTC)+LABEL_DAY = BIRTH + datetime.timedelta(days=7)+ORGANIZATIONS = {2: "org-a", 9: "org-b"}+TARGET, PAST = "report-target", "report-past"+++def _at(days: float) -> datetime.datetime:+ return BIRTH_START + datetime.timedelta(days=days)+++def _embeddings() -> pa.Table:+ # The target report is born and embedded on its birth day, after the start of that day.+ return pa.Table.from_pylist(+ [+ {+ "snapshot_date": LABEL_DAY,+ "report_id": report_id,+ "report_team_id": 2,+ "embedding_small": [value] * EMBEDDING_DIMENSIONS,+ "embedding_inserted_at": inserted_at,+ "embedding_rendering": "title_summary_v1",+ "is_tombstone": False,+ }+ for report_id, value, inserted_at in ((TARGET, 9.0, _at(0.4)), (PAST, 3.0, _at(-5)))+ ],+ schema=EMBEDDINGS_SCHEMA,+ )+++def _interactions(rows: list[dict[str, Any]], team_id: int = 2) -> pa.Table:+ return pa.Table.from_pylist(+ [{"snapshot_date": LABEL_DAY, "report_team_id": team_id, **row} for row in rows],+ schema=USER_INTERACTIONS_SCHEMA,+ )+++# u1 viewed the past report two days before the birth and the target report after it.+HISTORY_ROWS = [+ {"report_id": PAST, "user_distinct_id": "u1", "first_viewed_at": _at(-2)},+ {"report_id": TARGET, "user_distinct_id": "u1", "first_viewed_at": _at(0.5)},+]+IMPRESSION_ROWS = [+ {"report_id": TARGET, "user_distinct_id": "u1", "first_impressed_at": _at(0.5), "first_opened_at": _at(0.6)},+ # Acted only after the horizon ends.+ {+ "report_id": TARGET,+ "user_distinct_id": "u2",+ "first_impressed_at": _at(1),+ "impressed_as_suggested_reviewer_count": 1,+ "first_implement_clicked_at": _at(9),+ },+ # Opened without an impression, and impressed only after the horizon ends.+ {"report_id": TARGET, "user_distinct_id": "u3", "first_opened_at": _at(1)},+ {"report_id": TARGET, "user_distinct_id": "u4", "first_impressed_at": _at(8.1)},+ {"report_id": TARGET, "user_distinct_id": "u5", "first_impressed_at": _at(2), "first_create_pr_clicked_at": _at(3)},+]+REPORTS = pd.DataFrame({"report_id": [TARGET], "report_team_id": [2], "holdout": [True]})+++def _pair_rows(reviewers: pa.Table | None) -> list[dict[str, Any]]:+ return pair_rows(+ REPORTS,+ _interactions(IMPRESSION_ROWS),+ _interactions(HISTORY_ROWS),+ _embeddings(),+ birth_day=BIRTH,+ reviewers=reviewers,+ organizations=ORGANIZATIONS,+ )+++ [+ ("reviewers_snapshot", True, {"u1": False, "u2": False, "u5": True}, REVIEWERS_SOURCE),+ ("impressions_fallback", False, {"u1": False, "u2": True, "u5": False}, IMPRESSIONS_SOURCE),+ ]+)+def test_pairs_are_the_users_impressed_within_the_horizon_with_labels_inside_it(+ _name: str, with_reviewers: bool, suggested: dict[str, bool], source: str+) -> None:+ reviewers = (+ pa.Table.from_pylist([{"report_id": TARGET, "user_distinct_id": "u5"}], schema=REVIEWERS_SCHEMA)+ if with_reviewers+ else None+ )++ rows = {row["user_distinct_id"]: row for row in _pair_rows(reviewers)}++ assert {user: (row["open_label"], row["action_label"]) for user, row in rows.items()} == {+ "u1": (1.0, 0.0),+ "u2": (0.0, 0.0),+ "u5": (0.0, 1.0),+ }+ assert {user: row["is_suggested_reviewer"] for user, row in rows.items()} == suggested+ assert {row["suggested_reviewer_source"] for row in rows.values()} == {source}+ assert pa.Table.from_pylist(list(rows.values()), schema=PAIR_EXAMPLES_SCHEMA).num_rows == 3+++def test_a_pair_user_vector_never_includes_the_examples_own_report() -> None:+ rows = _pair_rows(reviewers=None)+ pairs = pd.DataFrame(rows)++ vectors = pair_user_vectors(pairs, _interactions(HISTORY_ROWS), _embeddings(), organizations=ORGANIZATIONS)++ u1 = pairs.index[pairs["user_distinct_id"] == "u1"][0]+ open_kind = USER_VECTOR_KINDS.index("open")+ assert vectors[u1, open_kind] == pytest.approx([3.0] * EMBEDDING_DIMENSIONS)+ assert (rows[u1]["user_open_reports"], rows[u1]["prior_views"], rows[u1]["days_since_last_view"]) == (1, 1, 2.0)+ assert rows[u1]["has_history"]+ others = pairs.index[pairs["user_distinct_id"] != "u1"]+ assert not np.any(vectors[others])+ assert not any(rows[index]["has_history"] for index in others)+++def test_a_partition_rerun_overwrites_only_its_own_object_and_drops_teams_without_consent() -> None:+ partition = LABEL_DAY.isoformat()+ earlier = partition_object_key(PREFIX, NN_PAIR_EXAMPLES_TABLE, (LABEL_DAY - datetime.timedelta(days=1)).isoformat())+ examples = pa.Table.from_pylist(+ [+ {"report_id": TARGET, "snapshot_date": BIRTH, "report_team_id": 2, "open_label": 1.0, "open_holdout": True},+ {+ "report_id": "report-other",+ "snapshot_date": BIRTH,+ "report_team_id": 9,+ "open_label": 0.0,+ "open_holdout": False,+ },+ ]+ )+ interactions = pa.concat_tables(+ [+ _interactions(HISTORY_ROWS + IMPRESSION_ROWS),+ _interactions([{"report_id": "report-other", "user_distinct_id": "u9", "first_impressed_at": _at(1)}], 9),+ ]+ )+ client = _S3(+ {+ partition_object_key(PREFIX, NN_EXAMPLES_TABLE, partition): _parquet(examples),+ partition_object_key(PREFIX, USER_INTERACTIONS_TABLE, partition): _parquet(interactions),+ partition_object_key(PREFIX, EMBEDDINGS_TABLE, partition): _parquet(_embeddings()),+ partition_object_key(PREFIX, NN_PAIR_EXAMPLES_TABLE, partition): b"previous run",+ earlier: b"earlier partition",+ }+ )+ before = dict(client.objects)++ with dagster.build_asset_context(partition_key=partition) as context:+ metadata = build_pair_examples(+ context, client, "bucket", partition, consent_team_ids=frozenset({2}), organizations=ORGANIZATIONS+ )++ written = partition_object_key(PREFIX, NN_PAIR_EXAMPLES_TABLE, partition)+ assert {key: body for key, body in client.objects.items() if key != written} == {+ key: body for key, body in before.items() if key != written+ }+ table = pq.read_table(io.BytesIO(client.objects[written]))+ assert table.schema == PAIR_EXAMPLES_SCHEMA+ assert set(table.column("user_distinct_id").to_pylist()) == {"u1", "u2", "u5"}+ assert metadata["excluded_no_training_consent_reports"].value == 1+ assert (metadata["rows"].value, metadata["history_0_rows"].value, metadata["history_1_9_rows"].value) == (3, 2, 1)diff --git a/products/signals/dags/inbox_ranking/tests/test_project_vectors.py b/products/signals/dags/inbox_ranking/tests/test_project_vectors.pyindex 12d706f75115..1d60284ea798 100644--- a/products/signals/dags/inbox_ranking/tests/test_project_vectors.py+++ b/products/signals/dags/inbox_ranking/tests/test_project_vectors.py@@ -23,14 +23,17 @@ from products.signals.backend.ranking.serving_manifest import ()from products.signals.dags.inbox_ranking import commonfrom products.signals.dags.inbox_ranking.dataset import dag-from products.signals.dags.inbox_ranking.dataset.dag import EMBEDDINGS_SCHEMA, LABELS_SCHEMA+from products.signals.dags.inbox_ranking.dataset.dag import EMBEDDINGS_SCHEMA, LABELS_SCHEMA, USER_INTERACTIONS_SCHEMAfrom products.signals.dags.inbox_ranking.dataset.project_vectors import (HISTORY_LIMIT,LABEL_COLUMNS,MIN_HISTORY_REPORTS,PROJECT_VECTORS_SCHEMA,PROJECT_VECTORS_SCHEMA_VERSION,+ USER_HISTORY_LIMIT,+ USER_VECTORS_SCHEMA,project_vectors,+ user_vectors,)from products.signals.dags.inbox_ranking.mirror import dag as mirror_dagfrom products.signals.dags.inbox_ranking.multihead.prototype import project_context@@ -140,6 +143,70 @@ def test_a_team_below_the_history_threshold_keeps_a_zero_vector_row() -> None:assert row["vector"] == [0.0] * EMBEDDING_DIMENSIONS+ORGANIZATIONS = {2: "org-a", 3: "org-a", 7: "org-b"}+++def _interactions(views: list[tuple[int, int, str, str, datetime.datetime]]) -> pa.Table:+ return pa.Table.from_pylist(+ [+ {+ "snapshot_date": AS_OF.date(),+ "report_id": _report_id(team_id, index),+ "report_team_id": team_id,+ "user_distinct_id": user,+ column: moment,+ }+ for team_id, index, user, column, moment in views+ ],+ schema=USER_INTERACTIONS_SCHEMA,+ )+++def _user_rows(table: pa.Table) -> dict[tuple[str, str, str], dict[str, Any]]:+ return {(row["organization_id"], row["user_distinct_id"], row["kind"]): row for row in table.to_pylist()}+++ [+ ("view_at_as_of", (2, 1, 9.0, BEFORE), (2, 1, "u1", "first_viewed_at", AS_OF)),+ ("dismiss_at_as_of", (2, 1, 9.0, BEFORE), (2, 1, "u1", "first_ui_dismissed_at", AS_OF)),+ ("embedding_after_as_of", (2, 1, 9.0, AFTER), (2, 1, "u1", "first_viewed_at", BEFORE)),+ ("other_organization", (7, 1, 9.0, BEFORE), (7, 1, "u1", "first_viewed_at", BEFORE)),+ ]+)+def test_user_vectors_take_only_history_known_at_as_of_from_the_same_organization(+ _name: str, report: tuple[int, int, float, datetime.datetime], view: tuple[int, int, str, str, datetime.datetime]+) -> None:+ # A view in another team of the same organization counts; the parameterized row must not.+ reports = [(2, 0, 1.0, BEFORE), (3, 0, 1.0, BEFORE), report]+ views = [(2, 0, "u1", "first_viewed_at", BEFORE), (3, 0, "u1", "first_viewed_at", BEFORE), view]++ table = user_vectors(_interactions(views), _embeddings(reports), AS_OF, organizations=ORGANIZATIONS)+ rows = _user_rows(table)++ assert table.schema == USER_VECTORS_SCHEMA+ assert rows[("org-a", "u1", "open")]["n_reports"] == 2+ assert rows[("org-a", "u1", "open")]["vector"] == pytest.approx([1.0] * EMBEDDING_DIMENSIONS)+ assert ("org-a", "u1", "dismiss") not in rows+++def test_user_vectors_average_the_last_reports_and_skip_a_user_without_history() -> None:+ history = USER_HISTORY_LIMIT + 5+ reports = [(2, index, float(index), BEFORE) for index in range(history)]+ views = [+ (2, index, "u1", "first_viewed_at", BEFORE + datetime.timedelta(minutes=index)) for index in range(history)+ ]+ views.append((2, 0, "u2", "first_viewed_at", AFTER))++ rows = _user_rows(user_vectors(_interactions(views), _embeddings(reports), AS_OF, organizations=ORGANIZATIONS))++ newest = range(history - USER_HISTORY_LIMIT, history)+ assert set(rows) == {("org-a", "u1", "open")}+ assert rows[("org-a", "u1", "open")]["n_reports"] == USER_HISTORY_LIMIT+ assert rows[("org-a", "u1", "open")]["vector"] == pytest.approx([sum(newest) / len(newest)] * EMBEDDING_DIMENSIONS)+ assert rows[("org-a", "u1", "open")]["last_at"] == BEFORE + datetime.timedelta(minutes=history - 1)++def test_the_asset_writes_only_teams_opted_in_to_ai_training(monkeypatch) -> None:partition_key = (AS_OF.date() - datetime.timedelta(days=1)).isoformat()reports = [(team_id, index, 1.0, BEFORE) for team_id in (2, 7) for index in range(MIN_HISTORY_REPORTS)]