Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

feat(flags): route realtime cohorts for teams at the serving level (#129544)

Matheus Batistace61344ce646
diff --git a/docs/internal/feature-flags/rust-service-overview.md b/docs/internal/feature-flags/rust-service-overview.md
index 0b47ba3f4f59..b70951996548 100644
--- a/docs/internal/feature-flags/rust-service-overview.md
+++ b/docs/internal/feature-flags/rust-service-overview.md
@@ -262,13 +262,13 @@ Exact property matching is selected per team through `TeamFeatureFlagsConfig.pro
### Behavioral cohorts
-| Variable | Default | Purpose |
-| -------------------------------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `BEHAVIORAL_COHORTS_READ_DATABASE_URL` | (empty) | Optional PostgreSQL connection for realtime cohort membership lookups. When empty, no pool is created and a NoOp provider resolves every realtime cohort lookup to non-membership |
-| `REALTIME_COHORT_EVALUATION_TEAM_IDS` | `none` | Rollout gate for realtime cohort evaluation: `none`, `all`, or specific team IDs / ranges (e.g. `1,5,300:400`) |
-| `COHORT_MEMBERSHIP_CACHE_TTL_SECONDS` | `60` | Cache TTL for cohort membership lookups |
-| `COHORT_MEMBERSHIP_CACHE_MAX_ENTRIES` | `500000` | Max entries in cohort membership cache |
-| `REALTIME_COHORT_LOOKUP_TIMEOUT_MS` | `1000` | Upper bound on one membership lookup (pool acquire + query), matching the pool's 1s statement timeout; on timeout the lookup degrades to non-membership |
+| Variable | Default | Purpose |
+| -------------------------------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `BEHAVIORAL_COHORTS_READ_DATABASE_URL` | (empty) | Optional PostgreSQL connection for realtime cohort membership lookups. When empty, no pool is created and a NoOp provider resolves every realtime cohort lookup to non-membership |
+| `REALTIME_COHORT_EVALUATION_TEAM_IDS` | `none` | Rollout gate for realtime cohort evaluation: `none`, `all`, or specific team IDs / ranges (e.g. `1,5,300:400`). Teams whose `cohort_realtime_team_access` row is at `serving` are evaluated too |
+| `COHORT_MEMBERSHIP_CACHE_TTL_SECONDS` | `60` | Cache TTL for cohort membership lookups |
+| `COHORT_MEMBERSHIP_CACHE_MAX_ENTRIES` | `500000` | Max entries in cohort membership cache |
+| `REALTIME_COHORT_LOOKUP_TIMEOUT_MS` | `1000` | Upper bound on one membership lookup (pool acquire + query), matching the pool's 1s statement timeout; on timeout the lookup degrades to non-membership |
The behavioral cohorts pool uses tight limits (max 5 connections, 1s statement timeout) since it only performs simple key lookups against the `cohort_membership` table.
Realtime cohort evaluation degrades gracefully at every layer, always treating people as non-members rather than failing flag evaluation:
diff --git a/docs/internal/realtime-cohorts/backfill-coordination.md b/docs/internal/realtime-cohorts/backfill-coordination.md
index 2f15c32e00f2..3467bdf0a259 100644
--- a/docs/internal/realtime-cohorts/backfill-coordination.md
+++ b/docs/internal/realtime-cohorts/backfill-coordination.md
@@ -165,14 +165,14 @@ The membership consumer does count those markers toward its sweep, see [membersh
Several independent gates decide whether anything happens.
Missing one of them fails quietly, so they are worth knowing.
-| Gate | Effect |
-| ---------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `REALTIME_COHORT_TEAM_ALLOWLIST` | Teams whose cohorts get hash maintenance, stamp invalidation, supersession and any run at all. Unset means no teams in Django, but the shuffler, processor and seeder default to team 2, so set it explicitly on every service. Set but empty means **all** teams. Django puts a listed team at the `serving` level, and adds the teams that have a team access row. The shufflers, the processor and the seeder add those teams too, the seeder 120 seconds after the grant. Never list a team granted `processing` here, because that makes it `serving` |
-| Team access rows | `cohort_realtime_team_access`, written only by `realtime_cohort_team_access grant\|revoke\|list`. Django, the shufflers, the processor and the seeder read it as a union with their `REALTIME_COHORT_TEAM_ALLOWLIST`. Node ingestion does not read the table, and emits person merges only for the teams in its `PERSON_MERGE_EVENTS_TEAM_ALLOWLIST`. Set that list to `*`, because the processor drops the merges of teams it does not process. The seeder works on a granted team's runs only once `processing_since` is 120 seconds old. After a revoke the seeder stops working on the team's runs, unless its own env list names the team, and each run keeps its cohort's run slot. The processor drops a revoked team from its catalog, and stage 2 GC then deletes the team's state without emitting `left`, so the team's `cohort_membership` rows stay `in_cohort=true`. A shuffler whose env list does not name the team stops forwarding its events. A row at `processing` gets the Django behavior above, but the team's users see no realtime state: no readiness, no realtime batch audience, and no backfilled cohort in flags or workflows, even one stamped before the grant, so a workflow that already uses one fails its next save. A row at `serving` also shows them. `revoke` sets `off`, which hides the stamps the same way, and a revoked team cannot be granted again. `grant` and `revoke` refuse a team on the env list |
-| `COHORT_BACKFILL_TRIGGER_TEAM_ALLOWLIST` | Teams whose saves create runs automatically. Unset or empty means **no** teams. A granted row with `save_triggers_backfill` adds its team, so `resave_cohorts --no-backfill-triggers`, not this setting, is what stops a resave from queueing runs |
-| Operator attestations | Settings an operator sets to declare that prerequisites hold. They are declarations, not live checks. Behavioral runs need `BEHAVIORAL_BACKFILL_MERGE_GATE_ATTESTED` and `BEHAVIORAL_BACKFILL_DURABILITY_ATTESTED`. Person runs also need the person TTL and sizing attestations and a positive seed-bytes budget |
-| Person sizing budget | A cohort-scoped person run is refused when the team's active person-run estimates plus its own would exceed the budget. The check reads the active estimates before the run exists and reserves nothing, so two cohorts sized at the same time can together exceed the budget. A team run checks only its own estimate. Hitting the size estimate's scan cap, or the cap on pinned conditions, also refuses |
-| Behavioral scan limit | `create_cohort_backfill_run` refuses a behavioral team run when its pinned event names had more events than `BEHAVIORAL_BACKFILL_MAX_SCAN_EVENTS_PER_DAY` on the busiest of the last 7 complete UTC days. `--max-scan-events-per-day` overrides it for one invocation, and 0 skips the count and the check. The save path does not check it |
+| Gate | Effect |
+| ---------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `REALTIME_COHORT_TEAM_ALLOWLIST` | Teams whose cohorts get hash maintenance, stamp invalidation, supersession and any run at all. Unset means no teams in Django, but the shuffler, processor and seeder default to team 2, so set it explicitly on every service. Set but empty means **all** teams. Django puts a listed team at the `serving` level, and adds the teams that have a team access row. The shufflers, the processor and the seeder add those teams too, the seeder 120 seconds after the grant. Never list a team granted `processing` here, because that makes it `serving` |
+| Team access rows | `cohort_realtime_team_access`, written only by `realtime_cohort_team_access grant\|revoke\|list`. Django, the shufflers, the processor and the seeder read it as a union with their `REALTIME_COHORT_TEAM_ALLOWLIST`. Node ingestion does not read the table, and emits person merges only for the teams in its `PERSON_MERGE_EVENTS_TEAM_ALLOWLIST`. Set that list to `*`, because the processor drops the merges of teams it does not process. The seeder works on a granted team's runs only once `processing_since` is 120 seconds old. After a revoke the seeder stops working on the team's runs, unless its own env list names the team, and each run keeps its cohort's run slot. The processor drops a revoked team from its catalog, and stage 2 GC then deletes the team's state without emitting `left`, so the team's `cohort_membership` rows stay `in_cohort=true`. A shuffler whose env list does not name the team stops forwarding its events. A row at `processing` gets the Django behavior above, but the team's users see no realtime state: no readiness, no realtime batch audience, and no backfilled cohort in flags or workflows, even one stamped before the grant, so a workflow that already uses one fails its next save. A row at `serving` also shows them, and the flags service routes the team's cohorts to the membership table once its team metadata cache carries the level. `revoke` sets `off`, which hides the stamps the same way, and a revoked team cannot be granted again. `grant` and `revoke` refuse a team on the env list |
+| `COHORT_BACKFILL_TRIGGER_TEAM_ALLOWLIST` | Teams whose saves create runs automatically. Unset or empty means **no** teams. A granted row with `save_triggers_backfill` adds its team, so `resave_cohorts --no-backfill-triggers`, not this setting, is what stops a resave from queueing runs |
+| Operator attestations | Settings an operator sets to declare that prerequisites hold. They are declarations, not live checks. Behavioral runs need `BEHAVIORAL_BACKFILL_MERGE_GATE_ATTESTED` and `BEHAVIORAL_BACKFILL_DURABILITY_ATTESTED`. Person runs also need the person TTL and sizing attestations and a positive seed-bytes budget |
+| Person sizing budget | A cohort-scoped person run is refused when the team's active person-run estimates plus its own would exceed the budget. The check reads the active estimates before the run exists and reserves nothing, so two cohorts sized at the same time can together exceed the budget. A team run checks only its own estimate. Hitting the size estimate's scan cap, or the cap on pinned conditions, also refuses |
+| Behavioral scan limit | `create_cohort_backfill_run` refuses a behavioral team run when its pinned event names had more events than `BEHAVIORAL_BACKFILL_MAX_SCAN_EVENTS_PER_DAY` on the busiest of the last 7 complete UTC days. `--max-scan-events-per-day` overrides it for one invocation, and 0 skips the count and the check. The save path does not check it |
The Rust services have their own switches, and Django cannot see them.
The processor must hold a team before the seeder works on the team's runs.
diff --git a/docs/internal/realtime-cohorts/completion-and-readiness.md b/docs/internal/realtime-cohorts/completion-and-readiness.md
index 2f3161835bca..2a0739dc15d3 100644
--- a/docs/internal/realtime-cohorts/completion-and-readiness.md
+++ b/docs/internal/realtime-cohorts/completion-and-readiness.md
@@ -230,7 +230,7 @@ Behind the same rollout flag, for teams at the `serving` level, the cohort API's
The Rust flags service reads cohorts from the flags cache, or from Postgres through a short-lived cache when the flags cache has none.
It routes a cohort to the membership table when all of these hold:
-- the team is in `REALTIME_COHORT_EVALUATION_TEAM_IDS`,
+- the team is in `REALTIME_COHORT_EVALUATION_TEAM_IDS`, or its team access row is at `serving`,
- the cohort's type is `realtime`, or the legacy `behavioral` type,
- its `condition_type` marks a behavioral or lifecycle condition,
- its stamps satisfy the configured stamp policy.
diff --git a/docs/internal/realtime-cohorts/membership-output-and-readers.md b/docs/internal/realtime-cohorts/membership-output-and-readers.md
index 9744d8d963f1..0cc6c7996a40 100644
--- a/docs/internal/realtime-cohorts/membership-output-and-readers.md
+++ b/docs/internal/realtime-cohorts/membership-output-and-readers.md
@@ -201,12 +201,16 @@ Its connection pool, cache and timeout settings are in [the flags service overvi
### When a cohort is read from the table
-For each request on a team in `REALTIME_COHORT_EVALUATION_TEAM_IDS`, the flags service routes a cohort to the table only if:
+For each request on a team in `REALTIME_COHORT_EVALUATION_TEAM_IDS`, or whose team access row is at `serving`, the flags service routes a cohort to the table only if:
- the cohort's type is `realtime`, or the legacy `behavioral` type,
- its `condition_type` marks a behavioral or lifecycle condition,
- its [readiness stamps](completion-and-readiness.md#what-readiness-unlocks) satisfy the configured stamp policy.
+`/flags` reads the team access level from the team metadata cache, or from Postgres when the cache has no entry for the team.
+Batch flag evaluation always reads it from Postgres.
+A cache entry without the level reads as `off`.
+
`condition_type` is derived by Django on save.
A cohort whose `condition_type` is empty is never routed, and resaving it fills the column.
diff --git a/rust/feature-flags/src/api/batch_flag_evaluation.rs b/rust/feature-flags/src/api/batch_flag_evaluation.rs
index 673bff1cf6f8..b954c5cfb8a1 100644
--- a/rust/feature-flags/src/api/batch_flag_evaluation.rs
+++ b/rust/feature-flags/src/api/batch_flag_evaluation.rs
@@ -614,10 +614,8 @@ async fn handle_batch_flag_evaluation(
state.database_pools.non_persons_reader.clone(),
state.database_pools.non_persons_writer.clone(),
);
- let enable_realtime_cohort_evaluation = state
- .config
- .realtime_cohort_evaluation_team_ids
- .includes_team(request.team_id);
+ let enable_realtime_cohort_evaluation =
+ team.evaluates_realtime_cohorts(&state.config.realtime_cohort_evaluation_team_ids);
let use_explicit_exact_matching = team.property_matching_version.uses_explicit_matching();
let team_timezone = team.parsed_timezone();
let optimize_experience_continuity_lookups =
diff --git a/rust/feature-flags/src/config.rs b/rust/feature-flags/src/config.rs
index 23b008e7b393..5dd3d58c3ff8 100644
--- a/rust/feature-flags/src/config.rs
+++ b/rust/feature-flags/src/config.rs
@@ -415,16 +415,16 @@ pub struct Config {
#[envconfig(from = "SECRET_KEY", default = "")]
pub secret_key: String,
- // Team-scoped gate for realtime cohort evaluation. When "none" (default), the
- // realtime cohort block in prepare_flag_evaluation_state is skipped entirely, even
- // if the behavioral cohorts DB is configured and cohorts with CohortType::Realtime
- // exist. Set to "all", specific team IDs, or ranges to enable realtime cohort
- // membership lookups on the hot path for those teams.
+ // Team-scoped gate for realtime cohort evaluation, unioned with the teams whose
+ // `cohort_realtime_team_access` row is at `serving` (`Team::realtime_cohort_level`). For
+ // every other team the realtime cohort block in prepare_flag_evaluation_state is skipped
+ // entirely, even if the behavioral cohorts DB is configured and cohorts with
+ // CohortType::Realtime exist. "none" (default), "all", specific team IDs, or ranges.
//
// Routing also requires `condition_type`, which was never backfilled, so every team listed
- // here needs `python manage.py resave_cohorts --team-id <id>` first. Without it behavioral
- // cohorts read as condition_type = NULL and fall back to dynamic evaluation, which resolves
- // every person as a non-member rather than just evaluating slower.
+ // here or granted `serving` needs `python manage.py resave_cohorts --team-id <id>` first.
+ // Without it behavioral cohorts read as condition_type = NULL and fall back to dynamic
+ // evaluation, which resolves every person as a non-member rather than just evaluating slower.
//
// A team listed here must also be in Django's REALTIME_COHORT_TEAM_ALLOWLIST: the stamps
// this predicate trusts are only invalidated on edit for allowlisted teams, so without it
diff --git a/rust/feature-flags/src/handler/flags.rs b/rust/feature-flags/src/handler/flags.rs
index 9f16aaf99a0b..089e7ec6f847 100644
--- a/rust/feature-flags/src/handler/flags.rs
+++ b/rust/feature-flags/src/handler/flags.rs
@@ -10,10 +10,10 @@ use crate::{
flag_models::{EvaluationMetadata, FeatureFlag, FeatureFlagList},
flag_service::FlagService,
},
+ team::team_models::Team,
utils::user_agent::{RuntimeType, UserAgentInfo},
};
use axum::extract::State;
-use chrono_tz::Tz;
use common_types::TeamId;
use serde::{Deserialize, Deserializer, Serialize};
use serde_json::Value;
@@ -388,9 +388,7 @@ fn collect_excluded_by_tags(
#[allow(clippy::too_many_arguments)]
pub async fn evaluate_for_request(
state: &State<router::State>,
- team_id: i32,
- team_timezone: Tz,
- use_explicit_exact_matching: bool,
+ team: &Team,
distinct_id: String,
device_id: Option<String>,
filtered_flags: FeatureFlagList,
@@ -426,8 +424,10 @@ pub async fn evaluate_for_request(
});
let ctx = FeatureFlagEvaluationContext {
- team_id,
- team_timezone,
+ team_id: team.id,
+ // Interpret naive datetime filter values in the team timezone so flag
+ // evaluation matches HogQL/ClickHouse cohort membership.
+ team_timezone: team.parsed_timezone(),
distinct_id,
device_id,
feature_flags: filtered_flags,
@@ -452,11 +452,9 @@ pub async fn evaluate_for_request(
|| only_use_override_person_properties.unwrap_or(false)
|| *state.config.skip_writes,
cohort_membership_provider: state.cohort_membership_provider.clone(),
- enable_realtime_cohort_evaluation: state
- .config
- .realtime_cohort_evaluation_team_ids
- .includes_team(team_id),
- use_explicit_exact_matching,
+ enable_realtime_cohort_evaluation: team
+ .evaluates_realtime_cohorts(&state.config.realtime_cohort_evaluation_team_ids),
+ use_explicit_exact_matching: team.property_matching_version.uses_explicit_matching(),
membership_stamp_policy: state.config.realtime_cohort_membership_stamp_policy,
detailed_analysis: detailed_analysis.unwrap_or(false),
only_use_override_person_properties: only_use_override_person_properties
diff --git a/rust/feature-flags/src/handler/mod.rs b/rust/feature-flags/src/handler/mod.rs
index fc5471e62102..e7f0cf0e9ab4 100644
--- a/rust/feature-flags/src/handler/mod.rs
+++ b/rust/feature-flags/src/handler/mod.rs
@@ -287,11 +287,7 @@ async fn process_request_inner(
let _phase = PhaseGuard::enter(Phase::Evaluate);
flags::evaluate_for_request(
&context.state,
- team.id,
- // Interpret naive datetime filter values in the team timezone so flag
- // evaluation matches HogQL/ClickHouse cohort membership.
- team.parsed_timezone(),
- team.property_matching_version.uses_explicit_matching(),
+ &team,
distinct_id.clone(),
device_id.clone(),
filtered_flags.clone(),
diff --git a/rust/feature-flags/src/server.rs b/rust/feature-flags/src/server.rs
index a6a044adf6a0..cb369c02202d 100644
--- a/rust/feature-flags/src/server.rs
+++ b/rust/feature-flags/src/server.rs
@@ -242,37 +242,35 @@ pub async fn serve_with_rate_limiter_clock<C>(
Some(config.group_type_cache_ttl_seconds),
));
- // Initialize the cohort membership provider for realtime/behavioral cohorts.
- // Requires both the behavioral cohorts DB pool AND a non-empty team ID collection.
- // When "none" (default), NoOp is used regardless of DB availability,
- // so no realtime cohort queries hit the hot path.
+ // Initialize the cohort membership provider for realtime/behavioral cohorts whenever the
+ // behavioral cohorts DB pool exists: a team access row at `serving` routes a team's cohorts
+ // even when REALTIME_COHORT_EVALUATION_TEAM_IDS is "none". A team that routes no cohort
+ // never calls the provider, so no realtime cohort query reaches its hot path.
let cohort_membership_provider: Arc<dyn CohortMembershipProvider> =
- if config.realtime_cohort_evaluation_team_ids != TeamIdCollection::None {
- if let Some(pool) = database_pools.behavioral_cohorts_reader.clone() {
- tracing::info!(
- cache_ttl_seconds = config.cohort_membership_cache_ttl_seconds,
- cache_max_entries = config.cohort_membership_cache_max_entries,
- lookup_timeout_ms = config.realtime_cohort_lookup_timeout_ms,
- "Realtime cohort evaluation enabled with behavioral cohorts DB"
- );
- let realtime = RealtimeCohortMembershipProvider::with_lookup_timeout(
- pool,
- Duration::from_millis(config.realtime_cohort_lookup_timeout_ms),
- );
- Arc::new(CachedCohortMembershipProvider::new(
- realtime,
- Some(config.cohort_membership_cache_ttl_seconds),
- Some(config.cohort_membership_cache_max_entries),
- ))
- } else {
+ if let Some(pool) = database_pools.behavioral_cohorts_reader.clone() {
+ tracing::info!(
+ cache_ttl_seconds = config.cohort_membership_cache_ttl_seconds,
+ cache_max_entries = config.cohort_membership_cache_max_entries,
+ lookup_timeout_ms = config.realtime_cohort_lookup_timeout_ms,
+ "Realtime cohort evaluation enabled with behavioral cohorts DB"
+ );
+ let realtime = RealtimeCohortMembershipProvider::with_lookup_timeout(
+ pool,
+ Duration::from_millis(config.realtime_cohort_lookup_timeout_ms),
+ );
+ Arc::new(CachedCohortMembershipProvider::new(
+ realtime,
+ Some(config.cohort_membership_cache_ttl_seconds),
+ Some(config.cohort_membership_cache_max_entries),
+ ))
+ } else {
+ if config.realtime_cohort_evaluation_team_ids != TeamIdCollection::None {
tracing::warn!(
"REALTIME_COHORT_EVALUATION_TEAM_IDS is set but \
BEHAVIORAL_COHORTS_READ_DATABASE_URL is not configured; realtime \
cohort lookups will treat everyone as a non-member"
);
- Arc::new(NoOpCohortMembershipProvider)
}
- } else {
Arc::new(NoOpCohortMembershipProvider)
};
diff --git a/rust/feature-flags/src/team/team_models.rs b/rust/feature-flags/src/team/team_models.rs
index 882449d06bd9..f2141166835a 100644
--- a/rust/feature-flags/src/team/team_models.rs
+++ b/rust/feature-flags/src/team/team_models.rs
@@ -1,6 +1,8 @@
+use crate::config::TeamIdCollection;
use crate::team::request_blocklist::{self, BlockReason, BlockedHosts, BlockedIps};
use common_types::{TeamId, TeamIdentifier};
use rust_decimal::Decimal;
+use serde::de::{value, IntoDeserializer};
use serde::{Deserialize, Serialize};
use sqlx::types::{Json, Uuid};
use std::net::IpAddr;
@@ -25,6 +27,39 @@ impl Default for PropertyMatchingVersion {
}
}
+/// The team's level in Django's `cohort_realtime_team_access` table, with the strings of
+/// `RealtimeCohortLevel` in `products/cohorts/backend/models/team_access.py`. A team without a
+/// row is `Off`.
+#[derive(Clone, Copy, Debug, Default, Deserialize, Eq, PartialEq, Serialize)]
+#[serde(rename_all = "snake_case")]
+pub enum RealtimeCohortLevel {
+ #[default]
+ Off,
+ Processing,
+ Serving,
+ /// A level Django writes that this build does not know. It does not serve, so a new level
+ /// never fails a team read.
+ #[serde(other)]
+ Unrecognized,
+}
+
+impl RealtimeCohortLevel {
+ fn serves(self) -> bool {
+ match self {
+ Self::Serving => true,
+ Self::Off | Self::Processing | Self::Unrecognized => false,
+ }
+ }
+}
+
+/// Parses with the serde names, so the Postgres read and the cache read accept the same strings.
+impl From<String> for RealtimeCohortLevel {
+ fn from(level: String) -> Self {
+ let level: value::StringDeserializer<value::Error> = level.into_deserializer();
+ Self::deserialize(level).expect("the catch-all variant accepts every string")
+ }
+}
+
#[derive(Clone, Debug, Default, Deserialize, Serialize, sqlx::FromRow)]
pub struct Team {
pub id: TeamId,
@@ -88,9 +123,20 @@ pub struct Team {
#[serde(default)]
#[sqlx(try_from = "Vec<String>")]
pub flags_blocked_ips: BlockedIps,
+ // Sourced from the cohorts product's team access row. Cache entries written before this field
+ // existed read `Off`, which leaves the team to REALTIME_COHORT_EVALUATION_TEAM_IDS.
+ #[serde(default)]
+ #[sqlx(try_from = "String")]
+ pub realtime_cohort_level: RealtimeCohortLevel,
}
impl Team {
+ /// Whether the team's realtime cohorts may read membership from the behavioral cohorts DB:
+ /// the rollout list names the team, or its team access row is at `serving`.
+ pub fn evaluates_realtime_cohorts(&self, rollout: &TeamIdCollection) -> bool {
+ rollout.includes_team(self.id) || self.realtime_cohort_level.serves()
+ }
+
/// The reason this team's request blocklist refuses a caller with this host and these
/// client IPs, or `None` when the caller is allowed.
pub fn request_block_reason(
@@ -142,6 +188,7 @@ impl TeamIdentifier for Team {
#[cfg(test)]
mod tests {
use super::*;
+ use rstest::rstest;
#[test]
fn test_minimal_flag_called_events_defaults_false_on_legacy_cache_blob() {
@@ -167,6 +214,60 @@ mod tests {
team.request_block_reason(Some("example.com"), ["203.0.113.7".parse().unwrap()]),
None
);
+ assert_eq!(team.realtime_cohort_level, RealtimeCohortLevel::Off);
+ }
+
+ #[rstest]
+ #[case("off", RealtimeCohortLevel::Off)]
+ #[case("processing", RealtimeCohortLevel::Processing)]
+ #[case("serving", RealtimeCohortLevel::Serving)]
+ #[case("beta", RealtimeCohortLevel::Unrecognized)]
+ fn test_realtime_cohort_level_reads_django_levels_and_unknown_ones(
+ #[case] level: &str,
+ #[case] expected: RealtimeCohortLevel,
+ ) {
+ let cached = serde_json::json!({
+ "id": 1,
+ "name": "test team",
+ "api_token": "test_token",
+ "uuid": Uuid::new_v4().to_string(),
+ "realtime_cohort_level": level,
+ });
+
+ let team: Team = serde_json::from_value(cached).expect("cache blob must deserialize");
+ let reserialized: Team =
+ serde_json::from_value(serde_json::to_value(&team).unwrap()).unwrap();
+
+ assert_eq!(team.realtime_cohort_level, expected);
+ assert_eq!(reserialized.realtime_cohort_level, expected);
+ assert_eq!(RealtimeCohortLevel::from(level.to_string()), expected);
+ }
+
+ #[rstest]
+ #[case::listed_without_a_row(TeamIdCollection::TeamIds(vec![1]), RealtimeCohortLevel::Off, true)]
+ #[case::unlisted_at_serving(TeamIdCollection::None, RealtimeCohortLevel::Serving, true)]
+ #[case::unlisted_at_processing(
+ TeamIdCollection::TeamIds(vec![2]),
+ RealtimeCohortLevel::Processing,
+ false
+ )]
+ #[case::unlisted_at_an_unknown_level(
+ TeamIdCollection::None,
+ RealtimeCohortLevel::Unrecognized,
+ false
+ )]
+ fn test_realtime_cohorts_follow_the_rollout_list_or_a_serving_row(
+ #[case] rollout: TeamIdCollection,
+ #[case] level: RealtimeCohortLevel,
+ #[case] expected: bool,
+ ) {
+ let team = Team {
+ id: 1,
+ realtime_cohort_level: level,
+ ..Default::default()
+ };
+
+ assert_eq!(team.evaluates_realtime_cohorts(&rollout), expected);
}
#[test]
diff --git a/rust/feature-flags/src/team/team_operations.rs b/rust/feature-flags/src/team/team_operations.rs
index 077c3ce95bf0..4d1726459313 100644
--- a/rust/feature-flags/src/team/team_operations.rs
+++ b/rust/feature-flags/src/team/team_operations.rs
@@ -59,7 +59,11 @@ const TEAM_COLUMNS: &str = "
COALESCE(
(SELECT blocked_ips FROM feature_flags_teamfeatureflagpolicyconfig WHERE team_id = posthog_team.id),
'{}'
- )::text[] AS flags_blocked_ips
+ )::text[] AS flags_blocked_ips,
+ COALESCE(
+ (SELECT level FROM cohort_realtime_team_access WHERE team_id = posthog_team.id),
+ 'off'
+ )::text AS realtime_cohort_level
";
impl Team {
@@ -124,7 +128,7 @@ impl Team {
mod tests {
use super::*;
use crate::team::request_blocklist::BlockReason;
- use crate::team::team_models::PropertyMatchingVersion;
+ use crate::team::team_models::{PropertyMatchingVersion, RealtimeCohortLevel};
use crate::utils::test_utils::{
insert_new_team_in_redis, setup_redis_client, setup_team_hypercache_reader, TestContext,
};
@@ -418,4 +422,51 @@ mod tests {
);
}
}
+
+ #[tokio::test]
+ async fn test_fetch_team_reads_the_realtime_cohort_access_level() {
+ let context = TestContext::new(None).await;
+ let team = context
+ .insert_new_team(None)
+ .await
+ .expect("Failed to insert team in pg");
+ let without_row = Team::from_pg(context.non_persons_reader.clone(), &team.api_token)
+ .await
+ .expect("Failed to fetch team from pg");
+
+ let mut conn = get_connection_with_metrics(
+ &context.non_persons_reader,
+ "non_persons_reader",
+ "test_insert_realtime_cohort_team_access",
+ )
+ .await
+ .expect("Failed to get connection");
+ sqlx::query(
+ "INSERT INTO cohort_realtime_team_access \
+ (team_id, level, purpose, save_triggers_backfill, processing_since, changed_at) \
+ VALUES ($1, 'serving', 'pilot', false, now(), now())",
+ )
+ .bind(team.id)
+ .execute(&mut *conn)
+ .await
+ .expect("Failed to insert the team access row");
+ let by_token = Team::from_pg(context.non_persons_reader.clone(), &team.api_token).await;
+ let by_id = Team::from_pg_by_id(context.non_persons_reader.clone(), team.id).await;
+ // Delete before asserting, so a failed assertion leaves no row behind.
+ sqlx::query("DELETE FROM cohort_realtime_team_access WHERE team_id = $1")
+ .bind(team.id)
+ .execute(&mut *conn)
+ .await
+ .expect("Failed to delete the team access row");
+
+ assert_eq!(without_row.realtime_cohort_level, RealtimeCohortLevel::Off);
+ for team_from_pg in [by_token, by_id] {
+ assert_eq!(
+ team_from_pg
+ .expect("Failed to fetch team from pg")
+ .realtime_cohort_level,
+ RealtimeCohortLevel::Serving
+ );
+ }
+ }
}
diff --git a/rust/feature-flags/tests/test_batch_flag_evaluation.rs b/rust/feature-flags/tests/test_batch_flag_evaluation.rs
index ef2da80f2302..c0b6c927779a 100644
--- a/rust/feature-flags/tests/test_batch_flag_evaluation.rs
+++ b/rust/feature-flags/tests/test_batch_flag_evaluation.rs
@@ -16,6 +16,7 @@ use uuid::Uuid;
pub mod common;
use crate::common::ServerHandle;
+use feature_flags::cohorts::cohort_models::CohortType;
use feature_flags::config::{Config, DEFAULT_TEST_CONFIG};
use feature_flags::flags::flag_matching_utils::calculate_hash;
use feature_flags::flags::flag_models::FeatureFlagRow;
@@ -1161,6 +1162,86 @@ async fn test_static_and_dynamic_cohort_conditions() -> Result<()> {
Ok(())
}
+#[tokio::test]
+async fn test_realtime_cohort_condition_reads_membership_for_a_serving_team() -> Result<()> {
+ let context = TestContext::new(None).await;
+ let team = context.insert_new_team(None).await?;
+ let server = ServerHandle::for_config(batch_eval_config()).await;
+
+ let cohort = context
+ .insert_cohort_with_type_and_condition_type(
+ team.id,
+ None,
+ json!({"properties": {"type": "OR", "values": [{
+ "type": "OR",
+ "values": [{"key": "plan", "value": "enterprise", "type": "person"}],
+ }]}}),
+ false,
+ Some(CohortType::Realtime),
+ None,
+ Some(chrono::Utc::now()),
+ Some(json!({"behavioral": true})),
+ None,
+ )
+ .await?;
+ context
+ .insert_flag(
+ team.id,
+ Some(flag_row(
+ team.id,
+ "realtime-cohort-flag",
+ json!({
+ "groups": [{
+ "properties": [{"key": "id", "value": cohort.id, "type": "cohort"}],
+ "rollout_percentage": 100,
+ }],
+ }),
+ )),
+ )
+ .await?;
+
+ // The person fails the cohort's person filter, so only their `cohort_membership` row can match them.
+ context
+ .insert_person(
+ team.id,
+ "realtime_member".to_string(),
+ Some(json!({"plan": "free"})),
+ )
+ .await?;
+ let member = context
+ .get_person_uuid_by_distinct_id(team.id, "realtime_member")
+ .await?;
+ context
+ .insert_cohort_membership(team.id, cohort.id, member, true)
+ .await?;
+
+ let mut conn = context.get_non_persons_connection().await?;
+ sqlx::query(
+ "INSERT INTO cohort_realtime_team_access \
+ (team_id, level, purpose, save_triggers_backfill, processing_since, changed_at) \
+ VALUES ($1, 'serving', 'pilot', false, now(), now())",
+ )
+ .bind(team.id)
+ .execute(&mut *conn)
+ .await?;
+ let res = send_batch_request(
+ &server,
+ Some(INTERNAL_TOKEN),
+ &batch_body(team.id, "realtime-cohort-flag", 0),
+ )
+ .await;
+ // Delete before asserting, so a failed assertion leaves no row behind.
+ sqlx::query("DELETE FROM cohort_realtime_team_access WHERE team_id = $1")
+ .bind(team.id)
+ .execute(&mut *conn)
+ .await?;
+
+ assert_eq!(res.status(), StatusCode::OK);
+ assert_eq!(matched_uuids(&res.json::<Value>().await?), vec![member]);
+
+ Ok(())
+}
+
#[tokio::test]
async fn test_missing_property_negative_operator_uses_live_semantics() -> Result<()> {
// Live Rust evaluation explicitly treats a missing property as matching `is_not`
diff --git a/rust/feature-flags/tests/test_flags.rs b/rust/feature-flags/tests/test_flags.rs
index 87e2534de5ba..ad9fd3d363ce 100644
--- a/rust/feature-flags/tests/test_flags.rs
+++ b/rust/feature-flags/tests/test_flags.rs
@@ -14,6 +14,7 @@ use crate::common::*;
use feature_flags::cohorts::cohort_models::CohortType;
use feature_flags::config::DEFAULT_TEST_CONFIG;
+use feature_flags::team::team_models::RealtimeCohortLevel;
use feature_flags::utils::test_utils::{
insert_active_flags, insert_config_in_hypercache, insert_flags_for_team_in_redis,
insert_flags_with_metadata_for_team_in_redis, insert_new_team_in_redis, setup_pg_reader_client,
@@ -6336,6 +6337,118 @@ async fn test_realtime_cohort_without_backfill_falls_through_to_dynamic_eval() -
Ok(())
}
+/// With REALTIME_COHORT_EVALUATION_TEAM_IDS at `none`, only the cached team's access level can
+/// route a backfilled realtime cohort to the `cohort_membership` table. The person is a member
+/// there but fails the cohort's person-property filters, so the flag is on only when the cohort
+/// routes to the table.
+#[rstest]
+#[case::serving_reads_the_membership_table(RealtimeCohortLevel::Serving, true)]
+#[case::processing_evaluates_dynamically(RealtimeCohortLevel::Processing, false)]
+#[tokio::test]
+async fn test_realtime_cohort_routing_follows_the_cached_team_access_level(
+ #[case] level: RealtimeCohortLevel,
+ #[case] expected_enabled: bool,
+) -> Result<()> {
+ let config = DEFAULT_TEST_CONFIG.clone();
+ let client = setup_redis_client(Some(config.redis_url.clone())).await;
+ let mut team = insert_new_team_in_redis(client.clone()).await.unwrap();
+ team.realtime_cohort_level = level;
+ update_team_in_hypercache(client.clone(), &team).await?;
+
+ let context = TestContext::new(None).await;
+ context.insert_new_team(Some(team.id)).await.unwrap();
+
+ let cohort = context
+ .insert_cohort_with_type_and_condition_type(
+ team.id,
+ Some("Realtime Backfilled".to_string()),
+ json!({
+ "properties": {
+ "type": "OR",
+ "values": [{
+ "type": "AND",
+ "values": [{
+ "key": "plan",
+ "type": "person",
+ "value": ["enterprise"],
+ "operator": "exact",
+ "negation": false
+ }]
+ }]
+ }
+ }),
+ false,
+ Some(CohortType::Realtime),
+ None,
+ Some(chrono::Utc::now()),
+ Some(json!({"behavioral": true})),
+ None,
+ )
+ .await
+ .unwrap();
+
+ let flag_json = json!([{
+ "id": 1,
+ "key": "realtime-level-flag",
+ "name": "Realtime Level Flag",
+ "active": true,
+ "deleted": false,
+ "team_id": team.id,
+ "filters": {
+ "groups": [{
+ "properties": [{
+ "key": "id",
+ "type": "cohort",
+ "value": cohort.id,
+ "operator": "in"
+ }],
+ "rollout_percentage": 100
+ }]
+ }
+ }]);
+ insert_flags_for_team_in_redis(client.clone(), team.id, Some(flag_json.to_string())).await?;
+
+ let distinct_id = "user_realtime_level".to_string();
+ context
+ .insert_person(
+ team.id,
+ distinct_id.clone(),
+ Some(json!({ "plan": "free" })),
+ )
+ .await
+ .unwrap();
+ let person_uuid = context
+ .get_person_uuid_by_distinct_id(team.id, &distinct_id)
+ .await?;
+ context
+ .insert_cohort_membership(team.id, cohort.id, person_uuid, true)
+ .await?;
+
+ let server = ServerHandle::for_config(config).await;
+ let payload = json!({
+ "token": team.api_token,
+ "distinct_id": distinct_id,
+ });
+ let res = server
+ .send_flags_request(payload.to_string(), Some("2"), None)
+ .await;
+
+ assert_eq!(res.status(), StatusCode::OK);
+ assert_json_include!(
+ actual: res.json::<Value>().await?,
+ expected: json!({
+ "flags": {
+ "realtime-level-flag": {
+ "key": "realtime-level-flag",
+ "enabled": expected_enabled
+ }
+ }
+ })
+ );
+
+ Ok(())
+}
+
#[tokio::test]
async fn it_degrades_person_flags_when_the_persons_db_stops_answering() -> Result<()> {
// Accepted connections sit in the listen backlog and never get a reply, like a frozen database.