:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(flags): read flag_evaluations with a 92-day TTL (#124555)
Phil Haackf88be17733b3
diff --git a/docs/internal/clickhouse-deletion-coverage.md b/docs/internal/clickhouse-deletion-coverage.mdindex 82bc0e15d2ef..3dd5932f74e9 100644--- a/docs/internal/clickhouse-deletion-coverage.md+++ b/docs/internal/clickhouse-deletion-coverage.md@@ -185,7 +185,7 @@ A table created before the switch to `ReplacingMergeTree` keeps every duplicate,#### Rows past the TTL-A request window can reach rows past the 90-day TTL that a part still holds, because `ttl_only_drop_parts = 1` drops a part only once its newest row expires.+A request window can reach rows past the 92-day TTL that a part still holds, because `ttl_only_drop_parts = 1` drops a part only once its newest row expires.The copy stages those rows like any others.A reingested part that holds only expired rows can then drop before the reingest or the shard verify counts it.`DeletionTarget.ttl_days` makes both checks compare only rows at least two days short of their TTL, on the table side and the staged side alike.@@ -218,7 +218,7 @@ The job finds typed columns through their comments in `system.columns`, not thro### Immediate event removal with a HogQL predicate does not sweep `flag_evaluations``get_event_removal_shards` leaves `flag_evaluations` out of the sweep for an immediate request that carries a HogQL predicate.-A row with an events copy ages out with the table's TTL. The table partitions by month with `ttl_only_drop_parts = 1`, so a part drops only once its newest row expires: the real wait is up to about 120 days, not the 90-day TTL.+A row with an events copy ages out with the table's TTL. The table partitions by month with `ttl_only_drop_parts = 1`, so a part drops only once its newest row expires: the real wait is up to about 122 days, not the 92-day TTL.The events copies that a deferred request deletes through do not help here, because the immediate path deletes the events rows without queueing their uuids.A row with no events copy is checked instead.The request refuses while such a row matches the predicate compiled for this table (below), or matches the request's other criteria when the predicate cannot run on this table.diff --git a/frontend/src/scenes/feature-flags/flagEvaluationsTable.ts b/frontend/src/scenes/feature-flags/flagEvaluationsTable.tsindex 6f18501de196..968b2339e080 100644--- a/frontend/src/scenes/feature-flags/flagEvaluationsTable.ts+++ b/frontend/src/scenes/feature-flags/flagEvaluationsTable.ts@@ -24,12 +24,13 @@ export function storesFlagCallsOutsideEvents(team: TeamPublicType | TeamType | n}/**- * How long a row stays in flag_evaluations. The events table keeps $feature_flag_called forever.- * Keep it equal to FLAG_EVALUATIONS_TTL_DAYS in posthog/models/flag_evaluations/sql.py.+ * How long PostHog tells customers that flag_evaluations keeps a row. The events table keeps $feature_flag_called+ * forever. Keep it equal to FLAG_EVALUATIONS_RETENTION_DAYS in posthog/models/flag_evaluations/sql.py.+ * FLAG_EVALUATIONS_TTL_DAYS in that file is longer, so the table holds all of a range that starts this many days back.*/export const FLAG_EVALUATIONS_RETENTION_DAYS = 90-// Start of the oldest day the table still holds. dateStringToDayJs resolves the ranges this is+// Start of the oldest day that FLAG_EVALUATIONS_RETENTION_DAYS covers. dateStringToDayJs resolves the ranges this is// compared to against UTC, so the boundary is UTC too: a browser-local midnight sits hours off it,// which drops the 90-day preset in a timezone ahead of UTC.export function flagEvaluationsRetentionStart(): dayjs.Dayjs {diff --git a/posthog/clickhouse/test/__snapshots__/test_schema.ambr b/posthog/clickhouse/test/__snapshots__/test_schema.ambrindex ade87a65fc05..e27b3b78382e 100644--- a/posthog/clickhouse/test/__snapshots__/test_schema.ambr+++ b/posthog/clickhouse/test/__snapshots__/test_schema.ambr@@ -8294,8 +8294,8 @@)ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/77f1df52-4b43-11e9-910f-b8ca3a9b9f3e_{shard}/posthog.flag_evaluations', '{replica}', _timestamp)- -- Monthly, matching the events family. Daily partitions would put ~90 of them- -- under a 90-day TTL, and enough parts across them to strain merges. The cost+ -- Monthly, matching the events family. Daily partitions would put one partition+ -- under the TTL for each retained day, and enough parts across them to strain merges. The cost-- is coarser expiry: with ttl_only_drop_parts a part only drops once its newest-- row expires, so rows survive up to a month past the TTL.PARTITION BY toYYYYMM(timestamp)@@ -14185,8 +14185,8 @@)ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/77f1df52-4b43-11e9-910f-b8ca3a9b9f3e_{shard}/posthog.flag_evaluations', '{replica}', _timestamp)- -- Monthly, matching the events family. Daily partitions would put ~90 of them- -- under a 90-day TTL, and enough parts across them to strain merges. The cost+ -- Monthly, matching the events family. Daily partitions would put one partition+ -- under the TTL for each retained day, and enough parts across them to strain merges. The cost-- is coarser expiry: with ttl_only_drop_parts a part only drops once its newest-- row expires, so rows survive up to a month past the TTL.PARTITION BY toYYYYMM(timestamp)diff --git a/posthog/dags/tests/test_flag_evaluations_backfill.py b/posthog/dags/tests/test_flag_evaluations_backfill.pyindex 5cc4d887952a..27da95f7e9d7 100644--- a/posthog/dags/tests/test_flag_evaluations_backfill.py+++ b/posthog/dags/tests/test_flag_evaluations_backfill.py@@ -144,7 +144,7 @@ SOURCE_EVENTS = [# Twelve hours before now is always after the start of yesterday, even when UTC midnight passes# between seeding and the run. A row from exactly one day ago would move into the window then.flag_called("half_a_day_ago", TEAM_ONE, timedelta(hours=12)),- flag_called("past_retention", TEAM_ONE, timedelta(days=91)),+ flag_called("past_retention", TEAM_ONE, timedelta(days=93)),]# The job stops unless each Kafka partition in flag_evaluations delivered a row within the@@ -581,7 +581,7 @@ def test_backfill_ignores_a_blocking_run_that_finished_while_it_waited_for_disk(assert totals == ShardBackfillTotals(days=1, rows=5)-# The earliest day the TTL keeps is 2025-12-11 on 2026-03-10, and 2025-12-12 on 2026-03-11.+# The earliest day the TTL keeps is 2025-12-09 on 2026-03-10, and 2025-12-10 on 2026-03-11.@pytest.mark.parametrize("start, step, step_day, effect",[@@ -595,21 +595,21 @@ def test_backfill_ignores_a_blocking_run_that_finished_while_it_waited_for_disk(pytest.param(datetime(2026, 3, 10, 23, tzinfo=UTC),"wait_for_parts_to_merge",- date(2025, 12, 11),+ date(2025, 12, 9),"pass_midnight",id="midnight_passes_during_the_waits_before_the_copy",),pytest.param(datetime(2026, 3, 11, 1, tzinfo=UTC),"wait_for_parts_to_merge",- date(2025, 12, 11),+ date(2025, 12, 9),"fail",id="day_expired_before_its_waits",),],)def test_backfill_stops_at_the_first_expired_day(start: datetime, step: str, step_day: date, effect: str) -> None:- days = [date(2026, 3, 9), date(2025, 12, 12), date(2025, 12, 11)]+ days = [date(2026, 3, 9), date(2025, 12, 10), date(2025, 12, 9)]with time_machine.travel(start, tick=False) as traveller:@@ -803,7 +803,7 @@ def test_backfill_waits_while_clickhouse_moves_parts_off_a_full_disk([pytest.param({"end_date": "2026-03-10"}, id="end_date_after_yesterday"),pytest.param({"start_date": "2026-03-01", "end_date": "2026-03-01"}, id="empty_window"),- pytest.param({"start_date": "2025-12-10"}, id="start_date_already_expired"),+ pytest.param({"start_date": "2025-12-08"}, id="start_date_already_expired"),],)def test_resolve_backfill_days_rejects_an_unsafe_window(overrides: dict[str, Any]) -> None:@@ -817,7 +817,7 @@ def test_resolve_backfill_days_rejects_an_unsafe_window(overrides: dict[str, Anypytest.param({"start_date": "2026-03-06", "end_date": "2026-03-09"}, date(2026, 3, 8), date(2026, 3, 6), 3, id="explicit"),- pytest.param({}, date(2026, 3, 8), date(2025, 12, 11), 88, id="default"),+ pytest.param({}, date(2026, 3, 8), date(2025, 12, 9), 90, id="default"),],)def test_resolve_backfill_days_lists_each_day_newest_first(diff --git a/posthog/hogql_queries/events_query_runner.py b/posthog/hogql_queries/events_query_runner.pyindex af7f711ef7ef..7c9f684e7908 100644--- a/posthog/hogql_queries/events_query_runner.py+++ b/posthog/hogql_queries/events_query_runner.py@@ -55,7 +55,7 @@ from posthog.hogql_queries.query_runner import AnalyticsQueryRunner, QueryRunnerfrom posthog.hogql_queries.utils.person_display_name import person_display_name_property_exprsfrom posthog.models import Person, PropertyDefinitionfrom posthog.models.element import chain_to_elements-from posthog.models.flag_evaluations.sql import FLAG_EVALUATIONS_SOURCE_EVENT, FLAG_EVALUATIONS_TTL_DAYS+from posthog.models.flag_evaluations.sql import FLAG_EVALUATIONS_RETENTION_DAYS, FLAG_EVALUATIONS_SOURCE_EVENTfrom posthog.models.person.person import MAX_LIMIT_DISTINCT_IDS, get_distinct_ids_for_subqueryfrom posthog.models.person.util import get_person_by_pk_or_uuid, get_persons_mapped_by_distinct_idfrom posthog.personhog_client.caller_tag import personhog_caller_tag@@ -125,9 +125,8 @@ FLAG_EVALUATIONS_LIST_TABLE = EventsListTable(joined_fields=EVENTS_LIST_JOINED_FIELDS,# flag_evaluations drops a monthly part only after its newest row passes the TTL. Rows older than the TTL can# therefore remain for up to a month. This bound ends the list at the first day that the Usage tab charts show.- # The TTL has already expired the rows of that day when today starts, so the list can show only part of that day.- # The day is empty after a TTL merge drops every part that holds it.- retention_days=FLAG_EVALUATIONS_TTL_DAYS,+ # The TTL is longer than the retention, so the table holds that day in full.+ retention_days=FLAG_EVALUATIONS_RETENTION_DAYS,cache_key_variant="_flag_evaluations",)diff --git a/posthog/hogql_queries/test/test_events_query_runner.py b/posthog/hogql_queries/test/test_events_query_runner.pyindex 3021df6b536c..9736d01c6cc3 100644--- a/posthog/hogql_queries/test/test_events_query_runner.py+++ b/posthog/hogql_queries/test/test_events_query_runner.py@@ -45,7 +45,7 @@ from posthog.date_util import start_of_dayfrom posthog.hogql_queries.events_query_runner import EventsQueryRunnerfrom posthog.models import Element, Organization, OrganizationMembership, PropertyDefinition, Teamfrom posthog.models.event.util import events_only_in_active_schema-from posthog.models.flag_evaluations.sql import FLAG_EVALUATIONS_TTL_DAYS+from posthog.models.flag_evaluations.sql import FLAG_EVALUATIONS_RETENTION_DAYSfrom posthog.models.group.util import create_groupfrom posthog.models.instance_setting import override_instance_configfrom posthog.models.person.util import get_person_by_distinct_id@@ -1631,10 +1631,13 @@ class TestEventsQueryRunner(ClickhouseTestMixin, APIBaseTest):self._insert_flag_evaluation("first-retained-day-user",uuid.uuid4(),- timestamp=start_of_day(FLAG_CALL_TIMESTAMP - timedelta(days=FLAG_EVALUATIONS_TTL_DAYS)),+ timestamp=start_of_day(FLAG_CALL_TIMESTAMP - timedelta(days=FLAG_EVALUATIONS_RETENTION_DAYS)),)self._insert_flag_evaluation(- "expired-user", uuid.uuid4(), timestamp=FLAG_CALL_TIMESTAMP - timedelta(days=FLAG_EVALUATIONS_TTL_DAYS + 30)+ "past-retention-user",+ uuid.uuid4(),+ timestamp=start_of_day(FLAG_CALL_TIMESTAMP - timedelta(days=FLAG_EVALUATIONS_RETENTION_DAYS))+ - timedelta(seconds=1),)with time_machine.travel("2020-01-11T12:01:00Z", tick=False):diff --git a/posthog/models/flag_evaluations/sql.py b/posthog/models/flag_evaluations/sql.pyindex b990a21d3ba4..fc39aa9d9de1 100644--- a/posthog/models/flag_evaluations/sql.py+++ b/posthog/models/flag_evaluations/sql.py@@ -23,8 +23,8 @@ from posthog.kafka_client.topics import KAFKA_CLICKHOUSE_FLAG_EVALUATIONS# person properties, lifecycle insights read the person's created_at, and a# persons join at query time does not fit in memory on the largest teams.# person_mode tells a call processed without a person profile from a person with-# no properties, because both store '{}' in person_properties. The 90-day TTL is-# what makes rows that wide affordable.+# no properties, because both store '{}' in person_properties. The TTL is what+# makes rows that wide affordable.## Naming convention follows the sharded main-cluster table family (see heatmaps):# * `sharded_flag_evaluations` — sharded ReplicatedReplacingMergeTree on DATA nodes.@@ -40,8 +40,17 @@ FLAG_EVALUATIONS_WRITABLE_TABLE = f"writable_{FLAG_EVALUATIONS_TABLE}"KAFKA_FLAG_EVALUATIONS_TABLE = f"kafka_{FLAG_EVALUATIONS_TABLE}"FLAG_EVALUATIONS_MV_TABLE = f"{FLAG_EVALUATIONS_TABLE}_mv"+# PostHog tells customers that this table keeps this many days. The frontend mirrors this value as+# FLAG_EVALUATIONS_RETENTION_DAYS in flagEvaluationsTable.ts. Change both together.+FLAG_EVALUATIONS_RETENTION_DAYS = 90++# A row dated day D expires when day D + FLAG_EVALUATIONS_TTL_DAYS starts in UTC. A "Last 90 days" range starts at+# midnight 90 days back in the team's timezone. That midnight falls on the UTC day 90 or 91 days back, depending on+# the timezone and the time of day. A 92-day TTL keeps every UTC day that such a range covers.+# This value sets the TTL only when the table is created. Changing the TTL of an existing table takes a+# migration, such as 0362_flag_evaluations_ttl_92_days.# The Node fork step mirrors this value in fork-flag-evaluations-step.ts. Change both together.-FLAG_EVALUATIONS_TTL_DAYS = 90+FLAG_EVALUATIONS_TTL_DAYS = 92# The only event this table stores. posthog/models/deletion_targets.py uses it to skip this table# for deletion requests that name other events, so a producer writing a second event name here has@@ -198,7 +207,7 @@ _FLAG_EVALUATIONS_PROXY_PROPERTY_GROUP_COLUMNS = _single_line(# person, session, or flags-service request). The minmax on inserted_at serves# incremental consumers that checkpoint on it: partitioning is on timestamp, so an# inserted_at range predicate prunes no partitions on its own and would otherwise-# read all 90 days. A skip index only covers parts written after it exists, so+# read every retained day. A skip index only covers parts written after it exists, so# retrofitting one means a full MATERIALIZE INDEX mutation — much cheaper to# declare up front._FLAG_EVALUATIONS_INDEXES = """@@ -238,8 +247,8 @@ CREATE TABLE IF NOT EXISTS {FLAG_EVALUATIONS_DATA_TABLE}{KAFKA_COLUMNS_WITH_PARTITION})ENGINE = {FLAG_EVALUATIONS_DATA_TABLE_ENGINE()}--- Monthly, matching the events family. Daily partitions would put ~90 of them--- under a 90-day TTL, and enough parts across them to strain merges. The cost+-- Monthly, matching the events family. Daily partitions would put one partition+-- under the TTL for each retained day, and enough parts across them to strain merges. The cost-- is coarser expiry: with ttl_only_drop_parts a part only drops once its newest-- row expires, so rows survive up to a month past the TTL.PARTITION BY toYYYYMM(timestamp)