Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

feat(flags): read flag_evaluations with a 92-day TTL (#124555)

Phil Haackf88be17733b3
diff --git a/docs/internal/clickhouse-deletion-coverage.md b/docs/internal/clickhouse-deletion-coverage.md
index 82bc0e15d2ef..3dd5932f74e9 100644
--- a/docs/internal/clickhouse-deletion-coverage.md
+++ b/docs/internal/clickhouse-deletion-coverage.md
@@ -185,7 +185,7 @@ A table created before the switch to `ReplacingMergeTree` keeps every duplicate,
#### Rows past the TTL
-A request window can reach rows past the 90-day TTL that a part still holds, because `ttl_only_drop_parts = 1` drops a part only once its newest row expires.
+A request window can reach rows past the 92-day TTL that a part still holds, because `ttl_only_drop_parts = 1` drops a part only once its newest row expires.
The copy stages those rows like any others.
A reingested part that holds only expired rows can then drop before the reingest or the shard verify counts it.
`DeletionTarget.ttl_days` makes both checks compare only rows at least two days short of their TTL, on the table side and the staged side alike.
@@ -218,7 +218,7 @@ The job finds typed columns through their comments in `system.columns`, not thro
### Immediate event removal with a HogQL predicate does not sweep `flag_evaluations`
`get_event_removal_shards` leaves `flag_evaluations` out of the sweep for an immediate request that carries a HogQL predicate.
-A row with an events copy ages out with the table's TTL. The table partitions by month with `ttl_only_drop_parts = 1`, so a part drops only once its newest row expires: the real wait is up to about 120 days, not the 90-day TTL.
+A row with an events copy ages out with the table's TTL. The table partitions by month with `ttl_only_drop_parts = 1`, so a part drops only once its newest row expires: the real wait is up to about 122 days, not the 92-day TTL.
The events copies that a deferred request deletes through do not help here, because the immediate path deletes the events rows without queueing their uuids.
A row with no events copy is checked instead.
The request refuses while such a row matches the predicate compiled for this table (below), or matches the request's other criteria when the predicate cannot run on this table.
diff --git a/frontend/src/scenes/feature-flags/flagEvaluationsTable.ts b/frontend/src/scenes/feature-flags/flagEvaluationsTable.ts
index 6f18501de196..968b2339e080 100644
--- a/frontend/src/scenes/feature-flags/flagEvaluationsTable.ts
+++ b/frontend/src/scenes/feature-flags/flagEvaluationsTable.ts
@@ -24,12 +24,13 @@ export function storesFlagCallsOutsideEvents(team: TeamPublicType | TeamType | n
}
/**
- * How long a row stays in flag_evaluations. The events table keeps $feature_flag_called forever.
- * Keep it equal to FLAG_EVALUATIONS_TTL_DAYS in posthog/models/flag_evaluations/sql.py.
+ * How long PostHog tells customers that flag_evaluations keeps a row. The events table keeps $feature_flag_called
+ * forever. Keep it equal to FLAG_EVALUATIONS_RETENTION_DAYS in posthog/models/flag_evaluations/sql.py.
+ * FLAG_EVALUATIONS_TTL_DAYS in that file is longer, so the table holds all of a range that starts this many days back.
*/
export const FLAG_EVALUATIONS_RETENTION_DAYS = 90
-// Start of the oldest day the table still holds. dateStringToDayJs resolves the ranges this is
+// Start of the oldest day that FLAG_EVALUATIONS_RETENTION_DAYS covers. dateStringToDayJs resolves the ranges this is
// compared to against UTC, so the boundary is UTC too: a browser-local midnight sits hours off it,
// which drops the 90-day preset in a timezone ahead of UTC.
export function flagEvaluationsRetentionStart(): dayjs.Dayjs {
diff --git a/posthog/clickhouse/test/__snapshots__/test_schema.ambr b/posthog/clickhouse/test/__snapshots__/test_schema.ambr
index ade87a65fc05..e27b3b78382e 100644
--- a/posthog/clickhouse/test/__snapshots__/test_schema.ambr
+++ b/posthog/clickhouse/test/__snapshots__/test_schema.ambr
@@ -8294,8 +8294,8 @@
)
ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/77f1df52-4b43-11e9-910f-b8ca3a9b9f3e_{shard}/posthog.flag_evaluations', '{replica}', _timestamp)
- -- Monthly, matching the events family. Daily partitions would put ~90 of them
- -- under a 90-day TTL, and enough parts across them to strain merges. The cost
+ -- Monthly, matching the events family. Daily partitions would put one partition
+ -- under the TTL for each retained day, and enough parts across them to strain merges. The cost
-- is coarser expiry: with ttl_only_drop_parts a part only drops once its newest
-- row expires, so rows survive up to a month past the TTL.
PARTITION BY toYYYYMM(timestamp)
@@ -14185,8 +14185,8 @@
)
ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/77f1df52-4b43-11e9-910f-b8ca3a9b9f3e_{shard}/posthog.flag_evaluations', '{replica}', _timestamp)
- -- Monthly, matching the events family. Daily partitions would put ~90 of them
- -- under a 90-day TTL, and enough parts across them to strain merges. The cost
+ -- Monthly, matching the events family. Daily partitions would put one partition
+ -- under the TTL for each retained day, and enough parts across them to strain merges. The cost
-- is coarser expiry: with ttl_only_drop_parts a part only drops once its newest
-- row expires, so rows survive up to a month past the TTL.
PARTITION BY toYYYYMM(timestamp)
diff --git a/posthog/dags/tests/test_flag_evaluations_backfill.py b/posthog/dags/tests/test_flag_evaluations_backfill.py
index 5cc4d887952a..27da95f7e9d7 100644
--- a/posthog/dags/tests/test_flag_evaluations_backfill.py
+++ b/posthog/dags/tests/test_flag_evaluations_backfill.py
@@ -144,7 +144,7 @@ SOURCE_EVENTS = [
# Twelve hours before now is always after the start of yesterday, even when UTC midnight passes
# between seeding and the run. A row from exactly one day ago would move into the window then.
flag_called("half_a_day_ago", TEAM_ONE, timedelta(hours=12)),
- flag_called("past_retention", TEAM_ONE, timedelta(days=91)),
+ flag_called("past_retention", TEAM_ONE, timedelta(days=93)),
]
# The job stops unless each Kafka partition in flag_evaluations delivered a row within the
@@ -581,7 +581,7 @@ def test_backfill_ignores_a_blocking_run_that_finished_while_it_waited_for_disk(
assert totals == ShardBackfillTotals(days=1, rows=5)
-# The earliest day the TTL keeps is 2025-12-11 on 2026-03-10, and 2025-12-12 on 2026-03-11.
+# The earliest day the TTL keeps is 2025-12-09 on 2026-03-10, and 2025-12-10 on 2026-03-11.
@pytest.mark.parametrize(
"start, step, step_day, effect",
[
@@ -595,21 +595,21 @@ def test_backfill_ignores_a_blocking_run_that_finished_while_it_waited_for_disk(
pytest.param(
datetime(2026, 3, 10, 23, tzinfo=UTC),
"wait_for_parts_to_merge",
- date(2025, 12, 11),
+ date(2025, 12, 9),
"pass_midnight",
id="midnight_passes_during_the_waits_before_the_copy",
),
pytest.param(
datetime(2026, 3, 11, 1, tzinfo=UTC),
"wait_for_parts_to_merge",
- date(2025, 12, 11),
+ date(2025, 12, 9),
"fail",
id="day_expired_before_its_waits",
),
],
)
def test_backfill_stops_at_the_first_expired_day(start: datetime, step: str, step_day: date, effect: str) -> None:
- days = [date(2026, 3, 9), date(2025, 12, 12), date(2025, 12, 11)]
+ days = [date(2026, 3, 9), date(2025, 12, 10), date(2025, 12, 9)]
with time_machine.travel(start, tick=False) as traveller:
@@ -803,7 +803,7 @@ def test_backfill_waits_while_clickhouse_moves_parts_off_a_full_disk(
[
pytest.param({"end_date": "2026-03-10"}, id="end_date_after_yesterday"),
pytest.param({"start_date": "2026-03-01", "end_date": "2026-03-01"}, id="empty_window"),
- pytest.param({"start_date": "2025-12-10"}, id="start_date_already_expired"),
+ pytest.param({"start_date": "2025-12-08"}, id="start_date_already_expired"),
],
)
def test_resolve_backfill_days_rejects_an_unsafe_window(overrides: dict[str, Any]) -> None:
@@ -817,7 +817,7 @@ def test_resolve_backfill_days_rejects_an_unsafe_window(overrides: dict[str, Any
pytest.param(
{"start_date": "2026-03-06", "end_date": "2026-03-09"}, date(2026, 3, 8), date(2026, 3, 6), 3, id="explicit"
),
- pytest.param({}, date(2026, 3, 8), date(2025, 12, 11), 88, id="default"),
+ pytest.param({}, date(2026, 3, 8), date(2025, 12, 9), 90, id="default"),
],
)
def test_resolve_backfill_days_lists_each_day_newest_first(
diff --git a/posthog/hogql_queries/events_query_runner.py b/posthog/hogql_queries/events_query_runner.py
index af7f711ef7ef..7c9f684e7908 100644
--- a/posthog/hogql_queries/events_query_runner.py
+++ b/posthog/hogql_queries/events_query_runner.py
@@ -55,7 +55,7 @@ from posthog.hogql_queries.query_runner import AnalyticsQueryRunner, QueryRunner
from posthog.hogql_queries.utils.person_display_name import person_display_name_property_exprs
from posthog.models import Person, PropertyDefinition
from posthog.models.element import chain_to_elements
-from posthog.models.flag_evaluations.sql import FLAG_EVALUATIONS_SOURCE_EVENT, FLAG_EVALUATIONS_TTL_DAYS
+from posthog.models.flag_evaluations.sql import FLAG_EVALUATIONS_RETENTION_DAYS, FLAG_EVALUATIONS_SOURCE_EVENT
from posthog.models.person.person import MAX_LIMIT_DISTINCT_IDS, get_distinct_ids_for_subquery
from posthog.models.person.util import get_person_by_pk_or_uuid, get_persons_mapped_by_distinct_id
from posthog.personhog_client.caller_tag import personhog_caller_tag
@@ -125,9 +125,8 @@ FLAG_EVALUATIONS_LIST_TABLE = EventsListTable(
joined_fields=EVENTS_LIST_JOINED_FIELDS,
# flag_evaluations drops a monthly part only after its newest row passes the TTL. Rows older than the TTL can
# therefore remain for up to a month. This bound ends the list at the first day that the Usage tab charts show.
- # The TTL has already expired the rows of that day when today starts, so the list can show only part of that day.
- # The day is empty after a TTL merge drops every part that holds it.
- retention_days=FLAG_EVALUATIONS_TTL_DAYS,
+ # The TTL is longer than the retention, so the table holds that day in full.
+ retention_days=FLAG_EVALUATIONS_RETENTION_DAYS,
cache_key_variant="_flag_evaluations",
)
diff --git a/posthog/hogql_queries/test/test_events_query_runner.py b/posthog/hogql_queries/test/test_events_query_runner.py
index 3021df6b536c..9736d01c6cc3 100644
--- a/posthog/hogql_queries/test/test_events_query_runner.py
+++ b/posthog/hogql_queries/test/test_events_query_runner.py
@@ -45,7 +45,7 @@ from posthog.date_util import start_of_day
from posthog.hogql_queries.events_query_runner import EventsQueryRunner
from posthog.models import Element, Organization, OrganizationMembership, PropertyDefinition, Team
from posthog.models.event.util import events_only_in_active_schema
-from posthog.models.flag_evaluations.sql import FLAG_EVALUATIONS_TTL_DAYS
+from posthog.models.flag_evaluations.sql import FLAG_EVALUATIONS_RETENTION_DAYS
from posthog.models.group.util import create_group
from posthog.models.instance_setting import override_instance_config
from posthog.models.person.util import get_person_by_distinct_id
@@ -1631,10 +1631,13 @@ class TestEventsQueryRunner(ClickhouseTestMixin, APIBaseTest):
self._insert_flag_evaluation(
"first-retained-day-user",
uuid.uuid4(),
- timestamp=start_of_day(FLAG_CALL_TIMESTAMP - timedelta(days=FLAG_EVALUATIONS_TTL_DAYS)),
+ timestamp=start_of_day(FLAG_CALL_TIMESTAMP - timedelta(days=FLAG_EVALUATIONS_RETENTION_DAYS)),
)
self._insert_flag_evaluation(
- "expired-user", uuid.uuid4(), timestamp=FLAG_CALL_TIMESTAMP - timedelta(days=FLAG_EVALUATIONS_TTL_DAYS + 30)
+ "past-retention-user",
+ uuid.uuid4(),
+ timestamp=start_of_day(FLAG_CALL_TIMESTAMP - timedelta(days=FLAG_EVALUATIONS_RETENTION_DAYS))
+ - timedelta(seconds=1),
)
with time_machine.travel("2020-01-11T12:01:00Z", tick=False):
diff --git a/posthog/models/flag_evaluations/sql.py b/posthog/models/flag_evaluations/sql.py
index b990a21d3ba4..fc39aa9d9de1 100644
--- a/posthog/models/flag_evaluations/sql.py
+++ b/posthog/models/flag_evaluations/sql.py
@@ -23,8 +23,8 @@ from posthog.kafka_client.topics import KAFKA_CLICKHOUSE_FLAG_EVALUATIONS
# person properties, lifecycle insights read the person's created_at, and a
# persons join at query time does not fit in memory on the largest teams.
# person_mode tells a call processed without a person profile from a person with
-# no properties, because both store '{}' in person_properties. The 90-day TTL is
-# what makes rows that wide affordable.
+# no properties, because both store '{}' in person_properties. The TTL is what
+# makes rows that wide affordable.
#
# Naming convention follows the sharded main-cluster table family (see heatmaps):
# * `sharded_flag_evaluations` — sharded ReplicatedReplacingMergeTree on DATA nodes.
@@ -40,8 +40,17 @@ FLAG_EVALUATIONS_WRITABLE_TABLE = f"writable_{FLAG_EVALUATIONS_TABLE}"
KAFKA_FLAG_EVALUATIONS_TABLE = f"kafka_{FLAG_EVALUATIONS_TABLE}"
FLAG_EVALUATIONS_MV_TABLE = f"{FLAG_EVALUATIONS_TABLE}_mv"
+# PostHog tells customers that this table keeps this many days. The frontend mirrors this value as
+# FLAG_EVALUATIONS_RETENTION_DAYS in flagEvaluationsTable.ts. Change both together.
+FLAG_EVALUATIONS_RETENTION_DAYS = 90
+
+# A row dated day D expires when day D + FLAG_EVALUATIONS_TTL_DAYS starts in UTC. A "Last 90 days" range starts at
+# midnight 90 days back in the team's timezone. That midnight falls on the UTC day 90 or 91 days back, depending on
+# the timezone and the time of day. A 92-day TTL keeps every UTC day that such a range covers.
+# This value sets the TTL only when the table is created. Changing the TTL of an existing table takes a
+# migration, such as 0362_flag_evaluations_ttl_92_days.
# The Node fork step mirrors this value in fork-flag-evaluations-step.ts. Change both together.
-FLAG_EVALUATIONS_TTL_DAYS = 90
+FLAG_EVALUATIONS_TTL_DAYS = 92
# The only event this table stores. posthog/models/deletion_targets.py uses it to skip this table
# for deletion requests that name other events, so a producer writing a second event name here has
@@ -198,7 +207,7 @@ _FLAG_EVALUATIONS_PROXY_PROPERTY_GROUP_COLUMNS = _single_line(
# person, session, or flags-service request). The minmax on inserted_at serves
# incremental consumers that checkpoint on it: partitioning is on timestamp, so an
# inserted_at range predicate prunes no partitions on its own and would otherwise
-# read all 90 days. A skip index only covers parts written after it exists, so
+# read every retained day. A skip index only covers parts written after it exists, so
# retrofitting one means a full MATERIALIZE INDEX mutation — much cheaper to
# declare up front.
_FLAG_EVALUATIONS_INDEXES = """
@@ -238,8 +247,8 @@ CREATE TABLE IF NOT EXISTS {FLAG_EVALUATIONS_DATA_TABLE}
{KAFKA_COLUMNS_WITH_PARTITION}
)
ENGINE = {FLAG_EVALUATIONS_DATA_TABLE_ENGINE()}
--- Monthly, matching the events family. Daily partitions would put ~90 of them
--- under a 90-day TTL, and enough parts across them to strain merges. The cost
+-- Monthly, matching the events family. Daily partitions would put one partition
+-- under the TTL for each retained day, and enough parts across them to strain merges. The cost
-- is coarser expiry: with ttl_only_drop_parts a part only drops once its newest
-- row expires, so rows survive up to a month past the TTL.
PARTITION BY toYYYYMM(timestamp)