Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

feat(retention): add default period deletion passes, off by default (#128746)

Andy Zhao29a597c36e63
diff --git a/.github/scripts/check-idor-model-coverage.py b/.github/scripts/check-idor-model-coverage.py
index d951b4765ae5..e7808978d70f 100644
--- a/.github/scripts/check-idor-model-coverage.py
+++ b/.github/scripts/check-idor-model-coverage.py
@@ -227,6 +227,9 @@ def get_scoped_models() -> tuple[dict[str, set[str]], set[str], set[str], set[st
# Staff-only OneToOne extension keyed on the team, edited in Django admin and read by the
# events cleanup schedule; no endpoint looks it up by a user-supplied ID.
"TeamEventsRetentionConfig",
+ # OneToOne extension keyed on the team, written and read only by the events cleanup schedule;
+ # no endpoint looks it up by a user-supplied ID.
+ "TeamEventsPlanDeletion",
"TeamFeatureFlagsConfig",
# Staff-only OneToOne extension keyed on the team, written by a management command and read
# by the realtime cohort pipeline; no endpoint looks it up by a user-supplied ID.
diff --git a/.semgrep/rules/security/idor-team-scoped-models.yaml b/.semgrep/rules/security/idor-team-scoped-models.yaml
index 03fa34f80597..465c8ecfb4a0 100644
--- a/.semgrep/rules/security/idor-team-scoped-models.yaml
+++ b/.semgrep/rules/security/idor-team-scoped-models.yaml
@@ -1306,6 +1306,7 @@ rules:
|TeamDataQualityConfig
|TeamDataWarehouseConfig
|TeamDefaultEvaluationContext
+ |TeamEventsPlanDeletion
|TeamEventsRetentionConfig
|TeamEventVolume
|TeamExperimentsConfig
diff --git a/docs/internal/clickhouse-deletion-coverage.md b/docs/internal/clickhouse-deletion-coverage.md
index d97875cc3190..c5787d7647c4 100644
--- a/docs/internal/clickhouse-deletion-coverage.md
+++ b/docs/internal/clickhouse-deletion-coverage.md
@@ -324,24 +324,39 @@ The job skips any of these tables that does not exist.
Other personal-data tables expire sooner under their own TTL.
It deletes a month only when every event in it is older than the retention. A month that is only partly past the retention waits for a later run, for manual runs too.
-Staff set the retention in Django admin, on the team and organization pages:
+Organization admins set the retention in project and organization settings, and staff set it in Django admin, on the team and organization pages:
- `OrganizationEventsRetentionConfig` holds the organization's default, minimum and maximum months.
- `TeamEventsRetentionConfig` holds a team's own months, which must fall within its organization's range. Saving an organization range that excludes an existing team value fails until that team changes.
-- A team's effective retention is its own value, else its organization's default. A team with neither keeps all its events.
-- This setting is separate from `Team.event_retention_months`, which the billing plan sets and which only hides older events from queries.
+- A team's set period is its own value, else its organization's default. A set period is deleted at every run.
+- A team with no set period uses its default period, which waits for the team's pass, every 3 months.
+- The default period is the plan's period, capped at 24 months when the organization has the `product_analytics_standard_data_retention` feature. Billing gives that feature to current plans only, so older plans default to their whole period. The default never goes below the organization's minimum.
+- `Team.event_retention_months` is the plan's period, which billing sets. A period never deletes past it before the team's pass.
- The Events retention page under Custom tools (`/admin/events-retention/`) lists every team that has a value of its own or belongs to an organization with a config.
It shows each team's billing months (`Team.event_retention_months`), its effective retention, where that comes from, and the first month the cleanup keeps, 100 teams per page.
-`monthly_retention_events_cleanup_job` has four steps, in this order:
-
-- `plan_old_events_cleanup` reads every team's effective retention when the run starts and logs each one as organization, team and months. It then finds the months that hold events for each team and keeps the ones entirely past that team's retention. It logs the teams due in each month.
+`monthly_retention_events_cleanup_job` has six steps:
+
+- `update_plan_deletions` tracks each team's pass period in `TeamEventsPlanDeletion`: the set period, never past the plan's period, else the default period. It gives a row only to the teams where `should_delete_past_default_period` is true, and removes the rows of all other teams. A team without a row gets no pass, and its set period is never capped by its plan.
+ - `should_delete_past_default_period` needs enforcement and the `events-data-retention-deletion` flag. The flag gets each team's plan period and organization, so a rollout can target plans and leave out organizations.
+ - A new row's first pass is at least 3 months out, so a team keeps its events for at least 3 months after the flag turns on.
+ - Turning the flag off removes the team's row at the next run. A run that already finished `update_plan_deletions` still deletes for that team.
+ - A shorter period, from a downgrade or a change in Django admin, moves the team's next pass to at least 3 months out.
+ - It locks the rows of 1000 teams at a time, reads their periods, and writes its changes in one transaction. A period saved in settings during the run is therefore kept. It then hands on the teams whose pass is due.
+- Saving a period in settings writes the row of each team that deletes past its default straight away, so the settings page can show the next deletion. A shorter period moves the next pass to at least 3 months out. The save writes the rows before and after it changes the period, so a row that still holds a period from before a plan change first catches up to the period in use.
+- The cleanup and settings write rows through `reschedule_plan_deletions`, which locks the rows before it reads the periods.
+- Pass dates fall on the 3rd, so the date shown is the run that deletes.
+- `plan_old_events_cleanup` reads every team's set period. A team with a set period uses it. A team whose pass is due uses its pass period, or the shorter of the two. It reads each due pass row again, so a period saved after the run started applies: a longer period raises the row's months, and a shorter one moves the pass past this run. It logs each team's period in months, then finds the months that hold events for each team and keeps the ones entirely past that team's period. It logs the teams due in each month.
- `cleanup_old_events_by_partition` deletes each planned month for the teams due in that month only. A team with 13 months and a team with 18 months share the older months, and the 13-month team also has months of its own.
-- `plan_old_sessions_cleanup` starts after the events are deleted. It finds the sessions partitions entirely past each team's retention, so a month whose events an earlier run deleted still gets its sessions deleted.
+- `plan_old_sessions_cleanup` starts after the events are deleted. It uses the period the events plan used for each team, and finds the sessions partitions entirely past it, so a month whose events an earlier run deleted still gets its sessions deleted.
- `cleanup_old_sessions_by_partition` deletes each planned sessions month for the teams due in it. Sessions partition by the month a session started, so a session that continues into a kept month is deleted with the due month.
+- `advance_plan_deletion_passes` moves the next pass of each team that both plans completed 3 months out, after the sessions deletes succeed. It keeps a later date that a shorter period saved during the run set.
-A manual run can narrow each plan with `team_ids` and `partitions`, set separately on the events and the sessions plan steps. A listed team with no retention is ignored.
-`monthly_retention_events_cleanup_schedule` starts one run on the 3rd of each month at 21:00 local time (Europe/Paris in EU, America/Los_Angeles elsewhere), and skips when no retention is set. It is stopped by default.
+A manual run can narrow each plan with its own `team_ids` and `partitions`. The sessions plan covers only the teams the events plan covers. A listed team with neither a set period nor a pass due is ignored.
+A pass completes only for a team that both plans include, and only when neither sets `partitions`. Otherwise the pass stays due for the next full run.
+The `update_plan_deletions` config has its own `team_ids`, which limits the rows it adds and removes.
+`monthly_retention_events_cleanup_schedule` starts one run on the 3rd of each month at 21:00 local time (Europe/Paris in EU, America/Los_Angeles elsewhere). It is stopped by default.
+Pass dates are dates in that local time zone, so the date shown in settings is the day its deletes start.
The job runs `ensure_no_concurrent_deletes_run` before it plans, and that guard also runs in `deletes_job`. Each job fails at the start while a `deletes_job`, cleanup or squash run is executing, so the two never mutate the events tables at the same time.
Both jobs and `squash_person_overrides` carry the `deletes_job_concurrency` tag. The Dagster deployment settings put a run-queue limit of 1 on that tag, so a second run of any of the three queues instead of starting.
`sharded_events_json` keeps events dated before 2020, because it stores them in its 202001 partition and the delete addresses the event's own month.
diff --git a/frontend/snapshots.yml b/frontend/snapshots.yml
index 3415876d0f40..d53234ac632c 100644
--- a/frontend/snapshots.yml
+++ b/frontend/snapshots.yml
@@ -14583,9 +14583,9 @@ snapshots:
scenes-app-notebooks-run-all-banner--starting--light:
hash: v1.k794b7964.6c7bd37efe1d58d68070782d7d1d0ce20be5abfebe1f829feb0376b584ad1158.CVBhfDqMZqLLuxmDgkiTAS9Vn2SANiX9cOWDF1Ui8hk
scenes-app-notebooks-sql-cell-chart--bar-chart--dark:
- hash: v1.k794b7964.c41b37a93d8837b3553828a9a0404b2339ff572b70d5caa2594cb844920f1a2e.yq6UDrqaC0R-JEjBrOyC1NlAOQ-mE7x8k0dvvrNNNbI
+ hash: v1.k794b7964.640c7f77576390755335ffcb5a4758ae8c40e45e552cc582b83df90703ca3ce6.Z0us6XyXMiI63VquF3aewikkU6DlXcpQaTTDOJeE3CQ
scenes-app-notebooks-sql-cell-chart--bar-chart--light:
- hash: v1.k794b7964.c17bb922582b7e626a3c0c6d52064553eb0746150d4efd364b7d22dcd7234ff7.NZkSVzLm69X_1mCUuYSxBnFA6BwEslWe1q9Brgh7J9M
+ hash: v1.k794b7964.86951626f47e403f310125c96e5350afda1e0c0530378bd424507080e53bb050.hro623qzB_dGhkC3oRrWNGSdJTZzVdPRQvItVmGvfFI
scenes-app-notebooks-sql-cell-chart--line-chart--dark:
hash: v1.k794b7964.d0680bc106516aaf11e4f59df950c72ff97db4a45b842289c38e99982f5270e8.ZCfy8qhd-5cuv7pFdcsLTt9eMSzxmRTeOyX7HlVlhLI
scenes-app-notebooks-sql-cell-chart--line-chart--light:
diff --git a/frontend/src/mocks/handlers.ts b/frontend/src/mocks/handlers.ts
index bfa6a7b814f7..a82153f67b8f 100644
--- a/frontend/src/mocks/handlers.ts
+++ b/frontend/src/mocks/handlers.ts
@@ -107,6 +107,11 @@ export const defaultMocks: Mocks = {
retained_from: null,
docs_url: 'https://posthog.com/docs/data/events-retention',
project_retention_months: null,
+ next_deletion_on: null,
+ next_deletion_before: null,
+ scheduled_pass_on: null,
+ first_pass_on: '2027-02-03',
+ default_period_deletes: false,
organization_retention_months: null,
default_retention_months: 84,
default_is_grandfathered: true,
@@ -114,6 +119,7 @@ export const defaultMocks: Mocks = {
min_retention_months: 1,
max_retention_months: 84,
retention_months_choices: [1, 3, 12, 24, 36, 60, 84],
+ next_monthly_cleanup_on: '2026-11-03',
},
'/api/projects/:team_id/annotations/': EMPTY_PAGINATED_RESPONSE,
'/api/projects/:team_id/event_definitions/': EMPTY_PAGINATED_RESPONSE,
@@ -188,6 +194,7 @@ export const defaultMocks: Mocks = {
min_retention_months: 1,
max_retention_months: 84,
retention_months_choices: [1, 3, 12, 24, 36, 60, 84],
+ next_monthly_cleanup_on: '2026-11-03',
},
'/api/organizations/:organization_id/resource_access': EMPTY_PAGINATED_RESPONSE,
'/api/organizations/:organization_id/members/': toPaginatedResponse([
diff --git a/posthog/admin/console/events_retention_tool.py b/posthog/admin/console/events_retention_tool.py
index d549078a6678..12bfd35baf60 100644
--- a/posthog/admin/console/events_retention_tool.py
+++ b/posthog/admin/console/events_retention_tool.py
@@ -29,7 +29,7 @@ def _source(retention: ConfiguredEventsRetention) -> str:
return "Team setting"
if retention.organization_default_months is not None:
return "Organization default"
- return "Not set, events are kept"
+ return "Not set"
def _organization_row(retention: ConfiguredEventsRetention) -> GroupRow:
@@ -95,8 +95,9 @@ class EventsRetentionTool(ConsoleTool):
Notice(
text=(
"Billing months come from the team's billing plan. They only hide older events in queries. "
- "Retention is the staff setting that the monthly cleanup enforces. The cleanup deletes one "
- "calendar month at a time, and only when every event in that month is older than the retention."
+ "Retention is the period set in settings. The cleanup deletes past it every month. The "
+ "cleanup deletes one calendar month at a time, and only when every event in that month is "
+ "older than the retention."
)
),
Table(
diff --git a/posthog/dags/deletes.py b/posthog/dags/deletes.py
index 4460569ab810..0e093b199ed4 100644
--- a/posthog/dags/deletes.py
+++ b/posthog/dags/deletes.py
@@ -2,13 +2,14 @@ import abc
import time
import uuid
from collections import defaultdict
-from collections.abc import Callable, Sequence
-from dataclasses import dataclass
-from datetime import UTC, datetime, timedelta
+from collections.abc import Callable, Iterator, Sequence
+from dataclasses import dataclass, field
+from datetime import UTC, date, datetime, timedelta
from functools import partial
from django.conf import settings
-from django.db.models import Count, Min, Q
+from django.db import transaction
+from django.db.models import Count, Min, Q, QuerySet
from django.utils import timezone
import dagster
@@ -55,6 +56,17 @@ from posthog.models.deletion_targets import (
surviving_rows_sql,
sweep_clusters,
)
+from posthog.models.events_plan_deletion import (
+ CLEANUP_DAY_OF_MONTH,
+ CLEANUP_HOUR,
+ PlanDeletionState,
+ TeamEventsPlanDeletion,
+ after_pass,
+ cleanup_date,
+ cleanup_timezone,
+ deletion_months,
+ reschedule_plan_deletions,
+)
from posthog.models.events_retention_config import effective_events_retention, whole_month_cutoff
from posthog.models.group.sql import GROUPS_TABLE
from posthog.models.person.sql import (
@@ -72,6 +84,8 @@ from posthog.models.sessions.sql import (
SESSIONS_DATA_TABLE,
TABLE_BASE_NAME as SESSIONS_TABLE_BASE_NAME,
)
+from posthog.models.team import Team
+from posthog.models.team.event_retention import retention_periods, should_delete_past_default_period, team_plan
from posthog.run_mode import run_mode
from products.error_tracking.backend.facade.api import (
@@ -158,15 +172,32 @@ class TeamProductDocumentsDeleteConfig(dagster.Config):
class MonthlyCleanupConfig(dagster.Config):
team_ids: list[int] = pydantic.Field(
default_factory=list,
- description="Only clean up these teams. Empty means every team with an events retention in Django admin. "
- "A listed team with no events retention is ignored.",
+ description="Only clean up these teams. Empty means every team with a period set in settings or a pass due. "
+ "A listed team with neither is ignored.",
)
partitions: list[int] = pydantic.Field(
default_factory=list,
- description="Only clean up these months, as YYYYMM (e.g. [202407]). Empty means every month that is due.",
+ description="Only clean up these months, as YYYYMM (e.g. [202407]). Empty means every month that is due. "
+ "A plan-period pass completes only in a run that lists no months.",
+ )
+
+
+_TEAM_CHUNK_SIZE = 1000
+
+
+class PlanDeletionConfig(dagster.Config):
+ team_ids: list[int] = pydantic.Field(
+ default_factory=list,
+ description="Only admit, update and pass these teams. Empty means every team.",
)
+@frozen
+class PlanDeletionPasses:
+ run_on: date
+ due_team_ids: list[int]
+
+
@frozen
class PartitionCleanup:
partition: int
@@ -176,11 +207,14 @@ class PartitionCleanup:
@frozen
class OldEventsCleanupPlan:
partitions: list[PartitionCleanup]
+ period_months_by_team: dict[int, int] = field(default_factory=dict)
+ completed_pass_team_ids: list[int] = field(default_factory=list)
@frozen
class OldSessionsCleanupPlan:
partitions: list[PartitionCleanup]
+ completed_pass_team_ids: list[int] = field(default_factory=list)
@frozen
@@ -1568,30 +1602,113 @@ def delete_team_product_documents_job():
delete_team_product_documents()
+def _plan_teams(teams: QuerySet[Team]) -> Iterator[Team]:
+ return (
+ teams.order_by("organization_id", "id")
+ .only("id", "organization_id", "event_retention_months")
+ .iterator(chunk_size=_TEAM_CHUNK_SIZE)
+ )
+
+
@dagster.op(ins={"start_after": dagster.In(dagster.Nothing)})
+def update_plan_deletions(context: dagster.OpExecutionContext, config: PlanDeletionConfig) -> PlanDeletionPasses:
+ """Give a pass to each team that deletes past its default period, remove the others', and find the passes due."""
+ today = cleanup_date()
+ teams = Team.objects.filter(id__in=config.team_ids) if config.team_ids else Team.objects.all()
+ plans = [team_plan(team) for team in _plan_teams(teams) if should_delete_past_default_period(team)]
+ rows = (
+ TeamEventsPlanDeletion.objects.filter(team_id__in=config.team_ids)
+ if config.team_ids
+ else TeamEventsPlanDeletion.objects.all()
+ )
+ removed = sorted(set(rows.values_list("team_id", flat=True)) - {plan.team_id for plan in plans})
+ for team_ids in chunked(removed, _TEAM_CHUNK_SIZE):
+ TeamEventsPlanDeletion.objects.filter(team_id__in=team_ids).delete()
+
+ states: dict[int, PlanDeletionState] = {}
+ for chunk in chunked(plans, _TEAM_CHUNK_SIZE):
+ with transaction.atomic():
+ states.update(reschedule_plan_deletions(chunk, today))
+ due_team_ids = sorted(team_id for team_id, state in states.items() if state.next_deletion_on <= today)
+
+ context.add_output_metadata(
+ {
+ "deleting_teams": dagster.MetadataValue.int(len(plans)),
+ "removed_passes": dagster.MetadataValue.int(len(removed)),
+ "passes_due": dagster.MetadataValue.int(len(due_team_ids)),
+ }
+ )
+ return PlanDeletionPasses(run_on=today, due_team_ids=due_team_ids)
+
+
+def _still_due_pass_months(passes: PlanDeletionPasses) -> dict[int, int]:
+ rows = TeamEventsPlanDeletion.objects.filter(team_id__in=passes.due_team_ids, next_deletion_on__lte=passes.run_on)
+ pass_months = dict(rows.values_list("team_id", "months"))
+ periods = retention_periods([team_plan(team) for team in _plan_teams(Team.objects.filter(id__in=pass_months))])
+ return {team_id: max(months, periods.get(team_id, months)) for team_id, months in pass_months.items()}
+
+
+def _planned_partitions(
+ context: dagster.OpExecutionContext, team_months: set[tuple[int, int]], cutoff_partitions: dict[int, int]
+) -> list[PartitionCleanup]:
+ teams_by_partition: dict[int, list[int]] = defaultdict(list)
+ for team_id, partition in sorted(team_months):
+ if partition < cutoff_partitions[team_id]:
+ teams_by_partition[partition].append(team_id)
+ partitions = [
+ PartitionCleanup(partition=partition, team_ids=team_ids)
+ for partition, team_ids in sorted(teams_by_partition.items(), reverse=True)
+ ]
+ for item in partitions:
+ context.log.info(f"{item.partition}: teams {item.team_ids}")
+ context.add_output_metadata(
+ {
+ "teams": dagster.MetadataValue.int(len(cutoff_partitions)),
+ "partitions": dagster.MetadataValue.int(len(partitions)),
+ }
+ )
+ return partitions
+
+
+def _deleted_partitions_metadata(partitions: list[PartitionCleanup]) -> dict[str, dagster.MetadataValue]:
+ return {
+ "partitions_processed": dagster.MetadataValue.int(len(partitions)),
+ "team_ids": dagster.MetadataValue.text(
+ ", ".join(str(tid) for tid in sorted({tid for item in partitions for tid in item.team_ids}))
+ ),
+ }
+
+
def plan_old_events_cleanup(
context: dagster.OpExecutionContext,
config: MonthlyCleanupConfig,
cluster: dagster.ResourceParam[ClickhouseCluster],
+ passes: PlanDeletionPasses,
) -> OldEventsCleanupPlan:
- """Find every team with an events retention, and for each month the teams whose events in it are all past it."""
- retention = effective_events_retention()
+ """Find each team's deletion period, and for each month the teams whose events in it are all past it."""
+ setting_by_team = {item.team_id: item for item in effective_events_retention()}
+ pass_months = _still_due_pass_months(passes)
if config.team_ids:
- if ignored := sorted(set(config.team_ids) - {item.team_id for item in retention}):
- context.log.info(f"Ignoring teams with no events retention: {ignored}")
- retention = [item for item in retention if item.team_id in config.team_ids]
- if not retention:
+ setting_by_team = {team_id: item for team_id, item in setting_by_team.items() if team_id in config.team_ids}
+ pass_months = {team_id: months for team_id, months in pass_months.items() if team_id in config.team_ids}
+ completed_pass_team_ids = [] if config.partitions else sorted(pass_months)
+ period_months = {
+ team_id: months
+ for team_id in sorted(setting_by_team.keys() | pass_months.keys())
+ if (months := deletion_months(setting_by_team.get(team_id), pass_months.get(team_id))) is not None
+ }
+ if ignored := sorted(set(config.team_ids) - set(period_months)):
+ context.log.info(f"Ignoring teams with neither a period set in settings nor a pass due: {ignored}")
+ if not period_months:
context.log.info("No team has an events retention to enforce")
- return OldEventsCleanupPlan(partitions=[])
+ return OldEventsCleanupPlan(partitions=[], completed_pass_team_ids=completed_pass_team_ids)
now = datetime.now(UTC).replace(tzinfo=None)
team_cutoffs: dict[int, datetime] = {}
- for item in retention:
- team_cutoffs[item.team_id] = whole_month_cutoff(now, item.months)
- context.log.info(
- f"org {item.organization_id}, team {item.team_id}, {item.months} months: "
- f"months before {team_cutoffs[item.team_id]:%Y-%m} are due"
- )
+ for team_id, months in period_months.items():
+ team_cutoffs[team_id] = whole_month_cutoff(now, months)
+ context.log.info(f"team {team_id}, {months} months: months before {team_cutoffs[team_id]:%Y-%m} are due")
partition_filter = "AND toYYYYMM(timestamp) IN %(partitions)s" if config.partitions else ""
parameters = {
@@ -1613,30 +1730,16 @@ def plan_old_events_cleanup(
results = placement.cluster.map_one_host_per_shard(Query(query, parameters=parameters)).result()
team_months.update((team_id, partition) for rows in results.values() for team_id, partition in rows)
- teams_by_partition: dict[int, list[int]] = defaultdict(list)
- for team_id, partition in sorted(team_months):
- if partition < _yyyymm(team_cutoffs[team_id]):
- teams_by_partition[partition].append(team_id)
-
- plan = OldEventsCleanupPlan(
- partitions=[
- PartitionCleanup(partition=partition, team_ids=team_ids)
- for partition, team_ids in sorted(teams_by_partition.items(), reverse=True)
- ]
- )
- for partition_cleanup in plan.partitions:
- context.log.info(f"{partition_cleanup.partition}: teams {partition_cleanup.team_ids}")
-
- context.add_output_metadata(
- {
- "teams": dagster.MetadataValue.int(len(team_cutoffs)),
- "partitions": dagster.MetadataValue.int(len(plan.partitions)),
- }
+ return OldEventsCleanupPlan(
+ partitions=_planned_partitions(
+ context, team_months, {team_id: _yyyymm(cutoff) for team_id, cutoff in team_cutoffs.items()}
+ ),
+ period_months_by_team=period_months,
+ completed_pass_team_ids=completed_pass_team_ids,
)
- return plan
[email protected](out=dagster.Out(dagster.Nothing))
def cleanup_old_events_by_partition(
context: dagster.OpExecutionContext,
cluster: dagster.ResourceParam[ClickhouseCluster],
@@ -1691,14 +1794,7 @@ def cleanup_old_events_by_partition(
context.log.info(f"Completed deletion for partition {item.partition}")
- context.add_output_metadata(
- {
- "partitions_processed": dagster.MetadataValue.int(total_partitions),
- "team_ids": dagster.MetadataValue.text(
- ", ".join(str(tid) for tid in sorted({tid for item in plan.partitions for tid in item.team_ids}))
- ),
- }
- )
+ context.add_output_metadata(_deleted_partitions_metadata(plan.partitions))
@dagster.op(ins={"start_after": dagster.In(dagster.Nothing)})
@@ -1706,19 +1802,22 @@ def plan_old_sessions_cleanup(
context: dagster.OpExecutionContext,
config: MonthlyCleanupConfig,
cluster: dagster.ResourceParam[ClickhouseCluster],
+ events_plan: OldEventsCleanupPlan,
) -> OldSessionsCleanupPlan:
- """Find every team with an events retention, and for each month the teams whose sessions in it are all past it."""
- retention = effective_events_retention()
+ """For each month, find the teams whose sessions in it are all past the period their events were cleaned up by."""
+ period_months = events_plan.period_months_by_team
+ completed_pass_team_ids = [] if config.partitions else events_plan.completed_pass_team_ids
if config.team_ids:
- if ignored := sorted(set(config.team_ids) - {item.team_id for item in retention}):
- context.log.info(f"Ignoring teams with no events retention: {ignored}")
- retention = [item for item in retention if item.team_id in config.team_ids]
- if not retention:
+ if ignored := sorted(set(config.team_ids) - set(period_months)):
+ context.log.info(f"Ignoring teams whose events this run did not clean up: {ignored}")
+ period_months = {team_id: months for team_id, months in period_months.items() if team_id in config.team_ids}
+ completed_pass_team_ids = [team_id for team_id in completed_pass_team_ids if team_id in config.team_ids]
+ if not period_months:
context.log.info("No team has an events retention to enforce")
return OldSessionsCleanupPlan(partitions=[])
now = datetime.now(UTC).replace(tzinfo=None)
- team_cutoffs = {item.team_id: _yyyymm(whole_month_cutoff(now, item.months)) for item in retention}
+ team_cutoffs = {team_id: _yyyymm(whole_month_cutoff(now, months)) for team_id, months in period_months.items()}
# Each sessions version partitions on a different expression of the session start, so the
# month comes from the partition id.
@@ -1740,30 +1839,13 @@ def plan_old_sessions_cleanup(
results = sessions.cluster.map_one_host_per_shard(Query(query, parameters=parameters)).result()
team_months.update((team_id, partition) for rows in results.values() for team_id, partition in rows)
- teams_by_partition: dict[int, list[int]] = defaultdict(list)
- for team_id, partition in sorted(team_months):
- if partition < team_cutoffs[team_id]:
- teams_by_partition[partition].append(team_id)
-
- plan = OldSessionsCleanupPlan(
- partitions=[
- PartitionCleanup(partition=partition, team_ids=team_ids)
- for partition, team_ids in sorted(teams_by_partition.items(), reverse=True)
- ]
- )
- for partition_cleanup in plan.partitions:
- context.log.info(f"{partition_cleanup.partition}: teams {partition_cleanup.team_ids}")
-
- context.add_output_metadata(
- {
- "teams": dagster.MetadataValue.int(len(team_cutoffs)),
- "partitions": dagster.MetadataValue.int(len(plan.partitions)),
- }
+ return OldSessionsCleanupPlan(
+ partitions=_planned_partitions(context, team_months, team_cutoffs),
+ completed_pass_team_ids=completed_pass_team_ids,
)
- return plan
[email protected](out=dagster.Out(dagster.Nothing))
def cleanup_old_sessions_by_partition(
context: dagster.OpExecutionContext,
cluster: dagster.ResourceParam[ClickhouseCluster],
@@ -1792,14 +1874,23 @@ def cleanup_old_sessions_by_partition(
_run_on_every_shard(sessions.cluster, delete_mutation_runner)
context.log.info(f"Completed sessions deletion for partition {item.partition}")
- context.add_output_metadata(
- {
- "partitions_processed": dagster.MetadataValue.int(total_partitions),
- "team_ids": dagster.MetadataValue.text(
- ", ".join(str(tid) for tid in sorted({tid for item in plan.partitions for tid in item.team_ids}))
- ),
- }
- )
+ context.add_output_metadata(_deleted_partitions_metadata(plan.partitions))
+
+
[email protected](ins={"start_after": dagster.In(dagster.Nothing)})
+def advance_plan_deletion_passes(
+ context: dagster.OpExecutionContext, passes: PlanDeletionPasses, plan: OldSessionsCleanupPlan
+) -> None:
+ """Move the next pass of each team whose pass this run completed three months out, once its sessions are deleted."""
+ advanced = 0
+ for chunk in chunked(plan.completed_pass_team_ids, _TEAM_CHUNK_SIZE):
+ with transaction.atomic():
+ rows = list(TeamEventsPlanDeletion.objects.select_for_update().filter(team_id__in=chunk))
+ for row in rows:
+ row.next_deletion_on = after_pass(row.state, passes.run_on).next_deletion_on
+ TeamEventsPlanDeletion.objects.bulk_update(rows, ["next_deletion_on"])
+ advanced += len(rows)
+ context.add_output_metadata({"passes": dagster.MetadataValue.int(advanced)})
@dagster.job(
@@ -1809,22 +1900,20 @@ def cleanup_old_sessions_by_partition(
}
)
def monthly_retention_events_cleanup_job():
- """Delete events and sessions past each team's events retention set in Django admin, a whole month at a time."""
- events_deleted = cleanup_old_events_by_partition(
- plan_old_events_cleanup(start_after=ensure_no_concurrent_deletes_run())
- )
- cleanup_old_sessions_by_partition(plan_old_sessions_cleanup(start_after=events_deleted))
+ """Delete each team's events and sessions past its period, monthly where set in settings, else on each pass."""
+ passes = update_plan_deletions(start_after=ensure_no_concurrent_deletes_run())
+ events_plan = plan_old_events_cleanup(passes)
+ events_deleted = cleanup_old_events_by_partition(events_plan)
+ sessions_plan = plan_old_sessions_cleanup(events_plan, start_after=events_deleted)
+ sessions_deleted = cleanup_old_sessions_by_partition(sessions_plan)
+ advance_plan_deletion_passes(passes, sessions_plan, start_after=sessions_deleted)
@dagster.schedule(
job=monthly_retention_events_cleanup_job,
- cron_schedule="0 21 3 * *",
- execution_timezone="Europe/Paris" if settings.CLOUD_DEPLOYMENT == "EU" else "America/Los_Angeles",
+ cron_schedule=f"0 {CLEANUP_HOUR} {CLEANUP_DAY_OF_MONTH} * *",
+ execution_timezone=cleanup_timezone(),
default_status=dagster.DefaultScheduleStatus.STOPPED,
)
-def monthly_retention_events_cleanup_schedule(
- context: dagster.ScheduleEvaluationContext,
-) -> dagster.RunRequest | dagster.SkipReason:
- if not effective_events_retention():
- return dagster.SkipReason("No team or organization has an events retention set")
+def monthly_retention_events_cleanup_schedule(context: dagster.ScheduleEvaluationContext) -> dagster.RunRequest:
return dagster.RunRequest(run_key=f"{context.scheduled_execution_time:%Y%m}")
diff --git a/posthog/dags/tests/test_deletes.py b/posthog/dags/tests/test_deletes.py
index 6a8caa1f6aee..04ef7b8d87e5 100644
--- a/posthog/dags/tests/test_deletes.py
+++ b/posthog/dags/tests/test_deletes.py
@@ -1,4 +1,5 @@
-from datetime import UTC, datetime, timedelta
+from collections.abc import Callable
+from datetime import UTC, date, datetime, timedelta
from functools import partial
from typing import cast
from uuid import UUID
@@ -8,6 +9,7 @@ import time_machine
from unittest.mock import Mock, patch
from django.conf import settings as django_settings
+from django.test import override_settings
import dagster
from clickhouse_driver import Client
@@ -29,6 +31,8 @@ from posthog.dags.deletes import (
PartitionCleanup,
PendingDeletesDictionary,
PendingDeletesTable,
+ PlanDeletionConfig,
+ PlanDeletionPasses,
StagedDictionary,
_count_through,
_count_unswept_rows,
@@ -46,6 +50,7 @@ from posthog.dags.deletes import (
plan_old_events_cleanup,
resolve_sweep_targets,
run_deletes_after_manual_trigger,
+ update_plan_deletions,
wait_for_delete_mutations_in_shards,
)
from posthog.dags.person_overrides import squash_person_overrides
@@ -53,10 +58,12 @@ from posthog.dags.tests.conftest import insert_flag_evaluations
from posthog.models.async_deletion import AsyncDeletion, DeletionRunLog, DeletionRunStatus, DeletionType
from posthog.models.deletion_targets import DEFAULT_DELETION_TARGETS, EVENTS, PERSONAL_DATA_TARGETS, TargetPlacement
from posthog.models.event.sql import EVENTS_DATA_TABLE, EVENTS_JSON_DATA_TABLE
-from posthog.models.events_retention_config import OrganizationEventsRetentionConfig, TeamEventsRetentionConfig
+from posthog.models.events_plan_deletion import TeamEventsPlanDeletion, cleanup_date, next_pass_on
+from posthog.models.events_retention_config import TeamEventsRetentionConfig
from posthog.models.organization import Organization
from posthog.models.person.sql import PERSON_DISTINCT_ID_OVERRIDES_TABLE
from posthog.models.team import Team
+from posthog.models.team.event_retention import EVENTS_DATA_RETENTION_FLAG
def surviving_flag_evaluations(client: Client) -> set[tuple[int, UUID, UUID]]:
@@ -759,25 +766,29 @@ def test_plan_old_events_cleanup_lists_the_due_teams_per_month(cluster: Clickhou
)
context = build_op_context()
- plan = plan_old_events_cleanup(context, MonthlyCleanupConfig(), cluster)
+ no_passes = PlanDeletionPasses(run_on=cleanup_date(), due_team_ids=[])
+
+ plan = plan_old_events_cleanup(context, MonthlyCleanupConfig(), cluster, no_passes)
assert plan == OldEventsCleanupPlan(
partitions=[
PartitionCleanup(partition=_partition(_mid_month(15)), team_ids=[thirteen.id]),
PartitionCleanup(partition=_partition(_mid_month(20)), team_ids=[thirteen.id, eighteen.id]),
- ]
+ ],
+ period_months_by_team={thirteen.id: 13, eighteen.id: 18},
)
only_oldest = plan_old_events_cleanup(
- context, MonthlyCleanupConfig(partitions=[_partition(_mid_month(20))]), cluster
+ context, MonthlyCleanupConfig(partitions=[_partition(_mid_month(20))]), cluster, no_passes
)
assert [item.partition for item in only_oldest.partitions] == [_partition(_mid_month(20))]
only_eighteen = plan_old_events_cleanup(
- context, MonthlyCleanupConfig(team_ids=[eighteen.id, no_retention.id]), cluster
+ context, MonthlyCleanupConfig(team_ids=[eighteen.id, no_retention.id]), cluster, no_passes
)
assert only_eighteen == OldEventsCleanupPlan(
- partitions=[PartitionCleanup(partition=_partition(_mid_month(20)), team_ids=[eighteen.id])]
+ partitions=[PartitionCleanup(partition=_partition(_mid_month(20)), team_ids=[eighteen.id])],
+ period_months_by_team={eighteen.id: 18},
)
@@ -850,19 +861,198 @@ def test_monthly_retention_events_cleanup_job(cluster: ClickhouseCluster):
)
-@time_machine.travel("2027-01-01", tick=False)
-def test_monthly_retention_events_cleanup_schedule_starts_one_run_when_a_retention_is_set():
+def test_monthly_retention_events_cleanup_schedule_starts_one_run_a_month():
with dagster.build_schedule_context(scheduled_execution_time=datetime(2027, 1, 1, tzinfo=UTC)) as context:
- assert isinstance(monthly_retention_events_cleanup_schedule(context), dagster.SkipReason)
-
- organization = Organization.objects.create(name="With default")
- OrganizationEventsRetentionConfig.objects.create(organization=organization, default_events_retention_months=13)
- Team.objects.create(organization=organization, name="Inherits default")
run_request = monthly_retention_events_cleanup_schedule(context)
assert isinstance(run_request, dagster.RunRequest)
- assert (run_request.run_key, run_request.run_config) == ("202701", {})
+ assert run_request.run_key == "202701"
+
+
+def _first_pass() -> date:
+ return next_pass_on(cleanup_date())
+
+
+def _plan_state(team: Team) -> tuple[int, date] | None:
+ row = TeamEventsPlanDeletion.objects.filter(team=team).first()
+ return (row.months, row.next_deletion_on) if row else None
+
+
+def test_update_plan_deletions_passes_only_teams_deleting_past_the_default():
+ organization = Organization.objects.create(name="Mixed plans")
+ free, paid, unenforced = (
+ Team.objects.create(organization=organization, name=name, event_retention_months=months)
+ for name, months in (("Free", 12), ("Paid", 84), ("Unenforced", 12))
+ )
+ TeamEventsPlanDeletion.objects.create(team=paid, months=24, next_deletion_on=cleanup_date())
+
+ def feature_enabled(
+ flag: str, distinct_id: str, *, person_properties: dict | None = None, **kwargs: object
+ ) -> bool:
+ if flag == EVENTS_DATA_RETENTION_FLAG:
+ return distinct_id != str(unenforced.id)
+ return person_properties == {"plan_retention_months": 12, "organization_id": str(organization.id)}
+
+ with (
+ patch("posthog.models.team.event_retention.is_cloud", return_value=True),
+ patch("posthoganalytics.feature_enabled", side_effect=feature_enabled),
+ ):
+ passes = update_plan_deletions(build_op_context(), PlanDeletionConfig())
+
+ assert passes.due_team_ids == []
+ assert [_plan_state(team) for team in (free, paid, unenforced)] == [(12, _first_pass()), None, None]
+
+
+def _both_plans(config: dict) -> dict:
+ return {"plan_old_events_cleanup": {"config": config}, "plan_old_sessions_cleanup": {"config": config}}
+
+
+ "cleanup_ops,months_left,sessions_left,pass_completed",
+ [
+ pytest.param(lambda other: {}, {11: 4}, {11}, True, id="every_team"),
+ pytest.param(
+ lambda other: _both_plans({"team_ids": [other.id]}),
+ {20: 2, 13: 3, 11: 4},
+ {20, 13, 11},
+ False,
+ id="another_team",
+ ),
+ pytest.param(
+ lambda other: _both_plans({"partitions": [_partition(_mid_month(13))]}),
+ {20: 2, 11: 4},
+ {20, 11},
+ False,
+ id="one_month",
+ ),
+ pytest.param(
+ lambda other: {"plan_old_sessions_cleanup": {"config": {"team_ids": [other.id]}}},
+ {11: 4},
+ {20, 13, 11},
+ False,
+ id="sessions_of_another_team",
+ ),
+ ],
+)
+@override_settings(EVENTS_DATA_RETENTION_ENFORCED=True)
+def test_pass_deletes_events_and_sessions_past_the_period_and_moves_only_a_completed_pass(
+ cluster: ClickhouseCluster,
+ cleanup_ops: Callable[[Team], dict],
+ months_left: dict[int, int],
+ sessions_left: set[int],
+ pass_completed: bool,
+):
+ organization = Organization.objects.create(name="Due")
+ team = Team.objects.create(organization=organization, name="Due", event_retention_months=12)
+ other = Team.objects.create(organization=organization, name="Other", event_retention_months=12)
+ TeamEventsPlanDeletion.objects.create(team=team, months=12, next_deletion_on=cleanup_date())
+ _insert_events(
+ cluster,
+ _events(team.id, _mid_month(20), 0, 2)
+ + _events(team.id, _mid_month(13), 100, 3)
+ + _events(team.id, _mid_month(11), 200, 4),
+ )
+ _insert_sessions(cluster, [(team.id, _mid_month(months)) for months in (20, 13, 11)])
+
+ monthly_retention_events_cleanup_job.execute_in_process(
+ run_config={"ops": cleanup_ops(other)}, resources={"cluster": cluster}
+ )
+
+ assert _count_by_team_and_partition(cluster, [team.id]) == {
+ (team.id, _partition(_mid_month(months))): events for months, events in months_left.items()
+ }
+ assert _sessions_by_team_and_partition(cluster, [team.id]) == dict.fromkeys(
+ SESSIONS_RETENTION_TABLES, {(team.id, _partition(_mid_month(months))) for months in sessions_left}
+ )
+ assert _plan_state(team) == (12, _first_pass() if pass_completed else cleanup_date())
+
+
+ "plan_months,months,postponed,deleted_months",
+ [
+ pytest.param(12, 12, False, [13, 20], id="unchanged"),
+ pytest.param(12, 24, False, [], id="longer_period"),
+ pytest.param(12, 6, True, [], id="shorter_period"),
+ pytest.param(24, 12, False, [], id="plan_raised"),
+ ],
+)
+@override_settings(EVENTS_DATA_RETENTION_ENFORCED=True)
+def test_plan_old_events_cleanup_uses_a_pass_changed_after_the_run_started(
+ cluster: ClickhouseCluster, plan_months: int, months: int, postponed: bool, deleted_months: list[int]
+):
+ team = Team.objects.create(
+ organization=Organization.objects.create(name="Due"), name="Due", event_retention_months=plan_months
+ )
+ _insert_events(cluster, _events(team.id, _mid_month(13), 0, 1) + _events(team.id, _mid_month(20), 100, 1))
+ passes = PlanDeletionPasses(run_on=cleanup_date(), due_team_ids=[team.id])
+ TeamEventsPlanDeletion.objects.create(
+ team=team, months=months, next_deletion_on=_first_pass() if postponed else cleanup_date()
+ )
+
+ plan = plan_old_events_cleanup(build_op_context(), MonthlyCleanupConfig(), cluster, passes)
+
+ assert [item.partition for item in plan.partitions] == [_partition(_mid_month(m)) for m in deleted_months]
+ assert plan.completed_pass_team_ids == ([] if postponed else [team.id])
+
+
+ "deployment,now,next_pass",
+ [
+ ("US", "2026-11-04T04:00:00Z", date(2027, 2, 3)),
+ ("EU", "2026-11-03T23:30:00Z", date(2027, 3, 3)),
+ ],
+)
+@override_settings(EVENTS_DATA_RETENTION_ENFORCED=True)
+def test_update_plan_deletions_shorter_period_pushes_the_next_pass(deployment: str, now: str, next_pass: date):
+ downgraded = Team.objects.create(
+ organization=Organization.objects.create(name="Downgraded"), name="Downgraded", event_retention_months=12
+ )
+ standard = Team.objects.create(
+ organization=Organization.objects.create(name="Standard"), name="Standard", event_retention_months=84
+ )
+ Organization.objects.filter(id=standard.organization_id).update(
+ available_product_features=[{"key": "product_analytics_standard_data_retention"}]
+ )
+ TeamEventsPlanDeletion.objects.create(team=downgraded, months=24, next_deletion_on=date(2026, 11, 10))
+ TeamEventsPlanDeletion.objects.create(team=standard, months=84, next_deletion_on=date(2026, 11, 10))
+
+ with override_settings(CLOUD_DEPLOYMENT=deployment), time_machine.travel(now, tick=False):
+ passes = update_plan_deletions(build_op_context(), PlanDeletionConfig())
+
+ assert passes.due_team_ids == []
+ assert (_plan_state(downgraded), _plan_state(standard)) == ((12, next_pass), (24, next_pass))
+
+
+@override_settings(EVENTS_DATA_RETENTION_ENFORCED=True)
+def test_a_downgrade_below_a_set_period_deletes_past_the_set_period_until_the_pass(cluster: ClickhouseCluster):
+ team = Team.objects.create(
+ organization=Organization.objects.create(name="Downgraded"), name="Downgraded", event_retention_months=24
+ )
+ TeamEventsRetentionConfig.objects.create(team=team, events_retention_months=36)
+ TeamEventsPlanDeletion.objects.create(team=team, months=36, next_deletion_on=cleanup_date())
+ _insert_events(cluster, _events(team.id, _mid_month(30), 0, 1) + _events(team.id, _mid_month(40), 100, 1))
+ past_the_set_period, past_the_plan = _partition(_mid_month(40)), _partition(_mid_month(30))
+
+ passes = update_plan_deletions(build_op_context(), PlanDeletionConfig())
+ plan = plan_old_events_cleanup(build_op_context(), MonthlyCleanupConfig(), cluster, passes)
+
+ first_pass = _first_pass()
+ assert _plan_state(team) == (24, first_pass)
+ assert [item.partition for item in plan.partitions] == [past_the_set_period]
+
+ with time_machine.travel(
+ datetime.combine(first_pass, datetime.min.time(), tzinfo=UTC) + timedelta(hours=20), tick=False
+ ):
+ passes = update_plan_deletions(build_op_context(), PlanDeletionConfig())
+ plan = plan_old_events_cleanup(build_op_context(), MonthlyCleanupConfig(), cluster, passes)
+
+ assert passes.due_team_ids == [team.id]
+ assert [item.partition for item in plan.partitions] == [past_the_plan, past_the_set_period]
def _insert_pending_deletes(table: PendingDeletesTable, client: Client, count: int = 5, first_id: int = 0) -> None:
diff --git a/posthog/migrations/1426_events_plan_deletion.py b/posthog/migrations/1426_events_plan_deletion.py
new file mode 100644
index 000000000000..8d262cf06bb8
--- /dev/null
+++ b/posthog/migrations/1426_events_plan_deletion.py
@@ -0,0 +1,67 @@
+# Generated by Django 5.2.18 on 2026-10-29 20:52
+
+import django.core.validators
+import django.db.models.deletion
+from django.db import migrations, models
+
+RETENTION_MONTHS_HELP = (
+ "Events are deleted one calendar month at a time. A month is deleted only when every event in it is "
+ "older than this many months. For example, with 13 months, the events of March 2025 are deleted on "
+ "the first cleanup after 1 May 2026."
+)
+
+
+class Migration(migrations.Migration):
+ dependencies = [("posthog", "1425_workload_trust_policy_request")]
+
+ operations = [
+ migrations.CreateModel(
+ name="TeamEventsPlanDeletion",
+ fields=[
+ (
+ "team",
+ models.OneToOneField(
+ db_constraint=False,
+ on_delete=django.db.models.deletion.CASCADE,
+ primary_key=True,
+ serialize=False,
+ to="posthog.team",
+ ),
+ ),
+ (
+ "months",
+ models.PositiveSmallIntegerField(
+ help_text="The period the team's passes delete past, as of the last cleanup or settings change."
+ ),
+ ),
+ (
+ "next_deletion_on",
+ models.DateField(
+ help_text="The cleanup on or after this date deletes the team's events older than the period."
+ ),
+ ),
+ ],
+ ),
+ migrations.AlterField(
+ model_name="organizationeventsretentionconfig",
+ name="default_events_retention_months",
+ field=models.PositiveSmallIntegerField(
+ blank=True,
+ help_text="Applies to every team in the organization that has no value of its own. Leave empty to use "
+ f"the plan's default period. {RETENTION_MONTHS_HELP}",
+ null=True,
+ validators=[django.core.validators.MinValueValidator(1)],
+ ),
+ ),
+ migrations.AlterField(
+ model_name="teameventsretentionconfig",
+ name="events_retention_months",
+ field=models.PositiveSmallIntegerField(
+ blank=True,
+ help_text="Overrides the organization's default and has to fall within the organization's range. "
+ f"Leave empty to use the organization's default. {RETENTION_MONTHS_HELP}",
+ null=True,
+ validators=[django.core.validators.MinValueValidator(1)],
+ ),
+ ),
+ ]
diff --git a/posthog/migrations/max_migration.txt b/posthog/migrations/max_migration.txt
index ac5fc3c07881..1e279af683b2 100644
--- a/posthog/migrations/max_migration.txt
+++ b/posthog/migrations/max_migration.txt
@@ -1 +1 @@
-1425_workload_trust_policy_request
+1426_events_plan_deletion
diff --git a/posthog/models/__init__.py b/posthog/models/__init__.py
index 3cb7bdf89975..88e878df0282 100644
--- a/posthog/models/__init__.py
+++ b/posthog/models/__init__.py
@@ -18,6 +18,7 @@ from .data_deletion_request import DataDeletionRequest
from .data_color_theme import DataColorTheme
from .element import Element
from .element_group import ElementGroup
+from .events_plan_deletion import TeamEventsPlanDeletion
from .events_retention_config import OrganizationEventsRetentionConfig, TeamEventsRetentionConfig
from .event.event import Event
from .event_buffer import EventBuffer
@@ -176,6 +177,7 @@ __all__ = [
"Tag",
"TaggedItem",
"Team",
+ "TeamEventsPlanDeletion",
"TeamEventsRetentionConfig",
"TeamRevenueAnalyticsConfig",
"TeamMarketingAnalyticsConfig",
diff --git a/posthog/models/events_plan_deletion.py b/posthog/models/events_plan_deletion.py
new file mode 100644
index 000000000000..bcab014cd4e6
--- /dev/null
+++ b/posthog/models/events_plan_deletion.py
@@ -0,0 +1,120 @@
+from collections.abc import Sequence
+from dataclasses import replace
+from datetime import date, datetime, timedelta
+from zoneinfo import ZoneInfo
+
+from django.conf import settings
+from django.db import models
+
+from dateutil.relativedelta import relativedelta
+
+from posthog.dataclasses import frozen
+from posthog.models.events_retention_config import TeamEventsRetention
+from posthog.models.team import Team
+from posthog.models.team.event_retention import TeamPlan, retention_periods
+
+PASS_INTERVAL_MONTHS = 3
+CLEANUP_DAY_OF_MONTH = 3
+CLEANUP_HOUR = 21
+_WRITE_BATCH_SIZE = 1000
+
+
+class TeamEventsPlanDeletion(models.Model):
+ team = models.OneToOneField(Team, on_delete=models.CASCADE, primary_key=True, db_constraint=False)
+ months = models.PositiveSmallIntegerField(
+ help_text="The period the team's passes delete past, as of the last cleanup or settings change."
+ )
+ next_deletion_on = models.DateField(
+ help_text="The cleanup on or after this date deletes the team's events older than the period."
+ )
+
+ @property
+ def state(self) -> "PlanDeletionState":
+ return PlanDeletionState(months=self.months, next_deletion_on=self.next_deletion_on)
+
+
+@frozen
+class PlanDeletionState:
+ months: int
+ next_deletion_on: date
+
+
+def cleanup_timezone() -> str:
+ return "Europe/Paris" if settings.CLOUD_DEPLOYMENT == "EU" else "America/Los_Angeles"
+
+
+def cleanup_date() -> date:
+ return datetime.now(ZoneInfo(cleanup_timezone())).date()
+
+
+def cleanup_on_or_after(day: date) -> date:
+ run_on = day.replace(day=CLEANUP_DAY_OF_MONTH)
+ return run_on if run_on >= day else run_on + relativedelta(months=1)
+
+
+def next_cleanup_on() -> date:
+ now = datetime.now(ZoneInfo(cleanup_timezone()))
+ return cleanup_on_or_after(now.date() + timedelta(days=1) if now.hour >= CLEANUP_HOUR else now.date())
+
+
+def next_pass_on(today: date) -> date:
+ return cleanup_on_or_after(today + relativedelta(months=PASS_INTERVAL_MONTHS))
+
+
+def admitted(period_months: int, today: date) -> PlanDeletionState:
+ return PlanDeletionState(months=period_months, next_deletion_on=next_pass_on(today))
+
+
+def is_shorter_period(state: PlanDeletionState, period_months: int) -> bool:
+ return period_months < state.months
+
+
+def with_period(state: PlanDeletionState, period_months: int, today: date) -> PlanDeletionState:
+ next_deletion_on = (
+ max(state.next_deletion_on, next_pass_on(today))
+ if is_shorter_period(state, period_months)
+ else state.next_deletion_on
+ )
+ return PlanDeletionState(months=period_months, next_deletion_on=next_deletion_on)
+
+
+def after_pass(state: PlanDeletionState, today: date) -> PlanDeletionState:
+ return replace(state, next_deletion_on=max(state.next_deletion_on, next_pass_on(today)))
+
+
+def deletion_months(setting: TeamEventsRetention | None, due_pass_months: int | None) -> int | None:
+ periods = [months for months in (setting.months if setting else None, due_pass_months) if months is not None]
+ return min(periods) if periods else None
+
+
+def reschedule_plan_deletions(plans: Sequence[TeamPlan], today: date) -> dict[int, PlanDeletionState]:
+ team_ids = [plan.team_id for plan in plans]
+ rows = {
+ row.team_id: row.state
+ for row in TeamEventsPlanDeletion.objects.select_for_update().filter(team_id__in=team_ids)
+ }
+ periods = retention_periods(plans)
+ states: dict[int, PlanDeletionState] = {}
+ for team_id in team_ids:
+ state = rows.get(team_id)
+ states[team_id] = (
+ with_period(state, periods[team_id], today) if state is not None else admitted(periods[team_id], today)
+ )
+
+ TeamEventsPlanDeletion.objects.bulk_create(
+ [_row(team_id, state) for team_id, state in states.items() if team_id in rows and state != rows[team_id]],
+ update_conflicts=True,
+ unique_fields=["team"],
+ update_fields=["months", "next_deletion_on"],
+ batch_size=_WRITE_BATCH_SIZE,
+ )
+ TeamEventsPlanDeletion.objects.bulk_create(
+ [_row(team_id, state) for team_id, state in states.items() if team_id not in rows],
+ ignore_conflicts=True,
+ batch_size=_WRITE_BATCH_SIZE,
+ )
+ return states
+
+
+def _row(team_id: int, state: PlanDeletionState) -> TeamEventsPlanDeletion:
+ return TeamEventsPlanDeletion(team_id=team_id, months=state.months, next_deletion_on=state.next_deletion_on)
diff --git a/posthog/models/events_retention_config.py b/posthog/models/events_retention_config.py
index 3b34ee1c6cc6..4b3c8bd1e4b2 100644
--- a/posthog/models/events_retention_config.py
+++ b/posthog/models/events_retention_config.py
@@ -15,7 +15,7 @@ from posthog.models.team import Team
RETENTION_MONTHS_HELP = (
"Events are deleted one calendar month at a time. A month is deleted only when every event in it is "
"older than this many months. For example, with 13 months, the events of March 2025 are deleted on "
- "the first monthly run after 1 May 2026."
+ "the first cleanup after 1 May 2026."
)
@@ -34,8 +34,8 @@ class OrganizationEventsRetentionConfig(models.Model):
"posthog.Organization", on_delete=models.CASCADE, primary_key=True, db_constraint=False, related_name="+"
)
default_events_retention_months = _months_field(
- "Applies to every team in the organization that has no value of its own. Leave empty to keep the events "
- f"of those teams. {RETENTION_MONTHS_HELP}"
+ "Applies to every team in the organization that has no value of its own. Leave empty to use the plan's "
+ f"default period. {RETENTION_MONTHS_HELP}"
)
min_events_retention_months = _months_field("The shortest retention a team in this organization may set.")
max_events_retention_months = _months_field("The longest retention a team in this organization may set.")
@@ -115,8 +115,7 @@ class TeamEventsRetentionConfig(models.Model):
team = models.OneToOneField(Team, on_delete=models.CASCADE, primary_key=True, db_constraint=False)
events_retention_months = _months_field(
"Overrides the organization's default and has to fall within the organization's range. Leave empty to use "
- "the organization's default. Events are kept only when neither this nor the organization's default is set. "
- f"{RETENTION_MONTHS_HELP}"
+ f"the organization's default. {RETENTION_MONTHS_HELP}"
)
class Meta:
diff --git a/posthog/models/team/event_retention.py b/posthog/models/team/event_retention.py
index f69a73ca741d..4be65ee6b3ec 100644
--- a/posthog/models/team/event_retention.py
+++ b/posthog/models/team/event_retention.py
@@ -23,6 +23,8 @@ DEFAULT_EVENT_RETENTION_MONTHS = 84
# Cohort feature flag — gates enforcement so the rollout can target a specific cohort.
EVENTS_DATA_RETENTION_FLAG = "events-data-retention"
+EVENTS_DATA_DELETION_FLAG = "events-data-retention-deletion"
+
# Billing entitlement key the sync job reconciles Team.event_retention_months against.
EVENTS_DATA_RETENTION_FEATURE = AvailableFeature.PRODUCT_ANALYTICS_DATA_RETENTION
@@ -104,6 +106,33 @@ def team_plan(team: Team) -> TeamPlan:
)
+def should_delete_past_default_period(team: Team) -> bool:
+ """Whether the cleanup deletes this team's events past its default period.
+
+ The settings override forces it the same way as enforcement. Otherwise the team needs enforcement, so the cleanup
+ never deletes events the team can still query, and the deletion flag. The flag gets the plan period and the
+ organization, so a rollout can target a plan such as the free plan's 12 months and leave out organizations.
+ """
+ if settings.EVENTS_DATA_RETENTION_ENFORCED is not None:
+ return settings.EVENTS_DATA_RETENTION_ENFORCED
+
+ if not should_enforce_events_retention(team.id):
+ return False
+
+ return bool(
+ posthoganalytics.feature_enabled(
+ EVENTS_DATA_DELETION_FLAG,
+ str(team.id),
+ person_properties={
+ "plan_retention_months": team_plan(team).plan_months,
+ "organization_id": str(team.organization_id),
+ },
+ only_evaluate_locally=True,
+ send_feature_flag_events=False,
+ )
+ )
+
+
def default_period(plan_months: int, standard: bool, min_months: int | None) -> int:
default = min(plan_months, STANDARD_DEFAULT_PERIOD_MONTHS) if standard else plan_months
return min(plan_months, max(default, min_months)) if min_months is not None else default
diff --git a/posthog/models/test/test_events_plan_deletion.py b/posthog/models/test/test_events_plan_deletion.py
new file mode 100644
index 000000000000..1cc67cea9214
--- /dev/null
+++ b/posthog/models/test/test_events_plan_deletion.py
@@ -0,0 +1,88 @@
+from datetime import date
+from uuid import UUID
+
+import time_machine
+
+from django.test import SimpleTestCase, override_settings
+
+from parameterized import parameterized
+
+from posthog.models.events_plan_deletion import (
+ PlanDeletionState,
+ admitted,
+ after_pass,
+ deletion_months,
+ is_shorter_period,
+ next_cleanup_on,
+ with_period,
+)
+from posthog.models.events_retention_config import TeamEventsRetention
+
+TODAY = date(2026, 11, 3)
+
+
+class TestPlanDeletionRules(SimpleTestCase):
+ @parameterized.expand(
+ [
+ ("on_a_cleanup_day", date(2026, 11, 3), date(2027, 2, 3)),
+ ("between_cleanup_days", date(2026, 11, 15), date(2027, 3, 3)),
+ ]
+ )
+ def test_admitted_lands_on_a_cleanup_day(self, _name: str, today: date, next_deletion_on: date) -> None:
+ assert admitted(12, today) == PlanDeletionState(months=12, next_deletion_on=next_deletion_on)
+
+ @parameterized.expand(
+ [
+ ("shorter_period_pushes_a_near_pass", 24, date(2026, 12, 3), 12, date(2027, 2, 3), True),
+ ("shorter_period_keeps_a_later_pass", 24, date(2027, 4, 3), 12, date(2027, 4, 3), True),
+ ("longer_period_keeps_the_pass", 12, date(2026, 12, 3), 24, date(2026, 12, 3), False),
+ ("same_period_keeps_the_pass", 12, date(2026, 12, 3), 12, date(2026, 12, 3), False),
+ ]
+ )
+ def test_with_period(
+ self, _name: str, months: int, next_on: date, period: int, expected_next_on: date, shorter: bool
+ ) -> None:
+ state = PlanDeletionState(months=months, next_deletion_on=next_on)
+
+ assert is_shorter_period(state, period) is shorter
+ assert with_period(state, period, TODAY) == PlanDeletionState(months=period, next_deletion_on=expected_next_on)
+
+ @parameterized.expand(
+ [
+ ("due_pass", TODAY, date(2027, 2, 3)),
+ ("pass_moved_later_during_the_run", date(2027, 3, 3), date(2027, 3, 3)),
+ ]
+ )
+ def test_after_pass(self, _name: str, next_on: date, expected_next_on: date) -> None:
+ state = PlanDeletionState(months=12, next_deletion_on=next_on)
+
+ assert after_pass(state, TODAY) == PlanDeletionState(months=12, next_deletion_on=expected_next_on)
+
+ @parameterized.expand(
+ [
+ ("setting", 6, None, 6),
+ ("pass_period", None, 6, 6),
+ ("setting_shorter_than_the_pass", 6, 12, 6),
+ ("pass_shorter_than_the_setting", 24, 12, 12),
+ ("nothing_set", None, None, None),
+ ]
+ )
+ def test_deletion_months(
+ self, _name: str, setting: int | None, pass_months: int | None, expected: int | None
+ ) -> None:
+ retention = (
+ TeamEventsRetention(organization_id=UUID(int=0), team_id=1, months=setting) if setting is not None else None
+ )
+
+ assert deletion_months(retention, pass_months) == expected
+
+ @parameterized.expand(
+ [
+ ("before_the_run", "2026-11-04T04:59:00Z", date(2026, 11, 3)),
+ ("after_the_run", "2026-11-04T05:00:00Z", date(2026, 12, 3)),
+ ]
+ )
+ @override_settings(CLOUD_DEPLOYMENT="US")
+ def test_next_cleanup_on(self, _name: str, now: str, expected: date) -> None:
+ with time_machine.travel(now, tick=False):
+ assert next_cleanup_on() == expected
diff --git a/posthog/settings/__init__.py b/posthog/settings/__init__.py
index b9af458de233..b8c63aa8c199 100644
--- a/posthog/settings/__init__.py
+++ b/posthog/settings/__init__.py
@@ -142,7 +142,8 @@ PERSON_ON_EVENTS_V2_OVERRIDE: bool = get_from_env("PERSON_ON_EVENTS_V2_OVERRIDE"
PERSON_BULK_DELETE_ASYNC: bool = get_from_env("PERSON_BULK_DELETE_ASYNC", False, type_cast=str_to_bool)
# Events data retention enforcement override (ops kill switch / local + test toggle). When unset (None),
-# enforcement falls back to the per-project `events-data-retention` cohort flag. When set, forces it on/off everywhere.
+# enforcement falls back to the per-project `events-data-retention` cohort flag, and deletion past the default period
+# also needs the `events-data-retention-deletion` flag. When set, forces both on/off everywhere.
EVENTS_DATA_RETENTION_ENFORCED: bool | None = get_from_env(
"EVENTS_DATA_RETENTION_ENFORCED", optional=True, type_cast=str_to_bool
)
diff --git a/products/product_analytics/backend/events_retention.py b/products/product_analytics/backend/events_retention.py
index 67ceb65b2f59..aa006cc1a2c7 100644
--- a/products/product_analytics/backend/events_retention.py
+++ b/products/product_analytics/backend/events_retention.py
@@ -1,3 +1,5 @@
+from collections.abc import Iterable, Sequence
+from datetime import date, datetime, time
from functools import partial
from django.core.exceptions import ValidationError
@@ -5,23 +7,43 @@ from django.db import transaction
from posthog.event_usage import EventSource, report_user_action, sanitize_header_value
from posthog.models.activity_logging.activity_log import Change, Detail, log_activity
+from posthog.models.events_plan_deletion import (
+ TeamEventsPlanDeletion,
+ cleanup_date,
+ deletion_months,
+ next_cleanup_on,
+ next_pass_on,
+ reschedule_plan_deletions,
+ with_period,
+)
from posthog.models.events_retention_config import (
OrganizationEventsRetentionConfig,
+ TeamEventsRetention,
TeamEventsRetentionConfig,
describe_months_range,
+ effective_events_retention,
lock_organization_retention,
+ whole_month_cutoff,
)
from posthog.models.organization import Organization
from posthog.models.team import Team
from posthog.models.team.event_retention import (
STANDARD_DEFAULT_PERIOD_MONTHS,
+ TeamPlan,
default_period,
+ events_retention_periods,
has_standard_retention,
organization_events_retention_months,
+ should_delete_past_default_period,
+ team_plan,
)
from posthog.models.user import User
-from products.product_analytics.backend.facade.contracts import OrganizationEventsRetention
+from products.product_analytics.backend.facade.contracts import (
+ OrganizationEventsRetention,
+ ProjectEventsRetention,
+ ScheduledEventsDeletion,
+)
RETENTION_MONTHS_CHOICES = (1, 3, 12, 24, 36, 60, 84)
@@ -95,15 +117,43 @@ def organization_events_retention(organization: Organization) -> OrganizationEve
min_months=staff_min or 1,
max_months=min(plan_months, staff_max) if staff_max else plan_months,
months_choices=list(RETENTION_MONTHS_CHOICES),
+ next_cleanup_on=next_cleanup_on(),
)
-def project_events_retention_months(team: Team) -> int | None:
- return (
+def _next_deletion(
+ setting: TeamEventsRetention | None, period_months: int, scheduled_pass_on: date | None
+) -> ScheduledEventsDeletion | None:
+ pass_on = max(scheduled_pass_on, next_cleanup_on()) if scheduled_pass_on is not None else None
+ deletes_on = next_cleanup_on() if setting is not None else pass_on
+ if deletes_on is None:
+ return None
+ months = deletion_months(setting, period_months if pass_on is not None and pass_on <= deletes_on else None)
+ if months is None:
+ return None
+ deletes_before = whole_month_cutoff(datetime.combine(deletes_on, time()), months).date()
+ return ScheduledEventsDeletion(deletes_on=deletes_on, deletes_before=deletes_before)
+
+
+def project_events_retention(team: Team) -> ProjectEventsRetention:
+ project_months = (
TeamEventsRetentionConfig.objects.filter(team_id=team.id)
.values_list("events_retention_months", flat=True)
.first()
)
+ setting = next(iter(effective_events_retention(team_ids=[team.id])), None)
+ period_months = events_retention_periods([team])[team.id]
+ today = cleanup_date()
+ row = TeamEventsPlanDeletion.objects.filter(team_id=team.id).first()
+ scheduled_pass_on = with_period(row.state, period_months, today).next_deletion_on if row else None
+ default_period_deletes = should_delete_past_default_period(team)
+ return ProjectEventsRetention(
+ project_months=project_months,
+ next_deletion=_next_deletion(setting, period_months, scheduled_pass_on if default_period_deletes else None),
+ scheduled_pass_on=scheduled_pass_on,
+ first_pass_on=next_pass_on(today),
+ default_period_deletes=default_period_deletes,
+ )
def _check_allowed(retention: OrganizationEventsRetention, months: int | None) -> None:
@@ -121,6 +171,14 @@ def _check_allowed(retention: OrganizationEventsRetention, months: int | None) -
)
+def _deleting_plans(teams: Iterable[Team]) -> list[TeamPlan]:
+ return [team_plan(team) for team in teams if should_delete_past_default_period(team)]
+
+
+def _schedule_passes(plans: Sequence[TeamPlan]) -> None:
+ reschedule_plan_deletions(plans, cleanup_date())
+
+
def set_organization_events_retention(
organization: Organization,
months: int | None,
@@ -137,6 +195,8 @@ def set_organization_events_retention(
if months == before:
return
_check_allowed(organization_events_retention(organization), months)
+ plans = _deleting_plans(Team.objects.filter(organization_id=organization.id))
+ _schedule_passes(plans)
config = config or OrganizationEventsRetentionConfig(organization_id=organization.id)
config.default_events_retention_months = months
config.full_clean()
@@ -145,6 +205,7 @@ def set_organization_events_retention(
else:
config.save()
_report_retention_set(organization, None, user, before, months, analytics_properties, was_impersonated)
+ _schedule_passes(plans)
log_activity(
organization_id=organization.id,
@@ -172,11 +233,14 @@ def set_project_events_retention(
analytics_properties: dict[str, object] | None = None,
) -> None:
with transaction.atomic():
+ lock_organization_retention(team.organization_id)
config = TeamEventsRetentionConfig.objects.filter(team_id=team.id).first()
before = config.events_retention_months if config else None
if months == before:
return
_check_allowed(organization_events_retention(team.organization), months)
+ plans = _deleting_plans([team])
+ _schedule_passes(plans)
if months is None:
TeamEventsRetentionConfig.objects.filter(team_id=team.id).delete()
else:
@@ -185,6 +249,7 @@ def set_project_events_retention(
config.full_clean()
config.save()
_report_retention_set(team.organization, team, user, before, months, analytics_properties, was_impersonated)
+ _schedule_passes(plans)
log_activity(
organization_id=team.organization_id,
diff --git a/products/product_analytics/backend/facade/contracts.py b/products/product_analytics/backend/facade/contracts.py
index 53a575ae60c4..a5595e1c634e 100644
--- a/products/product_analytics/backend/facade/contracts.py
+++ b/products/product_analytics/backend/facade/contracts.py
@@ -201,6 +201,22 @@ class OrganizationEventsRetention:
min_months: int
max_months: int
months_choices: list[int]
+ next_cleanup_on: date
+
+
+@dataclass(frozen=True)
+class ScheduledEventsDeletion:
+ deletes_on: date
+ deletes_before: date
+
+
+@dataclass(frozen=True)
+class ProjectEventsRetention:
+ project_months: int | None
+ next_deletion: ScheduledEventsDeletion | None
+ scheduled_pass_on: date | None
+ first_pass_on: date
+ default_period_deletes: bool
@dataclass(frozen=True)
diff --git a/products/product_analytics/backend/facade/events_retention.py b/products/product_analytics/backend/facade/events_retention.py
index 8ccf92cff8d3..7ef97eb39796 100644
--- a/products/product_analytics/backend/facade/events_retention.py
+++ b/products/product_analytics/backend/facade/events_retention.py
@@ -1,7 +1,8 @@
"""
Events retention settings for organizations and projects.
-The windows set here are the ones the monthly events cleanup deletes by, so a saved window deletes events.
+The periods set here are the ones the events cleanup deletes by, at every monthly cleanup. Projects without a period
+keep events for their plan's default period, which the cleanup every 3 months deletes by.
"""
from posthog.models.organization import Organization
@@ -9,17 +10,17 @@ from posthog.models.team import Team
from posthog.models.user import User
from products.product_analytics.backend import events_retention
-from products.product_analytics.backend.facade.contracts import OrganizationEventsRetention
+from products.product_analytics.backend.facade.contracts import OrganizationEventsRetention, ProjectEventsRetention
def organization_events_retention(organization: Organization) -> OrganizationEventsRetention:
- """The organization's window and the range its admins can set."""
+ """The organization's period and the periods its admins can set."""
return events_retention.organization_events_retention(organization)
-def project_events_retention_months(team: Team) -> int | None:
- """The window set for this project, or None while it uses the organization's window."""
- return events_retention.project_events_retention_months(team)
+def project_events_retention(team: Team) -> ProjectEventsRetention:
+ """The period set for this project and its next deletion."""
+ return events_retention.project_events_retention(team)
def set_organization_events_retention(
@@ -30,7 +31,7 @@ def set_organization_events_retention(
was_impersonated: bool,
analytics_properties: dict[str, object] | None = None,
) -> None:
- """Set or clear the organization's window. Raises a Django ValidationError for a window outside the range."""
+ """Set or clear the organization's period. Raises a Django ValidationError for a period its admins can't set."""
events_retention.set_organization_events_retention(
organization, months, user=user, was_impersonated=was_impersonated, analytics_properties=analytics_properties
)
@@ -44,7 +45,7 @@ def set_project_events_retention(
was_impersonated: bool,
analytics_properties: dict[str, object] | None = None,
) -> None:
- """Set or clear the project's window. Raises a Django ValidationError for a window outside the range."""
+ """Set or clear the project's period. Raises a Django ValidationError for a period its admins can't set."""
events_retention.set_project_events_retention(
team, months, user=user, was_impersonated=was_impersonated, analytics_properties=analytics_properties
)
diff --git a/products/product_analytics/backend/presentation/events_retention.py b/products/product_analytics/backend/presentation/events_retention.py
index 84f6de84c049..d9ffcfe41f20 100644
--- a/products/product_analytics/backend/presentation/events_retention.py
+++ b/products/product_analytics/backend/presentation/events_retention.py
@@ -18,7 +18,7 @@ from posthog.permissions import OrganizationAdminWritePermissions
from products.product_analytics.backend.facade.contracts import OrganizationEventsRetention
from products.product_analytics.backend.facade.events_retention import (
organization_events_retention,
- project_events_retention_months,
+ project_events_retention,
set_organization_events_retention,
set_project_events_retention,
)
@@ -61,15 +61,19 @@ class OrganizationEventsRetentionSerializer(serializers.Serializer):
)
plan_retention_months = serializers.IntegerField(
read_only=True,
- help_text="The longest window the organization's plan allows, in months.",
+ help_text="The longest period the organization's plan allows, in months.",
)
min_retention_months = serializers.IntegerField(
read_only=True,
- help_text="The shortest window an admin can set, in months.",
+ help_text="The shortest period an admin can set, in months.",
)
max_retention_months = serializers.IntegerField(
read_only=True,
- help_text="The longest window an admin can set, in months. Never longer than the plan's window.",
+ help_text="The longest period an admin can set, in months. Never longer than the plan's period.",
+ )
+ next_monthly_cleanup_on = serializers.DateField(
+ read_only=True,
+ help_text="The date of the next monthly cleanup, in the timezone it runs in.",
)
@@ -78,12 +82,12 @@ class EventsRetentionSerializer(OrganizationEventsRetentionSerializer):
retention_months = serializers.IntegerField(
read_only=True,
allow_null=True,
- help_text="How many months of events stay queryable, counted back from today. Null while no retention window applies to the project.",
+ help_text="How many months of events stay queryable, counted back from today. Null while no retention period applies to the project.",
)
retained_from = serializers.DateField(
read_only=True,
allow_null=True,
- help_text="The earliest date whose events are still queryable, in the project's timezone. Null while no retention window applies to the project.",
+ help_text="The earliest date whose events are still queryable, in the project's timezone. Null while no retention period applies to the project.",
)
docs_url = serializers.URLField(
read_only=True,
@@ -92,7 +96,30 @@ class EventsRetentionSerializer(OrganizationEventsRetentionSerializer):
project_retention_months = serializers.IntegerField(
read_only=True,
allow_null=True,
- help_text="The window, in months, set for this project. Null while the project uses the organization's window.",
+ help_text="The period, in months, set for this project. Null while the project uses the organization's period.",
+ )
+ next_deletion_on = serializers.DateField(
+ read_only=True,
+ allow_null=True,
+ help_text="The date of the next cleanup that deletes this project's events. Null while no deletion is scheduled.",
+ )
+ next_deletion_before = serializers.DateField(
+ read_only=True,
+ allow_null=True,
+ help_text="The next cleanup deletes this project's events from before this date. Null while no deletion is scheduled.",
+ )
+ scheduled_pass_on = serializers.DateField(
+ read_only=True,
+ allow_null=True,
+ help_text="The next cleanup every 3 months that deletes this project's events past its current period. Null while the project has none scheduled.",
+ )
+ first_pass_on = serializers.DateField(
+ read_only=True,
+ help_text="The earliest cleanup every 3 months for a period that gets shorter today.",
+ )
+ default_period_deletes = serializers.BooleanField(
+ read_only=True,
+ help_text="Whether the cleanup deletes this project's events past the default period while no period is set.",
)
@@ -100,7 +127,7 @@ class EventsRetentionUpdateSerializer(serializers.Serializer):
retention_months = serializers.IntegerField(
allow_null=True,
min_value=1,
- help_text="The window in months. Events older than this are deleted on the next monthly cleanup and can't be restored. Null removes the window.",
+ help_text="The period in months, one of retention_months_choices. Older events are deleted at the next monthly cleanup and can't be restored. Null removes the period.",
)
@@ -113,6 +140,7 @@ def _organization_fields(organization: OrganizationEventsRetention) -> dict[str,
"plan_retention_months": organization.plan_months,
"min_retention_months": organization.min_months,
"max_retention_months": organization.max_months,
+ "next_monthly_cleanup_on": organization.next_cleanup_on,
}
@@ -127,13 +155,19 @@ class EventsRetentionViewSet(TeamAndOrgViewSetMixin, viewsets.ViewSet):
retained_from = (
events_retention_floor_date(self.team, retention_months) if retention_months is not None else None
)
+ project = project_events_retention(self.team)
return Response(
EventsRetentionSerializer(
{
"retention_months": retention_months,
"retained_from": retained_from,
"docs_url": EVENTS_RETENTION_DOCS_URL,
- "project_retention_months": project_events_retention_months(self.team),
+ "project_retention_months": project.project_months,
+ "next_deletion_on": project.next_deletion.deletes_on if project.next_deletion else None,
+ "next_deletion_before": project.next_deletion.deletes_before if project.next_deletion else None,
+ "scheduled_pass_on": project.scheduled_pass_on,
+ "first_pass_on": project.first_pass_on,
+ "default_period_deletes": project.default_period_deletes,
**_organization_fields(organization_events_retention(self.organization)),
}
).data
@@ -141,11 +175,11 @@ class EventsRetentionViewSet(TeamAndOrgViewSetMixin, viewsets.ViewSet):
@extend_schema(
operation_id="events_retention_retrieve",
- summary="Get the events retention window for a project",
+ summary="Get the events retention period for a project",
description=(
- "Returns how far back events stay queryable for this project, the windows set for the project and its "
- "organization, and the range an admin can set. Both window fields are null while no retention window "
- "applies to the project."
+ "Returns how far back events stay queryable for this project, the periods set for the project and its "
+ "organization, the periods an admin can set, and the project's next deletion. Both queryable period fields "
+ "are null while no retention period applies to the project."
),
responses={200: EventsRetentionSerializer},
)
@@ -155,14 +189,15 @@ class EventsRetentionViewSet(TeamAndOrgViewSetMixin, viewsets.ViewSet):
@validated_request(
EventsRetentionUpdateSerializer,
operation_id="events_retention_update",
- summary="Set the events retention window for a project",
+ summary="Set the events retention period for a project",
description=(
- "Sets how long this project keeps events, overriding the organization's window. Events older than the "
- "window are deleted on the next monthly cleanup and can't be restored. Organization admins only."
+ "Sets how long this project keeps events, overriding the organization's period. The period has to be one "
+ "of retention_months_choices. Older events are deleted at the next monthly cleanup and can't be restored. "
+ "Organization admins only."
),
responses={
- 200: OpenApiResponse(response=EventsRetentionSerializer, description="The project's new window."),
- 400: OpenApiResponse(description="The window is outside the range an admin can set."),
+ 200: OpenApiResponse(response=EventsRetentionSerializer, description="The project's new period."),
+ 400: OpenApiResponse(description="The period isn't one of the periods an admin can set."),
403: OpenApiResponse(description="The user isn't an organization admin."),
},
)
@@ -195,10 +230,10 @@ class OrganizationEventsRetentionViewSet(TeamAndOrgViewSetMixin, viewsets.ViewSe
@extend_schema(
operation_id="organization_events_retention_retrieve",
- summary="Get the organization's events retention window",
+ summary="Get the organization's events retention period",
description=(
"Returns the events retention period that every project without its own period uses, the default period "
- "while none is set, and the range an organization admin can set."
+ "while none is set, and the periods an organization admin can set."
),
responses={200: OrganizationEventsRetentionSerializer},
)
@@ -208,16 +243,17 @@ class OrganizationEventsRetentionViewSet(TeamAndOrgViewSetMixin, viewsets.ViewSe
@validated_request(
EventsRetentionUpdateSerializer,
operation_id="organization_events_retention_update",
- summary="Set the organization's events retention window",
+ summary="Set the organization's events retention period",
description=(
- "Sets how long every project without its own window keeps events. Events older than the window are "
- "deleted on the next monthly cleanup and can't be restored. Organization admins only."
+ "Sets how long every project without its own period keeps events. The period has to be one of "
+ "retention_months_choices. Older events are deleted at the next monthly cleanup and can't be restored. "
+ "Organization admins only."
),
responses={
200: OpenApiResponse(
- response=OrganizationEventsRetentionSerializer, description="The organization's new window."
+ response=OrganizationEventsRetentionSerializer, description="The organization's new period."
),
- 400: OpenApiResponse(description="The window is outside the range an admin can set."),
+ 400: OpenApiResponse(description="The period isn't one of the periods an admin can set."),
403: OpenApiResponse(description="The user isn't an organization admin."),
},
)
diff --git a/products/product_analytics/backend/tests/api/test_events_retention.py b/products/product_analytics/backend/tests/api/test_events_retention.py
index b845ee74753c..4598e383dc82 100644
--- a/products/product_analytics/backend/tests/api/test_events_retention.py
+++ b/products/product_analytics/backend/tests/api/test_events_retention.py
@@ -1,3 +1,5 @@
+from datetime import date
+
import time_machine
from posthog.test.base import APIBaseTest
from unittest.mock import patch
@@ -10,8 +12,10 @@ from rest_framework import status
from posthog.models import Organization
from posthog.models.activity_logging.activity_log import ActivityLog
+from posthog.models.events_plan_deletion import TeamEventsPlanDeletion
from posthog.models.events_retention_config import OrganizationEventsRetentionConfig, TeamEventsRetentionConfig
from posthog.models.organization import OrganizationMembership
+from posthog.models.team.event_retention import EVENTS_DATA_RETENTION_FLAG
from products.access_control.backend.facade.testing import create_access_control
from products.product_analytics.backend.facade.events_retention import (
@@ -78,6 +82,11 @@ class TestEventsRetentionAPI(APIBaseTest):
"retained_from": retained_from,
"docs_url": "https://posthog.com/docs/data/events-retention",
"project_retention_months": None,
+ "next_deletion_on": None,
+ "next_deletion_before": None,
+ "scheduled_pass_on": None,
+ "first_pass_on": "2027-01-03",
+ "default_period_deletes": True,
"organization_retention_months": None,
"default_retention_months": 12,
"default_is_grandfathered": False,
@@ -85,8 +94,50 @@ class TestEventsRetentionAPI(APIBaseTest):
"min_retention_months": 1,
"max_retention_months": 12,
"retention_months_choices": [1, 3, 12, 24, 36, 60, 84],
+ "next_monthly_cleanup_on": "2026-10-03",
}
+ @parameterized.expand(
+ [
+ ("pass", 24, 24, None, ("2026-11-03", "2024-11-01", "2026-11-03")),
+ ("shorter_period_pushes_the_pass", 24, 12, None, ("2027-01-03", "2026-01-01", "2027-01-03")),
+ ("set_period_keeps_its_pass", 12, 24, 12, ("2026-10-03", "2025-10-01", "2026-11-03")),
+ ("set_period_without_a_pass", None, 24, 12, ("2026-10-03", "2025-10-01", None)),
+ ("not_admitted", None, 84, None, (None, None, None)),
+ (
+ "due_pass_shortens_a_set_period_past_the_plan",
+ 12,
+ 12,
+ 36,
+ ("2026-10-03", "2025-10-01", "2026-10-03"),
+ date(2026, 10, 3),
+ ),
+ ("overdue_pass", 24, 24, None, ("2026-10-03", "2024-10-01", "2026-09-03"), date(2026, 9, 3)),
+ ]
+ )
+ @time_machine.travel("2026-09-22T20:00:00Z", tick=False)
+ def test_reports_the_next_deletion(
+ self,
+ _name: str,
+ stored_months: int | None,
+ plan_months: int,
+ setting_months: int | None,
+ expected: tuple[str | None, str | None, str | None],
+ pass_on: date = date(2026, 11, 3),
+ ) -> None:
+ self.team.event_retention_months = plan_months
+ self.team.save()
+ if setting_months is not None:
+ TeamEventsRetentionConfig.objects.create(team=self.team, events_retention_months=setting_months)
+ if stored_months is not None:
+ TeamEventsPlanDeletion.objects.create(team=self.team, months=stored_months, next_deletion_on=pass_on)
+
+ response = self.client.get(self.url())
+
+ assert response.status_code == status.HTTP_200_OK
+ body = response.json()
+ assert (body["next_deletion_on"], body["next_deletion_before"], body["scheduled_pass_on"]) == expected
+
@parameterized.expand(
[
("standard_plan_defaults_to_24_months", True, None, 24, False),
@@ -114,12 +165,35 @@ class TestEventsRetentionAPI(APIBaseTest):
@override_settings(EVENTS_DATA_RETENTION_ENFORCED=False)
def test_reports_no_window_until_retention_is_enforced(self) -> None:
+ TeamEventsPlanDeletion.objects.create(team=self.team, months=24, next_deletion_on=date(2027, 2, 3))
+
response = self.client.get(self.url())
assert response.status_code == status.HTTP_200_OK
assert response.json()["retention_months"] is None
assert response.json()["retained_from"] is None
+ assert response.json()["next_deletion_on"] is None
+ assert response.json()["default_period_deletes"] is False
+
+ @override_settings(EVENTS_DATA_RETENTION_ENFORCED=None)
+ @time_machine.travel("2026-09-22T20:00:00Z", tick=False)
+ def test_reports_no_pass_while_deletion_is_off(self) -> None:
+ TeamEventsPlanDeletion.objects.create(team=self.team, months=24, next_deletion_on=date(2026, 11, 3))
+
+ with (
+ patch("posthog.models.team.event_retention.is_cloud", return_value=True),
+ patch(
+ "posthoganalytics.feature_enabled",
+ side_effect=lambda flag, *args, **kwargs: flag == EVENTS_DATA_RETENTION_FLAG,
+ ),
+ ):
+ response = self.client.get(self.url())
+
+ assert response.status_code == status.HTTP_200_OK
+ assert response.json()["retention_months"] is not None
+ assert (response.json()["next_deletion_on"], response.json()["default_period_deletes"]) == (None, False)
+ @time_machine.travel("2026-09-22T20:00:00Z", tick=False)
def test_organization_admin_sets_and_clears_the_project_window(self) -> None:
self._set_membership("organization_admin")
OrganizationEventsRetentionConfig.objects.create(
@@ -131,6 +205,7 @@ class TestEventsRetentionAPI(APIBaseTest):
assert response.status_code == status.HTTP_200_OK
assert response.json()["retention_months"] == 3
assert response.json()["project_retention_months"] == 3
+ assert response.json()["next_deletion_on"] == "2026-10-03"
assert ActivityLog.objects.filter(team_id=self.team.id, scope="Team", activity="updated").exists()
response = self.client.post(self.url(), {"retention_months": None}, format="json")
@@ -140,6 +215,20 @@ class TestEventsRetentionAPI(APIBaseTest):
assert response.json()["project_retention_months"] is None
assert not TeamEventsRetentionConfig.objects.filter(team=self.team).exists()
+ @time_machine.travel("2026-09-22T20:00:00Z", tick=False)
+ def test_clearing_a_set_period_after_a_plan_upgrade_waits_a_full_interval(self) -> None:
+ self._set_membership("organization_admin")
+ self._set_plan_years(7, standard=True)
+ self.team.event_retention_months = 84
+ self.team.save()
+ TeamEventsRetentionConfig.objects.create(team=self.team, events_retention_months=36)
+ TeamEventsPlanDeletion.objects.create(team=self.team, months=12, next_deletion_on=date(2026, 10, 3))
+
+ response = self.client.post(self.url(), {"retention_months": None}, format="json")
+
+ assert response.status_code == status.HTTP_200_OK
+ assert (response.json()["retention_months"], response.json()["scheduled_pass_on"]) == (24, "2027-01-03")
+
@parameterized.expand(
[
("longer_than_the_plan", "organization_admin", 36, status.HTTP_400_BAD_REQUEST),
@@ -159,6 +248,7 @@ class TestEventsRetentionAPI(APIBaseTest):
assert not TeamEventsRetentionConfig.objects.filter(team=self.team).exists()
assert not any(call.kwargs.get("event") == "events retention set" for call in capture.call_args_list)
+ @time_machine.travel("2026-09-22T20:00:00Z", tick=False)
def test_organization_admin_sets_and_clears_the_default_window(self) -> None:
self._set_membership("organization_admin")
@@ -173,8 +263,10 @@ class TestEventsRetentionAPI(APIBaseTest):
"min_retention_months": 1,
"max_retention_months": 24,
"retention_months_choices": [1, 3, 12, 24, 36, 60, 84],
+ "next_monthly_cleanup_on": "2026-10-03",
}
- assert self.client.get(self.url()).json()["retention_months"] == 12
+ project = self.client.get(self.url()).json()
+ assert (project["retention_months"], project["next_deletion_on"]) == (12, "2026-10-03")
assert ActivityLog.objects.filter(
organization_id=self.organization.id, scope="Organization", activity="updated"
).exists()
@@ -222,6 +314,7 @@ class TestEventsRetentionAPI(APIBaseTest):
== 12
)
assert TeamEventsRetentionConfig.objects.get(team=self.team).events_retention_months == 12
+ assert not TeamEventsPlanDeletion.objects.filter(team=self.team).exists()
@parameterized.expand([("patch",), ("put",), ("delete",)])
def test_rejects_other_writes(self, method: str) -> None:
diff --git a/products/product_analytics/frontend/generated/api.schemas.ts b/products/product_analytics/frontend/generated/api.schemas.ts
index 6f1f4ffab906..b9dcc0b2ec7b 100644
--- a/products/product_analytics/frontend/generated/api.schemas.ts
+++ b/products/product_analytics/frontend/generated/api.schemas.ts
@@ -19,17 +19,19 @@ export interface OrganizationEventsRetentionApi {
readonly default_is_grandfathered: boolean
/** The periods, in months, an admin can choose from. A save also has to be between min_retention_months and max_retention_months. Only PostHog staff can set other periods. */
readonly retention_months_choices: readonly number[]
- /** The longest window the organization's plan allows, in months. */
+ /** The longest period the organization's plan allows, in months. */
readonly plan_retention_months: number
- /** The shortest window an admin can set, in months. */
+ /** The shortest period an admin can set, in months. */
readonly min_retention_months: number
- /** The longest window an admin can set, in months. Never longer than the plan's window. */
+ /** The longest period an admin can set, in months. Never longer than the plan's period. */
readonly max_retention_months: number
+ /** The date of the next monthly cleanup, in the timezone it runs in. */
+ readonly next_monthly_cleanup_on: string
}
export interface EventsRetentionUpdateApi {
/**
- * The window in months. Events older than this are deleted on the next monthly cleanup and can't be restored. Null removes the window.
+ * The period in months, one of retention_months_choices. Older events are deleted at the next monthly cleanup and can't be restored. Null removes the period.
* @minimum 1
* @nullable
*/
@@ -252,29 +254,50 @@ export interface EventsRetentionApi {
readonly default_is_grandfathered: boolean
/** The periods, in months, an admin can choose from. A save also has to be between min_retention_months and max_retention_months. Only PostHog staff can set other periods. */
readonly retention_months_choices: readonly number[]
- /** The longest window the organization's plan allows, in months. */
+ /** The longest period the organization's plan allows, in months. */
readonly plan_retention_months: number
- /** The shortest window an admin can set, in months. */
+ /** The shortest period an admin can set, in months. */
readonly min_retention_months: number
- /** The longest window an admin can set, in months. Never longer than the plan's window. */
+ /** The longest period an admin can set, in months. Never longer than the plan's period. */
readonly max_retention_months: number
+ /** The date of the next monthly cleanup, in the timezone it runs in. */
+ readonly next_monthly_cleanup_on: string
/**
- * How many months of events stay queryable, counted back from today. Null while no retention window applies to the project.
+ * How many months of events stay queryable, counted back from today. Null while no retention period applies to the project.
* @nullable
*/
readonly retention_months: number | null
/**
- * The earliest date whose events are still queryable, in the project's timezone. Null while no retention window applies to the project.
+ * The earliest date whose events are still queryable, in the project's timezone. Null while no retention period applies to the project.
* @nullable
*/
readonly retained_from: string | null
/** Where the events retention policy is documented. */
readonly docs_url: string
/**
- * The window, in months, set for this project. Null while the project uses the organization's window.
+ * The period, in months, set for this project. Null while the project uses the organization's period.
* @nullable
*/
readonly project_retention_months: number | null
+ /**
+ * The date of the next cleanup that deletes this project's events. Null while no deletion is scheduled.
+ * @nullable
+ */
+ readonly next_deletion_on: string | null
+ /**
+ * The next cleanup deletes this project's events from before this date. Null while no deletion is scheduled.
+ * @nullable
+ */
+ readonly next_deletion_before: string | null
+ /**
+ * The next cleanup every 3 months that deletes this project's events past its current period. Null while the project has none scheduled.
+ * @nullable
+ */
+ readonly scheduled_pass_on: string | null
+ /** The earliest cleanup every 3 months for a period that gets shorter today. */
+ readonly first_pass_on: string
+ /** Whether the cleanup deletes this project's events past the default period while no period is set. */
+ readonly default_period_deletes: boolean
}
export type InsightVizNodeApiKind = (typeof InsightVizNodeApiKind)[keyof typeof InsightVizNodeApiKind]
diff --git a/products/product_analytics/frontend/generated/api.ts b/products/product_analytics/frontend/generated/api.ts
index f91789b0edfd..88e6fbd4af41 100644
--- a/products/product_analytics/frontend/generated/api.ts
+++ b/products/product_analytics/frontend/generated/api.ts
@@ -86,8 +86,8 @@ export const getOrganizationEventsRetentionRetrieveUrl = (organizationId: string
}
/**
- * Returns the events retention period that every project without its own period uses, the default period while none is set, and the range an organization admin can set.
- * @summary Get the organization's events retention window
+ * Returns the events retention period that every project without its own period uses, the default period while none is set, and the periods an organization admin can set.
+ * @summary Get the organization's events retention period
*/
export const organizationEventsRetentionRetrieve = async (
organizationId: string,
@@ -104,8 +104,8 @@ export const getOrganizationEventsRetentionUpdateUrl = (organizationId: string)
}
/**
- * Sets how long every project without its own window keeps events. Events older than the window are deleted on the next monthly cleanup and can't be restored. Organization admins only.
- * @summary Set the organization's events retention window
+ * Sets how long every project without its own period keeps events. The period has to be one of retention_months_choices. Older events are deleted at the next monthly cleanup and can't be restored. Organization admins only.
+ * @summary Set the organization's events retention period
*/
export const organizationEventsRetentionUpdate = async (
organizationId: string,
@@ -396,8 +396,8 @@ export const getEventsRetentionRetrieveUrl = (projectId: string) => {
}
/**
- * Returns how far back events stay queryable for this project, the windows set for the project and its organization, and the range an admin can set. Both window fields are null while no retention window applies to the project.
- * @summary Get the events retention window for a project
+ * Returns how far back events stay queryable for this project, the periods set for the project and its organization, the periods an admin can set, and the project's next deletion. Both queryable period fields are null while no retention period applies to the project.
+ * @summary Get the events retention period for a project
*/
export const eventsRetentionRetrieve = async (
projectId: string,
@@ -414,8 +414,8 @@ export const getEventsRetentionUpdateUrl = (projectId: string) => {
}
/**
- * Sets how long this project keeps events, overriding the organization's window. Events older than the window are deleted on the next monthly cleanup and can't be restored. Organization admins only.
- * @summary Set the events retention window for a project
+ * Sets how long this project keeps events, overriding the organization's period. The period has to be one of retention_months_choices. Older events are deleted at the next monthly cleanup and can't be restored. Organization admins only.
+ * @summary Set the events retention period for a project
*/
export const eventsRetentionUpdate = async (
projectId: string,
diff --git a/products/product_analytics/frontend/generated/api.zod.ts b/products/product_analytics/frontend/generated/api.zod.ts
index 21b558aa2206..4fc8e0129b34 100644
--- a/products/product_analytics/frontend/generated/api.zod.ts
+++ b/products/product_analytics/frontend/generated/api.zod.ts
@@ -10,8 +10,8 @@
import * as zod from 'zod'
/**
- * Sets how long every project without its own window keeps events. Events older than the window are deleted on the next monthly cleanup and can't be restored. Organization admins only.
- * @summary Set the organization's events retention window
+ * Sets how long every project without its own period keeps events. The period has to be one of retention_months_choices. Older events are deleted at the next monthly cleanup and can't be restored. Organization admins only.
+ * @summary Set the organization's events retention period
*/
export const OrganizationEventsRetentionUpdateBody = /* @__PURE__ */ zod.object({
@@ -20,7 +20,7 @@ export const OrganizationEventsRetentionUpdateBody = /* @__PURE__ */ zod.object(
.min(1)
.nullable()
.describe(
- "The window in months. Events older than this are deleted on the next monthly cleanup and can't be restored. Null removes the window."
+ "The period in months, one of retention_months_choices. Older events are deleted at the next monthly cleanup and can't be restored. Null removes the period."
),
})
@@ -207,8 +207,8 @@ export const ElementsPartialUpdateBody = /* @__PURE__ */ zod.object({
})
/**
- * Sets how long this project keeps events, overriding the organization's window. Events older than the window are deleted on the next monthly cleanup and can't be restored. Organization admins only.
- * @summary Set the events retention window for a project
+ * Sets how long this project keeps events, overriding the organization's period. The period has to be one of retention_months_choices. Older events are deleted at the next monthly cleanup and can't be restored. Organization admins only.
+ * @summary Set the events retention period for a project
*/
export const EventsRetentionUpdateBody = /* @__PURE__ */ zod.object({
@@ -217,7 +217,7 @@ export const EventsRetentionUpdateBody = /* @__PURE__ */ zod.object({
.min(1)
.nullable()
.describe(
- "The window in months. Events older than this are deleted on the next monthly cleanup and can't be restored. Null removes the window."
+ "The period in months, one of retention_months_choices. Older events are deleted at the next monthly cleanup and can't be restored. Null removes the period."
),
})
diff --git a/products/product_analytics/frontend/settings/EventsRetentionSettings.stories.tsx b/products/product_analytics/frontend/settings/EventsRetentionSettings.stories.tsx
index 47dfa95cf893..34c33c112e20 100644
--- a/products/product_analytics/frontend/settings/EventsRetentionSettings.stories.tsx
+++ b/products/product_analytics/frontend/settings/EventsRetentionSettings.stories.tsx
@@ -31,6 +31,7 @@ const retentionMocks = (
min_retention_months: 1,
max_retention_months: 36,
retention_months_choices: [1, 3, 12, 24, 36, 60, 84],
+ next_monthly_cleanup_on: '2026-11-03',
}
const project: EventsRetentionApi = {
...organization,
@@ -38,6 +39,11 @@ const retentionMocks = (
retained_from: '2024-11-01',
docs_url: 'https://posthog.com/docs/data/events-retention',
project_retention_months: projectMonths,
+ next_deletion_on: null,
+ next_deletion_before: null,
+ scheduled_pass_on: null,
+ first_pass_on: '2027-02-03',
+ default_period_deletes: false,
}
return mswDecorator({
get: {
diff --git a/services/mcp/src/api/generated.ts b/services/mcp/src/api/generated.ts
index a890fe5fcaf9..b082f80083e1 100644
--- a/services/mcp/src/api/generated.ts
+++ b/services/mcp/src/api/generated.ts
@@ -42506,34 +42506,55 @@ export namespace Schemas {
readonly default_is_grandfathered: boolean;
/** The periods, in months, an admin can choose from. A save also has to be between min_retention_months and max_retention_months. Only PostHog staff can set other periods. */
readonly retention_months_choices: readonly number[];
- /** The longest window the organization's plan allows, in months. */
+ /** The longest period the organization's plan allows, in months. */
readonly plan_retention_months: number;
- /** The shortest window an admin can set, in months. */
+ /** The shortest period an admin can set, in months. */
readonly min_retention_months: number;
- /** The longest window an admin can set, in months. Never longer than the plan's window. */
+ /** The longest period an admin can set, in months. Never longer than the plan's period. */
readonly max_retention_months: number;
+ /** The date of the next monthly cleanup, in the timezone it runs in. */
+ readonly next_monthly_cleanup_on: string;
/**
- * How many months of events stay queryable, counted back from today. Null while no retention window applies to the project.
+ * How many months of events stay queryable, counted back from today. Null while no retention period applies to the project.
* @nullable
*/
readonly retention_months: number | null;
/**
- * The earliest date whose events are still queryable, in the project's timezone. Null while no retention window applies to the project.
+ * The earliest date whose events are still queryable, in the project's timezone. Null while no retention period applies to the project.
* @nullable
*/
readonly retained_from: string | null;
/** Where the events retention policy is documented. */
readonly docs_url: string;
/**
- * The window, in months, set for this project. Null while the project uses the organization's window.
+ * The period, in months, set for this project. Null while the project uses the organization's period.
* @nullable
*/
readonly project_retention_months: number | null;
+ /**
+ * The date of the next cleanup that deletes this project's events. Null while no deletion is scheduled.
+ * @nullable
+ */
+ readonly next_deletion_on: string | null;
+ /**
+ * The next cleanup deletes this project's events from before this date. Null while no deletion is scheduled.
+ * @nullable
+ */
+ readonly next_deletion_before: string | null;
+ /**
+ * The next cleanup every 3 months that deletes this project's events past its current period. Null while the project has none scheduled.
+ * @nullable
+ */
+ readonly scheduled_pass_on: string | null;
+ /** The earliest cleanup every 3 months for a period that gets shorter today. */
+ readonly first_pass_on: string;
+ /** Whether the cleanup deletes this project's events past the default period while no period is set. */
+ readonly default_period_deletes: boolean;
}
export interface EventsRetentionUpdate {
/**
- * The window in months. Events older than this are deleted on the next monthly cleanup and can't be restored. Null removes the window.
+ * The period in months, one of retention_months_choices. Older events are deleted at the next monthly cleanup and can't be restored. Null removes the period.
* @minimum 1
* @nullable
*/
@@ -70027,12 +70048,14 @@ export namespace Schemas {
readonly default_is_grandfathered: boolean;
/** The periods, in months, an admin can choose from. A save also has to be between min_retention_months and max_retention_months. Only PostHog staff can set other periods. */
readonly retention_months_choices: readonly number[];
- /** The longest window the organization's plan allows, in months. */
+ /** The longest period the organization's plan allows, in months. */
readonly plan_retention_months: number;
- /** The shortest window an admin can set, in months. */
+ /** The shortest period an admin can set, in months. */
readonly min_retention_months: number;
- /** The longest window an admin can set, in months. Never longer than the plan's window. */
+ /** The longest period an admin can set, in months. Never longer than the plan's period. */
readonly max_retention_months: number;
+ /** The date of the next monthly cleanup, in the timezone it runs in. */
+ readonly next_monthly_cleanup_on: string;
}
export interface OrganizationFeatureFlagRow {