:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(aio): expire unused datasets and historical versions (#128324)
Radu Raicea57187cab0fc1
diff --git a/docs/internal/ai-offline-evaluation-reporting.md b/docs/internal/ai-offline-evaluation-reporting.mdindex 919779c2216d..30035c66eb90 100644--- a/docs/internal/ai-offline-evaluation-reporting.md+++ b/docs/internal/ai-offline-evaluation-reporting.md@@ -492,6 +492,24 @@ The reporting integration uses the existing ledger schema without an additionalTemporal computes the offline ledger's exclusive day end inside the aggregation activity.The shared workflow window and already serialized reports are unchanged.+## Dataset retention++Datasets expire after inactivity: 90 days for active datasets and their current revision, and 30 days for archived datasets and historical revisions. Creating a dataset starts its first retention period. Archiving or restoring a dataset restarts its period, and replacing the current revision starts a new 30-day period for that historical revision. Existing data gets a full period from the retention migration, rather than being judged by its creation date.++Opening a dataset, reading its items through the UI or API, editing it, exporting it, and using it in an offline evaluation count as use. Reading a particular snapshot refreshes that revision and its dataset; it does not refresh unrelated historical revisions. Reading item version history preserves the returned item versions. Dataset lists, revision lists, and export status polling do not refresh usage.++Usage is recorded in Postgres at most once per 12 hours per resource on read paths. Requests with a recent timestamp do not issue a usage write. Expiry includes an extra 12 hours so coalescing cannot shorten the promised inactivity period. Usage writes and cleanup lock the same dataset row; cleanup rechecks eligibility under that lock. Uploading experiments protect their referenced snapshots while snapshot use is within the 30-day history window, including the 12-hour margin. Uploads of new items or results refresh usage, as does closing an experiment. An abandoned upload cannot protect an unused snapshot indefinitely.++The hourly cleanup marks expired datasets and snapshots unavailable before deleting item versions in bounded batches. A version is removable only when no retained snapshot needs it, it is not the item's current version, and it has not been read recently. Archive markers follow the same rule so pruning cannot make an archived item reappear. Expiring a whole dataset immediately releases its name for reuse and eventually removes all its item versions, items, and revisions. Names remain unique across active and archived datasets that have not expired. Completed experiments, evaluation items, scores, scorer versions, and usage ledger rows remain; optional dataset foreign keys become null in bounded batches while their provenance identifiers remain intact. Versions and revisions are deleted only after their references have been detached. Completed evaluation reads do not keep source datasets alive.++A failed dataset cleanup rolls back its content changes but preserves its attempt timestamp, so later candidates can still be processed. The next sweep rotates past the failed dataset before retrying it. Task time limits still stop the sweep. Cleanup reports failed dataset counts and uses `completed_with_errors` when it finishes with individual failures.++Dataset and revision responses expose `expires_at` and `expiring_soon`. The UI shows **Expiring soon** in the final seven days, replacing the Active or Archived status label, without an extension button. Normal use refreshes retention. The deadline is the earliest cleanup eligibility time, not a promise of immediate deletion. Cleanup emits a `dataset retention run` event with expiry and deletion counts, duration, completion status, and whether the time budget was exhausted.++Staff can pause or resume cleanup under **Organization tools > AI observability dataset retention** in Django admin (`/admin/posthog/organization/ai-observability-dataset-retention/`). The `AI_OBSERVABILITY_DATASET_RETENTION_ENABLED` instance setting defaults to enabled and applies separately to each region. Disabling stops both new expiry and unfinished deletion; a running sweep checks the setting before each dataset and finishes its current dataset before stopping. Re-enabling resumes the hourly schedule. Disabling does not restore deleted content or make already expired datasets available. Successful expiry/deletion logs include the team and dataset IDs after the transaction commits, and setting changes log the staff user and old/new state.++Candidate selection skips expired revisions that are retained only for shared versions or evaluation provenance, while continuing unfinished version and reference deletion. HogQL's `system.datasets` and `system.dataset_revisions` expose `expired_at`; queries must check the dataset and requested snapshot are available. Do not filter out a version solely because its introducing revision expired: retained snapshots can still need that version. HogQL queries and listing metadata do not refresh usage.+## Resource tagsDatasets, stable scorer definitions, and offline experiments use the shared project tag vocabulary.diff --git a/posthog/admin/admins/organization_admin.py b/posthog/admin/admins/organization_admin.pyindex 4c63758efe32..57b42a5da6fa 100644--- a/posthog/admin/admins/organization_admin.py+++ b/posthog/admin/admins/organization_admin.py@@ -12,12 +12,16 @@ from django.core.exceptions import PermissionDeniedfrom django.core.management import call_commandfrom django.core.management.base import CommandErrorfrom django.db import transaction+from django.http import HttpRequest, HttpResponsefrom django.shortcuts import redirect, renderfrom django.template.loader import render_to_string+from django.template.response import TemplateResponsefrom django.urls import path, reversefrom django.utils import timezone+from django.utils.decorators import method_decoratorfrom django.utils.html import format_htmlfrom django.utils.safestring import mark_safe+from django.views.decorators.http import require_http_methodsfrom posthog.admin.ai_training_opt_in_history import get_ai_training_opt_in_historyfrom posthog.admin.authorization import can_trigger_admin_deletion@@ -35,6 +39,7 @@ from posthog.person_db_router import PERSONS_DB_MODELSfrom posthog.tasks.ai_observability_usage_report import internal_reporting_team_idfrom posthog.utils import pluralize+from products.ai_observability.backend.facade.admin import dataset_retention_enabled, set_dataset_retention_enabledfrom products.batch_exports.backend.facade import api as batch_exports_apifrom products.batch_exports.backend.facade.contracts import BATCH_EXPORT_MODEL_LABEL@@ -133,6 +138,15 @@ class UsageReportForm(forms.Form):return report_date+class DatasetRetentionForm(forms.Form):+ enabled = forms.TypedChoiceField(+ label="Dataset cleanup",+ choices=[("enabled", "Enabled"), ("disabled", "Disabled")],+ coerce=lambda value: value == "enabled",+ widget=forms.RadioSelect,+ )++class BulkDeactivateOrganizationsForm(forms.Form):CUSTOM_REASON = "__custom__"MAX_ORGANIZATIONS = 100@@ -624,6 +638,11 @@ class OrganizationAdmin(admin.ModelAdmin):self.admin_site.admin_view(self.send_ai_observability_usage_report_view),name="send-ai-observability-usage-report",),+ path(+ "ai-observability-dataset-retention/",+ self.admin_site.admin_view(self.dataset_retention_view),+ name="ai-observability-dataset-retention",+ ),path("bulk-deactivate/",self.admin_site.admin_view(self.bulk_deactivate_view),@@ -822,6 +841,34 @@ class OrganizationAdmin(admin.ModelAdmin):{"form": form, "can_skip_existing_reports": internal_reporting_team_id() is not None},)+ @method_decorator(require_http_methods(["GET", "POST"]))+ def dataset_retention_view(self, request: HttpRequest) -> HttpResponse:+ user_id = request.user.pk+ if user_id is None or not request.user.has_perm("posthog.change_instancesetting"):+ raise PermissionDenied+ enabled = dataset_retention_enabled()+ form = DatasetRetentionForm(+ request.POST if request.method == "POST" else None,+ initial={"enabled": "enabled" if enabled else "disabled"},+ )+ if request.method == "POST" and form.is_valid():+ new_enabled = form.cleaned_data["enabled"]+ set_dataset_retention_enabled(enabled=new_enabled, user_id=user_id)+ messages.success(+ request, f"Dataset cleanup is now {'enabled' if new_enabled else 'disabled'} in this region."+ )+ return redirect(reverse("admin:ai-observability-dataset-retention"))+ return TemplateResponse(+ request,+ "admin/posthog/organization/ai_observability_dataset_retention.html",+ {+ **self.admin_site.each_context(request),+ "title": "AI observability dataset retention",+ "form": form,+ "enabled": enabled,+ },+ )+def change_view(self, request, object_id, form_url="", extra_context=None):extra_context = extra_context or {}# Store request for access in display methods (needed for CSRF tokens in templates)diff --git a/posthog/hogql/database/schema/system.py b/posthog/hogql/database/schema/system.pyindex fe1f39049099..e60b098e8241 100644--- a/posthog/hogql/database/schema/system.py+++ b/posthog/hogql/database/schema/system.py@@ -485,6 +485,9 @@ datasets: PostgresTable = PostgresTable("description": StringDatabaseField(name="description", description="What the dataset contains."),"metadata": StringJSONDatabaseField(name="metadata", description="JSON dataset metadata."),"archived": BooleanDatabaseField(name="archived", description="Whether the dataset is archived."),+ "expired_at": DateTimeDatabaseField(+ name="expired_at", nullable=True, description="When the dataset expired; NULL while available."+ ),"current_revision_id": UUIDDatabaseField(name="current_revision_id",nullable=True,@@ -511,6 +514,11 @@ dataset_revisions: PostgresTable = PostgresTable("team_id": IntegerDatabaseField(name="team_id"),"dataset_id": UUIDDatabaseField(name="dataset_id", description="Parent dataset; joins to datasets.id."),"revision": IntegerDatabaseField(name="revision", description="Monotonic revision number within the dataset."),+ "expired_at": DateTimeDatabaseField(+ name="expired_at",+ nullable=True,+ description="When the snapshot expired; NULL while available. Expired snapshots may be incomplete.",+ ),"created_by_id": IntegerDatabaseField(name="created_by_id", nullable=True, description="User who committed the revision."),diff --git a/posthog/hogql/database/test/__snapshots__/test_database.ambr b/posthog/hogql/database/test/__snapshots__/test_database.ambrindex 3e941711b2f6..ace6a1233b38 100644--- a/posthog/hogql/database/test/__snapshots__/test_database.ambr+++ b/posthog/hogql/database/test/__snapshots__/test_database.ambr@@ -5229,6 +5229,16 @@"table": null,"type": "integer"},+ "expired_at": {+ "chain": null,+ "fields": null,+ "hogql_value": "expired_at",+ "id": null,+ "name": "expired_at",+ "schema_valid": true,+ "table": null,+ "type": "datetime"+ },"created_by_id": {"chain": null,"fields": null,@@ -5308,6 +5318,16 @@"table": null,"type": "boolean"},+ "expired_at": {+ "chain": null,+ "fields": null,+ "hogql_value": "expired_at",+ "id": null,+ "name": "expired_at",+ "schema_valid": true,+ "table": null,+ "type": "datetime"+ },"current_revision_id": {"chain": null,"fields": null,@@ -20185,6 +20205,16 @@"table": null,"type": "integer"},+ "expired_at": {+ "chain": null,+ "fields": null,+ "hogql_value": "expired_at",+ "id": null,+ "name": "expired_at",+ "schema_valid": true,+ "table": null,+ "type": "datetime"+ },"created_by_id": {"chain": null,"fields": null,@@ -20264,6 +20294,16 @@"table": null,"type": "boolean"},+ "expired_at": {+ "chain": null,+ "fields": null,+ "hogql_value": "expired_at",+ "id": null,+ "name": "expired_at",+ "schema_valid": true,+ "table": null,+ "type": "datetime"+ },"current_revision_id": {"chain": null,"fields": null,diff --git a/posthog/settings/dynamic_settings.py b/posthog/settings/dynamic_settings.pyindex a888f7ceea4b..e90dda2579f3 100644--- a/posthog/settings/dynamic_settings.py+++ b/posthog/settings/dynamic_settings.py@@ -9,6 +9,11 @@ CONSTANCE_DATABASE_PREFIX = "constance:posthog:"# To edit, visit: ${SITE_URL}/admin/posthog/instancesetting/CONSTANCE_CONFIG = {+ "AI_OBSERVABILITY_DATASET_RETENTION_ENABLED": (+ True,+ "Enable expiry and deletion of unused AI observability datasets and historical versions in this region.",+ bool,+ ),"WAREHOUSE_PERSON_PROPERTY_SET_RATE_PER_SEC": (5000,"Global max rate (events/sec) at which the warehouse person-property consumer sends $set "diff --git a/posthog/tasks/scheduled.py b/posthog/tasks/scheduled.pyindex c29d83d8c339..4f293b658cdd 100644--- a/posthog/tasks/scheduled.py+++ b/posthog/tasks/scheduled.py@@ -78,7 +78,7 @@ from posthog.utils import get_crontab, get_instance_regionfrom products.account_audits.backend.facade.tasks import reconcile_account_auditsfrom products.aeo.backend.facade.tasks import run_aeo_citation_checks_task-from products.ai_observability.backend.facade.tasks import expire_offline_evaluation_payloads+from products.ai_observability.backend.facade.tasks import expire_datasets, expire_offline_evaluation_payloadsfrom products.ai_training.backend.facade.api import privacy_enabledfrom products.ai_training.backend.facade.tasks import process_ai_training_privacy_requestsfrom products.approvals.backend.tasks import (@@ -276,6 +276,7 @@ def add_periodic_task_with_expiry(def setup_periodic_tasks(sender: Celery, **kwargs: Any) -> None:+ add_periodic_task_with_expiry(sender, crontab(minute="37"), expire_datasets.s(), name="expire unused datasets")add_periodic_task_with_expiry(sender,crontab(minute="23"),diff --git a/posthog/templates/organization_tools/app.html b/posthog/templates/organization_tools/app.htmlindex 5177ed076cf5..6afd39c13333 100644--- a/posthog/templates/organization_tools/app.html+++ b/posthog/templates/organization_tools/app.html@@ -26,6 +26,14 @@<td>Deactivate organizations from pasted IDs after a preview step</td></tr>{% endif %}+ {% if perms.posthog.change_instancesetting %}+ <tr class="model-aiobservabilitydatasetretention">+ <th scope="row">+ <a href="{% url 'admin:ai-observability-dataset-retention' %}">AI observability dataset retention</a>+ </th>+ <td>Enable or disable automatic dataset cleanup in this region</td>+ </tr>+ {% endif %}</table></div>{% endif %}diff --git a/products/ai_observability/backend/admin.py b/products/ai_observability/backend/admin.pynew file mode 100644index 000000000000..9743f2ec14d8--- /dev/null+++ b/products/ai_observability/backend/admin.py@@ -0,0 +1,15 @@+import structlog++from posthog.models.instance_setting import set_instance_setting++from products.ai_observability.backend.dataset_retention import RETENTION_ENABLED_SETTING, dataset_retention_enabled++logger = structlog.get_logger(__name__)+++def set_dataset_retention_enabled(*, enabled: bool, user_id: int) -> None:+ previous_enabled = dataset_retention_enabled()+ set_instance_setting(RETENTION_ENABLED_SETTING, enabled)+ logger.info(+ "dataset_retention_setting_changed", user_id=user_id, previous_enabled=previous_enabled, enabled=enabled+ )diff --git a/products/ai_observability/backend/api/dataset_exports.py b/products/ai_observability/backend/api/dataset_exports.pyindex 289b4ef6cad6..2492e525f031 100644--- a/products/ai_observability/backend/api/dataset_exports.py+++ b/products/ai_observability/backend/api/dataset_exports.py@@ -11,7 +11,7 @@ from posthog.models import Team, Userfrom posthog.models.activity_logging.activity_log import Change, Detail, log_activityfrom products.ai_observability.backend.api.dataset_serializers import StrictDatasetSerializer-from products.ai_observability.backend.dataset_queries import latest_dataset_revision+from products.ai_observability.backend.dataset_retention import DatasetUsagefrom products.ai_observability.backend.models.datasets import Dataset, DatasetRevisionfrom products.exports.backend.facade.api import (DATASET_EXPORT_KIND,@@ -83,21 +83,13 @@ def create_dataset_export(was_impersonated: bool,revision: int | None,) -> ExportedAsset:- selected_revision = revision- if selected_revision is None:- latest_revision = latest_dataset_revision(team_id=team.id, dataset_id=dataset.id)- if latest_revision is None:- raise DatasetExportUnavailableError("Add an item before exporting this dataset.")- selected_revision = latest_revision.revision- elif (- not DatasetRevision.objects.for_team(team.id, canonical=True)- .filter(- dataset_id=dataset.id,- revision=selected_revision,- )- .exists()- ):- raise DatasetExportUnavailableError("This dataset revision does not exist.")+ try:+ selected = DatasetUsage.read_revision(dataset, revision)+ except (Dataset.DoesNotExist, DatasetRevision.DoesNotExist) as error:+ raise DatasetExportUnavailableError("This dataset revision is no longer available.") from error+ if selected is None:+ raise DatasetExportUnavailableError("Add an item before exporting this dataset.")+ selected_revision = selected.revisionasset = create_export_asset_async(team=team,diff --git a/products/ai_observability/backend/api/datasets.py b/products/ai_observability/backend/api/datasets.pyindex befbc29f9154..cfcb8ce547f8 100644--- a/products/ai_observability/backend/api/datasets.py+++ b/products/ai_observability/backend/api/datasets.py@@ -1,3 +1,4 @@+from datetime import datetimefrom functools import partialfrom typing import NoReturn, castfrom uuid import UUID@@ -62,6 +63,7 @@ from products.ai_observability.backend.dataset_queries import (consistent_dataset_item_versions,dataset_item_versions_at_revision,)+from products.ai_observability.backend.dataset_retention import DatasetRetentionPolicy, DatasetUsagefrom products.ai_observability.backend.dataset_service import (UNSET,DatasetItemMutationResult,@@ -199,6 +201,8 @@ def _resolved_dataset_revision_id(dataset: Dataset) -> UUID | None:class DatasetReadSerializer(ResourceTagsReadSerializer, UserAccessControlSerializerMixin, serializers.ModelSerializer):+ expires_at = serializers.SerializerMethodField(help_text="Earliest cleanup time if the dataset remains unused.")+ expiring_soon = serializers.SerializerMethodField(help_text="Whether the dataset is within seven days of expiry.")created_by = UserBasicSerializer(read_only=True, allow_null=True)metadata = JSONObjectField(read_only=True, help_text="JSON object with descriptive dataset metadata.")current_revision = serializers.SerializerMethodField(@@ -218,6 +222,8 @@ class DatasetReadSerializer(ResourceTagsReadSerializer, UserAccessControlSeriali"metadata","tags","archived",+ "expires_at",+ "expiring_soon","current_revision","current_revision_id","created_at",@@ -228,6 +234,14 @@ class DatasetReadSerializer(ResourceTagsReadSerializer, UserAccessControlSeriali]read_only_fields = fields+ @extend_schema_field(serializers.DateTimeField(allow_null=True))+ def get_expires_at(self, instance: Dataset) -> datetime | None:+ return DatasetRetentionPolicy.dataset_expires_at(instance)++ @extend_schema_field(serializers.BooleanField())+ def get_expiring_soon(self, instance: Dataset) -> bool:+ return DatasetRetentionPolicy.expiring_soon(self.get_expires_at(instance))+@extend_schema_field(serializers.IntegerField(allow_null=True))def get_current_revision(self, instance: Dataset) -> int | None:return _resolved_dataset_revision(instance)@@ -305,15 +319,25 @@ class DatasetListQuerySerializer(ResourceTagFilterSerializer):class DatasetRevisionReadSerializer(serializers.ModelSerializer):+ expires_at = serializers.SerializerMethodField(help_text="Earliest expiry time of this dataset snapshot if unused.")+ expiring_soon = serializers.SerializerMethodField(help_text="Whether this snapshot is within seven days of expiry.")created_by = UserBasicSerializer(read_only=True, allow_null=True)dataset_id = serializers.UUIDField(read_only=True, help_text="Dataset this revision belongs to.")team_id = serializers.IntegerField(read_only=True, help_text="Project that owns the revision.")class Meta:model = DatasetRevision- fields = ["id", "dataset_id", "revision", "created_at", "created_by", "team_id"]+ fields = ["id", "dataset_id", "revision", "created_at", "created_by", "team_id", "expires_at", "expiring_soon"]read_only_fields = fields+ @extend_schema_field(serializers.DateTimeField(allow_null=True))+ def get_expires_at(self, instance: DatasetRevision) -> datetime | None:+ return DatasetRetentionPolicy.revision_expires_at(instance)++ @extend_schema_field(serializers.BooleanField())+ def get_expiring_soon(self, instance: DatasetRevision) -> bool:+ return DatasetRetentionPolicy.expiring_soon(self.get_expires_at(instance))+class DatasetItemReadSerializer(serializers.ModelSerializer):id = serializers.UUIDField(@@ -595,7 +619,9 @@ def _conflict_response(error: DatasetMutationConflict | DatasetLimitExceeded) ->def _item_version_queryset() -> QuerySet[DatasetItemVersion, DatasetItemVersion]:return consistent_dataset_item_versions(- DatasetItemVersion.objects.unscoped().select_related(+ DatasetItemVersion.objects.unscoped()+ .filter(dataset__expired_at__isnull=True)+ .select_related("created_by","dataset","dataset_revision",@@ -609,6 +635,7 @@ def _dataset_queryset() -> QuerySet[Dataset, Dataset]:latest_revision = DatasetRevision.objects.unscoped().filter(dataset_id=OuterRef("id")).order_by("-revision")return (Dataset.objects.unscoped()+ .filter(expired_at__isnull=True).select_related("created_by", "current_revision").prefetch_related(ResourceTags.prefetch()).annotate(@@ -743,7 +770,12 @@ class DatasetViewSet(TeamAndOrgViewSetMixin, AccessControlViewSetMixin, GenericV@llma_track_latency("llma_datasets_retrieve")@monitor(feature=None, endpoint="llma_datasets_retrieve", method="GET")def retrieve(self, request: Request, *args: object, **kwargs: object) -> Response:- return Response(DatasetReadSerializer(self.get_object(), context=self.get_serializer_context()).data)+ dataset = self.get_object()+ try:+ DatasetUsage.record(dataset)+ except Dataset.DoesNotExist as error:+ raise Http404("Dataset not found.") from error+ return Response(DatasetReadSerializer(dataset, context=self.get_serializer_context()).data)@extend_schema(request=DatasetCreateSerializer,@@ -869,8 +901,8 @@ class DatasetViewSet(TeamAndOrgViewSetMixin, AccessControlViewSetMixin, GenericVdataset = self.get_object()queryset = (DatasetRevision.objects.for_team(self.team.id, canonical=True)- .filter(dataset_id=dataset.id)- .select_related("created_by")+ .filter(dataset_id=dataset.id, expired_at__isnull=True)+ .select_related("created_by", "dataset").order_by("-revision"))page = self.paginate_queryset(queryset)@@ -974,6 +1006,10 @@ class DatasetViewSet(TeamAndOrgViewSetMixin, AccessControlViewSetMixin, GenericV{"detail": "The export is still being prepared."},status=status.HTTP_409_CONFLICT,)+ try:+ DatasetUsage.read_revision(dataset, int((asset.export_context or {})["dataset_revision"]))+ except (Dataset.DoesNotExist, DatasetRevision.DoesNotExist) as error:+ raise Http404("Dataset revision not found.") from errorresponse = get_export_asset_content_response(asset=asset, download=True)self._report_action("llma dataset export downloaded",@@ -1100,22 +1136,16 @@ class DatasetItemViewSet(TeamAndOrgViewSetMixin, GenericViewSet):archived: bool = query["archived"]revision: int | None = query.get("revision")+ try:+ DatasetUsage.read_revision(dataset, revision)+ except (Dataset.DoesNotExist, DatasetRevision.DoesNotExist) as error:+ raise Http404("Dataset revision not found.") from errorif revision is None:queryset = self.get_queryset().filter(dataset_id=dataset.id,archived=archived,)else:- if (- not DatasetRevision.objects.for_team(self.team.id, canonical=True)- .filter(- dataset_id=dataset.id,- revision=revision,- )- .exists()- ):- raise Http404("Dataset revision not found.")-version_ids = dataset_item_versions_at_revision(team_id=self.team.id,dataset_id=dataset.id,@@ -1145,20 +1175,14 @@ class DatasetItemViewSet(TeamAndOrgViewSetMixin, GenericViewSet):query_serializer.is_valid(raise_exception=True)revision: int | None = query_serializer.validated_data.get("revision")current = self.get_object()+ try:+ DatasetUsage.read_revision(current.dataset, revision)+ except (Dataset.DoesNotExist, DatasetRevision.DoesNotExist) as error:+ raise Http404("Dataset revision not found.") from errorif revision is None:return Response(DatasetItemReadSerializer(current).data)dataset = current.dataset- if (- not DatasetRevision.objects.for_team(self.team.id, canonical=True)- .filter(- dataset_id=dataset.id,- revision=revision,- )- .exists()- ):- raise Http404("Dataset revision not found.")-version = (_item_version_queryset().filter(@@ -1326,6 +1350,11 @@ class DatasetItemViewSet(TeamAndOrgViewSetMixin, GenericViewSet):dataset_item_id=current.dataset_item_id,)page = self.paginate_queryset(queryset)+ versions = page if page is not None else list(queryset)+ try:+ DatasetUsage.record_versions(current.dataset, versions)+ except (Dataset.DoesNotExist, DatasetItemVersion.DoesNotExist) as error:+ raise Http404("Dataset item version not found.") from errorif page is not None:- return self.get_paginated_response(DatasetItemReadSerializer(page, many=True).data)- return Response(DatasetItemReadSerializer(queryset, many=True).data)+ return self.get_paginated_response(DatasetItemReadSerializer(versions, many=True).data)+ return Response(DatasetItemReadSerializer(versions, many=True).data)diff --git a/products/ai_observability/backend/api/test/test_datasets.py b/products/ai_observability/backend/api/test/test_datasets.pyindex 29df1fa75cf9..ce969493b1fa 100644--- a/products/ai_observability/backend/api/test/test_datasets.py+++ b/products/ai_observability/backend/api/test/test_datasets.py@@ -22,7 +22,7 @@ from posthog.models.utils import generate_random_token_personal, hash_key_valuefrom products.ai_observability.backend.api.dataset_exports import DATASET_EXPORT_STUCK_MESSAGEfrom products.ai_observability.backend.dataset_service import create_datasetfrom products.ai_observability.backend.models.datasets import Dataset, DatasetItem, DatasetItemVersion, DatasetRevision-from products.exports.backend.facade.api import DATASET_EXPORT_KIND, ExportedAsset+from products.exports.backend.facade.api import DATASET_EXPORT_KIND, JSONL_EXPORT_FORMAT, ExportedAssetclass TestDatasetsApi(APIBaseTest):@@ -68,6 +68,73 @@ class TestDatasetsApi(APIBaseTest):self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)+ def test_lists_do_not_refresh_usage_but_snapshot_reads_do(self) -> None:+ dataset = self._create_dataset()+ first = self._create_item(dataset["id"])+ self._create_item(dataset["id"], input="Second item")+ old = now() - timedelta(days=27)+ datasets = Dataset.objects.for_team(self.team.id, canonical=True).filter(id=dataset["id"])+ revisions = DatasetRevision.objects.for_team(self.team.id, canonical=True).filter(dataset_id=dataset["id"])+ datasets.update(last_used_at=old, archived=True)+ revisions.update(last_used_at=old)++ response = self.client.get(self.datasets_url, {"archived": "true"})+ self.assertEqual(response.status_code, status.HTTP_200_OK)+ self.assertTrue(response.data["results"][0]["expiring_soon"])+ response = self.client.get(f"{self.datasets_url}{dataset['id']}/revisions/")+ self.assertEqual(response.status_code, status.HTTP_200_OK)+ self.assertTrue(all(revision["expiring_soon"] for revision in response.data["results"]))+ self.assertEqual(datasets.get().last_used_at, old)+ self.assertTrue(all(revision.last_used_at == old for revision in revisions))++ response = self.client.get(self.items_url, {"dataset": dataset["id"], "revision": first["dataset_revision"]})+ self.assertEqual(response.status_code, status.HTTP_200_OK)+ last_used_at = datasets.get().last_used_at+ revision_last_used_at = revisions.get(revision=first["dataset_revision"]).last_used_at+ assert last_used_at is not None+ assert revision_last_used_at is not None+ self.assertGreater(last_used_at, old)+ self.assertGreater(revision_last_used_at, old)+ self.assertEqual(revisions.get(revision=2).last_used_at, old)++ def test_expired_snapshots_and_datasets_are_unavailable(self) -> None:+ dataset = self._create_dataset()+ first = self._create_item(dataset["id"])+ self._create_item(dataset["id"], input="Second item")+ asset = ExportedAsset.objects.create(+ team=self.team,+ created_by=self.user,+ export_format=JSONL_EXPORT_FORMAT,+ export_context={"kind": DATASET_EXPORT_KIND, "dataset_id": dataset["id"], "dataset_revision": 1},+ content=b'{"input":"Expired snapshot"}\n',+ )+ DatasetRevision.objects.for_team(self.team.id, canonical=True).filter(id=first["dataset_revision_id"]).update(+ expired_at=now()+ )+ for url, query in [+ (self.items_url, {"dataset": dataset["id"], "revision": 1}),+ (f"{self.items_url}{first['id']}/", {"revision": 1}),+ (f"{self.datasets_url}{dataset['id']}/exports/{asset.id}/content/", {}),+ ]:+ self.assertEqual(self.client.get(url, query).status_code, status.HTTP_404_NOT_FOUND)+ revisions = self.client.get(f"{self.datasets_url}{dataset['id']}/revisions/")+ self.assertEqual([revision["revision"] for revision in revisions.data["results"]], [2])+ Dataset.objects.for_team(self.team.id, canonical=True).filter(id=dataset["id"]).update(expired_at=now())+ self.assertEqual(self.client.get(f"{self.datasets_url}{dataset['id']}/").status_code, status.HTTP_404_NOT_FOUND)+ self.assertEqual(+ self.client.get(self.items_url, {"dataset": dataset["id"]}).status_code, status.HTTP_404_NOT_FOUND+ )+ self.assertEqual(self.client.get(f"{self.items_url}{first['id']}/").status_code, status.HTTP_404_NOT_FOUND)++ def test_dataset_detail_renews_usage_and_hides_expiring_soon(self) -> None:+ dataset = self._create_dataset()+ Dataset.objects.for_team(self.team.id, canonical=True).filter(id=dataset["id"]).update(+ last_used_at=now() - timedelta(days=88)+ )+ response = self.client.get(f"{self.datasets_url}{dataset['id']}/")+ self.assertEqual(response.status_code, status.HTTP_200_OK)+ self.assertFalse(response.data["expiring_soon"])+@patch("products.ai_observability.backend.api.datasets.report_user_action")def test_dataset_lifecycle_and_name_uniqueness(self, report_action: Mock) -> None:with self.captureOnCommitCallbacks(execute=True):diff --git a/products/ai_observability/backend/dataset_export.py b/products/ai_observability/backend/dataset_export.pyindex f10d3d8e4c83..a6cbe0a50864 100644--- a/products/ai_observability/backend/dataset_export.py+++ b/products/ai_observability/backend/dataset_export.py@@ -14,6 +14,7 @@ from posthog.permissions import posthog_feature_flag_valuefrom products.access_control.backend.facade.user_access_control import UserAccessControlfrom products.ai_observability.backend.dataset_limits import MAX_DATASET_EXPORT_BYTES, MAX_DATASET_EXPORT_MEGABYTESfrom products.ai_observability.backend.dataset_queries import dataset_item_versions_at_revision+from products.ai_observability.backend.dataset_retention import DatasetUsagefrom products.ai_observability.backend.models.datasets import Dataset, DatasetItemVersion, DatasetRevisionfrom products.exports.backend.facade.api import (ExportedAsset,@@ -75,6 +76,16 @@ def _iter_versions_in_batches(last_item_id = last_version.dataset_item_id+def _read_export_revision(dataset: Dataset, revision: int) -> DatasetRevision:+ try:+ selected_revision = DatasetUsage.read_revision(dataset, revision)+ except (Dataset.DoesNotExist, DatasetRevision.DoesNotExist) as error:+ raise DatasetExportError("The selected dataset revision is no longer available.") from error+ if selected_revision is None:+ raise DatasetExportError("The selected dataset revision is no longer available.")+ return selected_revision++def export_dataset_jsonl(asset: ExportedAsset) -> None:context = asset.export_context or {}try:@@ -86,7 +97,9 @@ def export_dataset_jsonl(asset: ExportedAsset) -> None:if not asset.is_dataset_export or revision < 1 or asset.created_by is None:raise DatasetExportError("The dataset export configuration is invalid.", reportable=True)- dataset = Dataset.objects.for_team(asset.team_id, canonical=True).filter(id=dataset_id).first()+ dataset = (+ Dataset.objects.for_team(asset.team_id, canonical=True).filter(id=dataset_id, expired_at__isnull=True).first()+ )if dataset is None:raise DatasetExportError("The dataset is no longer available.")if not UserAccessControl(user=asset.created_by, team=asset.team).check_access_level_for_object(dataset, "viewer"):@@ -101,16 +114,7 @@ def export_dataset_jsonl(asset: ExportedAsset) -> None:raise RetryableExportError("Couldn't verify whether dataset exports are available. Try again.")if not feature_flag_value:raise DatasetExportError("Dataset exports are not available for this project.")- selected_revision = (- DatasetRevision.objects.for_team(asset.team_id, canonical=True)- .filter(- dataset_id=dataset.id,- revision=revision,- )- .first()- )- if selected_revision is None:- raise DatasetExportError("The selected dataset revision is no longer available.")+ selected_revision = _read_export_revision(dataset, revision)versions = (dataset_item_versions_at_revision(diff --git a/products/ai_observability/backend/dataset_queries.py b/products/ai_observability/backend/dataset_queries.pyindex 902f8a77078e..ac1032089a55 100644--- a/products/ai_observability/backend/dataset_queries.py+++ b/products/ai_observability/backend/dataset_queries.py@@ -20,7 +20,7 @@ def consistent_dataset_item_versions(def latest_dataset_revision(*, team_id: int, dataset_id: UUID) -> DatasetRevision | None:return (DatasetRevision.objects.for_team(team_id, canonical=True)- .filter(dataset_id=dataset_id)+ .filter(dataset_id=dataset_id, expired_at__isnull=True, dataset__expired_at__isnull=True).order_by("-revision").first())diff --git a/products/ai_observability/backend/dataset_retention.py b/products/ai_observability/backend/dataset_retention.pynew file mode 100644index 000000000000..0f42ec1b41e1--- /dev/null+++ b/products/ai_observability/backend/dataset_retention.py@@ -0,0 +1,379 @@+import time+from collections.abc import Sequence+from datetime import datetime, timedelta+from functools import partial+from uuid import UUID++from django.db import transaction+from django.db.models import Exists, F, OuterRef, Q, QuerySet, Subquery+from django.db.models.functions import Coalesce+from django.utils import timezone++import structlog+from celery.exceptions import SoftTimeLimitExceeded++from posthog.models.instance_setting import get_instance_settings++from products.ai_observability.backend.models.datasets import Dataset, DatasetItem, DatasetItemVersion, DatasetRevision+from products.ai_observability.backend.models.offline_evaluations import OfflineExperiment, OfflineExperimentItem++logger = structlog.get_logger(__name__)++ACTIVE_RETENTION = timedelta(days=90)+HISTORY_RETENTION = timedelta(days=30)+USAGE_REFRESH_INTERVAL = timedelta(hours=12)+EXPIRING_SOON_WINDOW = timedelta(days=7)+BATCH_SIZE = 500+MAX_DATASETS_PER_RUN = 100+TIME_BUDGET_SECONDS = 180+RETENTION_ENABLED_SETTING = "AI_OBSERVABILITY_DATASET_RETENTION_ENABLED"+++def dataset_retention_enabled() -> bool:+ return get_instance_settings([RETENTION_ENABLED_SETTING])[RETENTION_ENABLED_SETTING] is True+++class DatasetRetentionPolicy:+ @staticmethod+ def dataset_expires_at(dataset: Dataset) -> datetime | None:+ if dataset.last_used_at is None:+ return None+ return (+ dataset.last_used_at+ + (HISTORY_RETENTION if dataset.archived else ACTIVE_RETENTION)+ + USAGE_REFRESH_INTERVAL+ )++ @staticmethod+ def revision_expires_at(revision: DatasetRevision) -> datetime | None:+ dataset = revision.dataset+ if dataset.current_revision_id == revision.id:+ return DatasetRetentionPolicy.dataset_expires_at(dataset)+ if revision.last_used_at is None:+ return None+ expires_at = revision.last_used_at + HISTORY_RETENTION + USAGE_REFRESH_INTERVAL+ dataset_expires_at = DatasetRetentionPolicy.dataset_expires_at(dataset)+ return min(expires_at, dataset_expires_at) if dataset_expires_at else expires_at++ @staticmethod+ def expiring_soon(expires_at: datetime | None) -> bool:+ return expires_at is not None and expires_at <= timezone.now() + EXPIRING_SOON_WINDOW+++class DatasetUsage:+ @staticmethod+ def record(dataset: Dataset, revision: DatasetRevision | None = None) -> None:+ if dataset.expired_at is not None:+ raise Dataset.DoesNotExist+ if revision is not None and (revision.expired_at is not None or revision.dataset_id != dataset.id):+ raise DatasetRevision.DoesNotExist+ now = timezone.now()+ cutoff = now - USAGE_REFRESH_INTERVAL+ if dataset.last_used_at is not None and dataset.last_used_at > cutoff:+ if revision is None or (revision.last_used_at is not None and revision.last_used_at > cutoff):+ return+ # Usage and cleanup serialize on the dataset, including accesses to its older revisions.+ with transaction.atomic():+ locked = (+ Dataset.objects.for_team(dataset.team_id, canonical=True)+ .select_for_update()+ .get(id=dataset.id, expired_at__isnull=True)+ )+ if revision is not None:+ revisions = DatasetRevision.objects.for_team(dataset.team_id, canonical=True).filter(+ id=revision.id, dataset_id=dataset.id, expired_at__isnull=True+ )+ if not revisions.exists():+ raise DatasetRevision.DoesNotExist+ if revisions.filter(Q(last_used_at__isnull=True) | Q(last_used_at__lte=cutoff)).update(+ last_used_at=now+ ):+ revision.last_used_at = now+ else:+ revision.last_used_at = revisions.values_list("last_used_at", flat=True).get()+ if locked.last_used_at is None or locked.last_used_at <= cutoff:+ Dataset.objects.for_team(dataset.team_id, canonical=True).filter(id=dataset.id).update(last_used_at=now)+ locked.last_used_at = now+ dataset.last_used_at = locked.last_used_at++ @staticmethod+ def record_versions(dataset: Dataset, versions: Sequence[DatasetItemVersion]) -> None:+ now = timezone.now()+ cutoff = now - USAGE_REFRESH_INTERVAL+ stale_ids = [v.id for v in versions if v.last_used_at is None or v.last_used_at <= cutoff]+ DatasetUsage.record(dataset)+ if not stale_ids:+ return+ with transaction.atomic():+ Dataset.objects.for_team(dataset.team_id, canonical=True).select_for_update().get(+ id=dataset.id, expired_at__isnull=True+ )+ queryset = DatasetItemVersion.objects.for_team(dataset.team_id, canonical=True).filter(+ dataset_id=dataset.id, id__in=stale_ids+ )+ if queryset.count() != len(stale_ids):+ raise DatasetItemVersion.DoesNotExist+ queryset.filter(Q(last_used_at__isnull=True) | Q(last_used_at__lte=cutoff)).update(last_used_at=now)++ @staticmethod+ def read_revision(dataset: Dataset, number: int | None = None) -> DatasetRevision | None:+ revisions = DatasetRevision.objects.for_team(dataset.team_id, canonical=True).filter(+ dataset_id=dataset.id, expired_at__isnull=True+ )+ revision: DatasetRevision | None+ if number is not None:+ revision = revisions.get(revision=number)+ elif dataset.current_revision_id is not None:+ revision = revisions.get(id=dataset.current_revision_id)+ else:+ revision = revisions.order_by("-revision").first()+ DatasetUsage.record(dataset, revision)+ return revision+++class DatasetRetention:+ def __init__(self) -> None:+ self.datasets_expired = 0+ self.revisions_expired = 0+ self.versions_deleted = 0+ self.datasets_deleted = 0+ self.datasets_failed = 0+ self.budget_exhausted = False+ self.disabled = False++ @staticmethod+ def _uploading_revisions(team_id: int) -> QuerySet[OfflineExperiment]:+ cutoff = timezone.now() - HISTORY_RETENTION - USAGE_REFRESH_INTERVAL+ return (+ OfflineExperiment.objects.for_team(team_id)+ .filter(status=OfflineExperiment.Status.UPLOADING)+ .filter(Q(dataset_revision__last_used_at__isnull=True) | Q(dataset_revision__last_used_at__gt=cutoff))+ )++ @staticmethod+ def _unused_versions(+ versions: QuerySet[DatasetItemVersion], revisions: QuerySet[DatasetRevision], cutoff: datetime+ ) -> QuerySet[DatasetItemVersion]:+ next_revision = (+ versions.filter(dataset_item_id=OuterRef("dataset_item_id"), version__gt=OuterRef("version"))+ .order_by("version")+ .values("dataset_revision__revision")[:1]+ )+ retained_revision = revisions.filter(+ dataset_id=OuterRef("dataset_id"),+ expired_at__isnull=True,+ revision__gte=OuterRef("dataset_revision__revision"),+ revision__lt=OuterRef("next_revision_number"),+ )+ # The next version bounds the snapshots that need this one, including archive markers.+ return (+ versions.filter(last_used_at__lte=cutoff)+ .exclude(id=F("dataset_item__current_version_id"))+ .annotate(next_revision_number=Coalesce(Subquery(next_revision), 2_147_483_647))+ .filter(~Exists(retained_revision))+ )++ def _delete_versions(self, dataset: Dataset, ids: list[UUID]) -> None:+ references = OfflineExperimentItem.objects.for_team(dataset.team_id).filter(dataset_item_version_id__in=ids)+ reference_ids = list(references.order_by("id").values_list("id", flat=True)[:BATCH_SIZE])+ references.filter(id__in=reference_ids).update(dataset_item_version=None)+ versions = DatasetItemVersion.objects.for_team(dataset.team_id, canonical=True).filter(+ dataset_id=dataset.id, id__in=ids+ )+ removable_ids = list(+ versions.filter(~Exists(references.filter(dataset_item_version_id=OuterRef("id")))).values_list(+ "id", flat=True+ )+ )+ if dataset.expired_at is not None:+ DatasetItem.objects.for_team(dataset.team_id, canonical=True).filter(+ dataset_id=dataset.id, current_version_id__in=removable_ids+ ).update(current_version=None)+ versions.filter(id__in=removable_ids).only("id").delete()+ self.versions_deleted += len(removable_ids)++ @staticmethod+ def _delete_revisions(dataset: Dataset, revisions: QuerySet[DatasetRevision], ids: list[UUID]) -> None:+ references = OfflineExperiment.objects.for_team(dataset.team_id).filter(dataset_revision_id__in=ids)+ reference_ids = list(references.order_by("id").values_list("id", flat=True)[:BATCH_SIZE])+ references.filter(id__in=reference_ids).update(dataset_revision=None)+ removable_ids = list(+ revisions.filter(id__in=ids)+ .filter(~Exists(references.filter(dataset_revision_id=OuterRef("id"))))+ .values_list("id", flat=True)+ )+ Dataset.objects.for_team(dataset.team_id, canonical=True).filter(+ id=dataset.id, current_revision_id__in=removable_ids+ ).update(current_revision=None)+ revisions.filter(id__in=removable_ids).delete()++ def _delete_expired_dataset_batch(self, dataset: Dataset) -> None:+ versions = DatasetItemVersion.objects.for_team(dataset.team_id, canonical=True).filter(dataset_id=dataset.id)+ ids = list(versions.order_by("id").values_list("id", flat=True)[:BATCH_SIZE])+ if ids:+ self._delete_versions(dataset, ids)+ return+ revisions = DatasetRevision.objects.for_team(dataset.team_id, canonical=True).filter(dataset_id=dataset.id)+ revision_ids = list(revisions.order_by("id").values_list("id", flat=True)[:BATCH_SIZE])+ if revision_ids:+ self._delete_revisions(dataset, revisions, revision_ids)+ return+ dataset.delete()+ self.datasets_deleted += 1++ def _clean_dataset_contents(self, dataset: Dataset) -> None:+ team_id = dataset.team_id+ now = timezone.now()+ uploading = self._uploading_revisions(team_id).filter(dataset_revision__dataset_id=dataset.id)+ if dataset.expired_at is not None:+ self._delete_expired_dataset_batch(dataset)+ return+ if dataset.last_used_at is None:+ Dataset.objects.for_team(team_id, canonical=True).filter(id=dataset.id).update(last_used_at=now)+ return+ expires_at = DatasetRetentionPolicy.dataset_expires_at(dataset)+ if expires_at is not None and expires_at <= now and not uploading.exists():+ dataset.expired_at = now+ dataset.save(update_fields=["expired_at"])+ self._delete_expired_dataset_batch(dataset)+ self.datasets_expired += 1+ return+ cutoff = now - HISTORY_RETENTION - USAGE_REFRESH_INTERVAL+ revisions = DatasetRevision.objects.for_team(team_id, canonical=True).filter(dataset_id=dataset.id)+ current_revision_id = dataset.current_revision_id+ if current_revision_id is None:+ current_revision_id = revisions.order_by("-revision").values_list("id", flat=True).first()+ uninitialized_revisions = list(+ revisions.filter(last_used_at__isnull=True).values_list("id", flat=True)[:BATCH_SIZE]+ )+ revisions.filter(id__in=uninitialized_revisions).update(last_used_at=now)+ versions = DatasetItemVersion.objects.for_team(team_id, canonical=True).filter(dataset_id=dataset.id)+ uninitialized_versions = list(+ versions.filter(last_used_at__isnull=True).values_list("id", flat=True)[:BATCH_SIZE]+ )+ versions.filter(id__in=uninitialized_versions).update(last_used_at=now)+ eligible_revisions = revisions.filter(expired_at__isnull=True, last_used_at__lte=cutoff)+ if current_revision_id is not None:+ eligible_revisions = eligible_revisions.exclude(id=current_revision_id)+ revision_ids = list(+ eligible_revisions.exclude(id__in=uploading.values("dataset_revision_id"))+ .order_by("last_used_at", "id")+ .values_list("id", flat=True)[:BATCH_SIZE]+ )+ expired_count = revisions.filter(id__in=revision_ids).update(expired_at=now)+ ids = list(+ self._unused_versions(versions, revisions, cutoff).order_by("id").values_list("id", flat=True)[:BATCH_SIZE]+ )+ self._delete_versions(dataset, ids)+ # Keep revision identities while a surviving version or an evaluation still references them.+ removable_revisions = revisions.filter(expired_at__isnull=False).filter(+ ~Exists(+ DatasetItemVersion.objects.for_team(team_id, canonical=True).filter(dataset_revision_id=OuterRef("id"))+ ),+ ~Exists(OfflineExperiment.objects.for_team(team_id).filter(dataset_revision_id=OuterRef("id"))),+ )+ empty_ids = list(removable_revisions.order_by("id").values_list("id", flat=True)[:BATCH_SIZE])+ revisions.filter(id__in=empty_ids).delete()+ self.revisions_expired += expired_count++ def _clean_dataset(self, team_id: int, dataset_id: UUID) -> None:+ failure: Exception | None = None+ before = (self.datasets_expired, self.revisions_expired, self.versions_deleted, self.datasets_deleted)+ with transaction.atomic():+ dataset = (+ Dataset.objects.for_team(team_id, canonical=True)+ .select_for_update(skip_locked=True)+ .filter(id=dataset_id)+ .first()+ )+ if dataset is None:+ return+ Dataset.objects.for_team(team_id, canonical=True).filter(id=dataset.id).update(+ retention_checked_at=timezone.now()+ )+ # A savepoint rolls back deletion without losing the attempt timestamp that rotates candidates.+ try:+ with transaction.atomic():+ self._clean_dataset_contents(dataset)+ counts = {+ "datasets_expired": self.datasets_expired - before[0],+ "revisions_expired": self.revisions_expired - before[1],+ "versions_deleted": self.versions_deleted - before[2],+ "datasets_deleted": self.datasets_deleted - before[3],+ }+ if any(counts.values()):+ transaction.on_commit(+ partial(+ logger.info,+ "dataset_retention_dataset_cleaned",+ team_id=team_id,+ dataset_id=str(dataset_id),+ **counts,+ )+ )+ except Exception as error:+ failure = error+ if failure is not None:+ raise failure++ def run(self) -> None:+ if not dataset_retention_enabled():+ self.disabled = True+ return+ deadline = time.monotonic() + TIME_BUDGET_SECONDS+ now = timezone.now()+ history_cutoff = now - HISTORY_RETENTION - USAGE_REFRESH_INTERVAL+ active_cutoff = now - ACTIVE_RETENTION - USAGE_REFRESH_INTERVAL+ old_revisions = (+ DatasetRevision.objects.unscoped()+ .filter(expired_at__isnull=True)+ .filter(Q(last_used_at__isnull=True) | Q(last_used_at__lte=history_cutoff))+ .exclude(id=F("dataset__current_revision_id"))+ )+ unused_versions = self._unused_versions(+ DatasetItemVersion.objects.unscoped(), DatasetRevision.objects.unscoped(), history_cutoff+ )+ empty_revisions = (+ DatasetRevision.objects.unscoped()+ .filter(expired_at__isnull=False)+ .filter(+ ~Exists(DatasetItemVersion.objects.unscoped().filter(dataset_revision_id=OuterRef("id"))),+ ~Exists(OfflineExperiment.objects.unscoped().filter(dataset_revision_id=OuterRef("id"))),+ )+ )+ candidates = (+ Dataset.objects.unscoped()+ .filter(+ Q(expired_at__isnull=False)+ | Q(last_used_at__isnull=True)+ | Q(archived=True, last_used_at__lte=history_cutoff)+ | Q(archived=False, last_used_at__lte=active_cutoff)+ | Q(Exists(old_revisions.filter(dataset_id=OuterRef("id"))))+ | Q(Exists(unused_versions.filter(dataset_id=OuterRef("id"))))+ | Q(Exists(empty_revisions.filter(dataset_id=OuterRef("id"))))+ )+ .order_by(F("retention_checked_at").asc(nulls_first=True), "id")+ .values_list("id", "team_id")[:MAX_DATASETS_PER_RUN]+ )+ for dataset_id, team_id in candidates:+ if not dataset_retention_enabled():+ self.disabled = True+ break+ if time.monotonic() >= deadline:+ self.budget_exhausted = True+ break+ datasets_expired = self.datasets_expired+ revisions_expired = self.revisions_expired+ versions_deleted = self.versions_deleted+ datasets_deleted = self.datasets_deleted+ try:+ self._clean_dataset(team_id, dataset_id)+ except Exception as error:+ self.datasets_expired = datasets_expired+ self.revisions_expired = revisions_expired+ self.versions_deleted = versions_deleted+ self.datasets_deleted = datasets_deleted+ self.datasets_failed += 1+ if isinstance(error, SoftTimeLimitExceeded):+ raise+ logger.exception("dataset_retention_dataset_failed", team_id=team_id, dataset_id=str(dataset_id))diff --git a/products/ai_observability/backend/dataset_service.py b/products/ai_observability/backend/dataset_service.pyindex a3f06e7cba43..26b7a0515d11 100644--- a/products/ai_observability/backend/dataset_service.py+++ b/products/ai_observability/backend/dataset_service.py@@ -8,6 +8,7 @@ from uuid import UUIDfrom django.db import IntegrityError, connection, transactionfrom django.db.models import Max+from django.utils import timezoneimport posthoganalytics@@ -17,6 +18,7 @@ from posthog.models import Team, Userfrom products.ai_observability.backend.dataset_limits import MAX_DATASET_ITEM_PAYLOAD_BYTES, MAX_ITEMS_PER_DATASETfrom products.ai_observability.backend.dataset_queries import latest_dataset_revision+from products.ai_observability.backend.dataset_retention import DatasetUsagefrom products.ai_observability.backend.models.datasets import Dataset, DatasetItem, DatasetItemVersion, DatasetRevisiontype JSONValue = None | bool | int | float | str | list[JSONValue] | dict[str, JSONValue]@@ -257,7 +259,7 @@ def _lock_dataset(*, team_id: int, dataset_id: UUID) -> Dataset:Dataset.objects.for_team(team_id, canonical=True).select_for_update(of=("self",)).select_related("current_revision")- .get(id=dataset_id)+ .get(id=dataset_id, expired_at__isnull=True))current_revision = dataset.current_revisionif current_revision is None:@@ -330,6 +332,10 @@ def _check_base_version(*, current_version: DatasetItemVersion, base_version: indef _create_revision(*, dataset: Dataset, created_by: User | None) -> DatasetRevision:+ if dataset.current_revision_id is not None:+ DatasetRevision.objects.for_team(dataset.team_id, canonical=True).filter(id=dataset.current_revision_id).update(+ last_used_at=timezone.now()+ )latest_revision_number = (DatasetRevision.objects.for_team(dataset.team_id, canonical=True).filter(dataset_id=dataset.id)@@ -404,7 +410,8 @@ def _create_item_version(item.current_version = versionitem.save(update_fields=["current_version", "updated_at"])dataset.current_revision = revision- dataset.save(update_fields=["current_revision", "updated_at"])+ dataset.last_used_at = timezone.now()+ dataset.save(update_fields=["current_revision", "updated_at", "last_used_at"])return version@@ -459,7 +466,7 @@ def create_dataset(normalized_name = _validate_dataset_fields(name=name, description=description, metadata=normalized_metadata)with connection.cursor() as cursor:cursor.execute("SELECT pg_advisory_xact_lock(hashtextextended(%s, 0))", [f"aio-dataset-allocation:{team.id}"])- current_count = Dataset.objects.for_team(team.id, canonical=True).count()+ current_count = Dataset.objects.for_team(team.id, canonical=True).filter(expired_at__isnull=True).count()if current_count >= max_datasets:raise DatasetLimitExceeded(resource="datasets",@@ -511,7 +518,8 @@ def update_dataset()changed_fields: list[str] = []- update_fields = ["updated_at"]+ dataset.last_used_at = timezone.now()+ update_fields = ["updated_at", "last_used_at"]if not isinstance(name, _Unset):if dataset.name != normalized_name:changed_fields.append("name")@@ -543,7 +551,8 @@ def archive_dataset(*, team_id: int, dataset_id: UUID) -> DatasetMutationResult:if dataset.archived:return DatasetMutationResult(dataset=dataset)dataset.archived = True- dataset.save(update_fields=["archived", "updated_at"])+ dataset.last_used_at = timezone.now()+ dataset.save(update_fields=["archived", "updated_at", "last_used_at"])return DatasetMutationResult(dataset=dataset, changed_fields=("archived",))@@ -553,7 +562,8 @@ def restore_dataset(*, team_id: int, dataset_id: UUID) -> DatasetMutationResult:if not dataset.archived:return DatasetMutationResult(dataset=dataset)dataset.archived = False- dataset.save(update_fields=["archived", "updated_at"])+ dataset.last_used_at = timezone.now()+ dataset.save(update_fields=["archived", "updated_at", "last_used_at"])return DatasetMutationResult(dataset=dataset, changed_fields=("archived",))@@ -609,6 +619,7 @@ def create_dataset_item(current_item_id=existing_item.id,)if _item_contents_equal(_DatasetItemContent.from_version(current_version), content):+ DatasetUsage.record(dataset, dataset.current_revision)return DatasetItemMutationResult(item=existing_item,version=current_version,@@ -689,6 +700,7 @@ def update_dataset_item(_validate_item_content(content=next_content)if _item_contents_equal(current_content, next_content):+ DatasetUsage.record(dataset, dataset.current_revision)return DatasetItemMutationResult(item=item,version=current_version,diff --git a/products/ai_observability/backend/facade/admin.py b/products/ai_observability/backend/facade/admin.pynew file mode 100644index 000000000000..4dc9eaaf3309--- /dev/null+++ b/products/ai_observability/backend/facade/admin.py@@ -0,0 +1,2 @@+from products.ai_observability.backend.admin import set_dataset_retention_enabled as set_dataset_retention_enabled+from products.ai_observability.backend.dataset_retention import dataset_retention_enabled as dataset_retention_enableddiff --git a/products/ai_observability/backend/facade/tasks.py b/products/ai_observability/backend/facade/tasks.pyindex 109cd1364bcb..bd9fcbac8197 100644--- a/products/ai_observability/backend/facade/tasks.py+++ b/products/ai_observability/backend/facade/tasks.py@@ -1,3 +1,4 @@+from products.ai_observability.backend.tasks.dataset_retention import expire_datasetsfrom products.ai_observability.backend.tasks.offline_payload_retention import expire_offline_evaluation_payloads-__all__ = ["expire_offline_evaluation_payloads"]+__all__ = ["expire_datasets", "expire_offline_evaluation_payloads"]diff --git a/products/ai_observability/backend/migrations/0071_dataset_retention.py b/products/ai_observability/backend/migrations/0071_dataset_retention.pynew file mode 100644index 000000000000..6eb37326b7d3--- /dev/null+++ b/products/ai_observability/backend/migrations/0071_dataset_retention.py@@ -0,0 +1,43 @@+# Generated by Django 6.1.2 on 2026-11-03 15:08++import django.utils.timezone+from django.db import migrations, models+++class Migration(migrations.Migration):+ dependencies = [+ ("ai_observability", "0070_offline_payload_expiry_indexes"),+ ]++ operations = [+ migrations.AddField(+ model_name="dataset",+ name="expired_at",+ field=models.DateTimeField(blank=True, null=True),+ ),+ migrations.AddField(+ model_name="dataset",+ name="last_used_at",+ field=models.DateTimeField(default=django.utils.timezone.now, null=True),+ ),+ migrations.AddField(+ model_name="dataset",+ name="retention_checked_at",+ field=models.DateTimeField(blank=True, null=True),+ ),+ migrations.AddField(+ model_name="datasetitemversion",+ name="last_used_at",+ field=models.DateTimeField(default=django.utils.timezone.now, null=True),+ ),+ migrations.AddField(+ model_name="datasetrevision",+ name="expired_at",+ field=models.DateTimeField(blank=True, null=True),+ ),+ migrations.AddField(+ model_name="datasetrevision",+ name="last_used_at",+ field=models.DateTimeField(default=django.utils.timezone.now, null=True),+ ),+ ]diff --git a/products/ai_observability/backend/migrations/0072_dataset_retention_indexes.py b/products/ai_observability/backend/migrations/0072_dataset_retention_indexes.pynew file mode 100644index 000000000000..d342b6e39924--- /dev/null+++ b/products/ai_observability/backend/migrations/0072_dataset_retention_indexes.py@@ -0,0 +1,24 @@+from django.db import migrations, models++from posthog.migration_helpers import SafeAddIndexConcurrently+++class Migration(migrations.Migration):+ atomic = False++ dependencies = [("ai_observability", "0071_dataset_retention")]++ operations = [+ SafeAddIndexConcurrently(+ model_name="dataset",+ index=models.Index(fields=["last_used_at", "id"], name="llma_dataset_retention_idx"),+ ),+ SafeAddIndexConcurrently(+ model_name="dataset",+ index=models.Index(fields=["retention_checked_at", "id"], name="llma_dataset_retention_chk_idx"),+ ),+ SafeAddIndexConcurrently(+ model_name="datasetrevision",+ index=models.Index(fields=["last_used_at", "id"], name="llma_revision_retention_idx"),+ ),+ ]diff --git a/products/ai_observability/backend/migrations/0073_dataset_live_name_uniqueness.py b/products/ai_observability/backend/migrations/0073_dataset_live_name_uniqueness.pynew file mode 100644index 000000000000..4ac331d047e5--- /dev/null+++ b/products/ai_observability/backend/migrations/0073_dataset_live_name_uniqueness.py@@ -0,0 +1,37 @@+# Generated by Django 6.1.2 on 2026-11-03 16:17++from django.db import migrations, models++from posthog.migration_helpers import CreateIndexConcurrently+++class Migration(migrations.Migration):+ atomic = False++ dependencies = [+ ("ai_observability", "0072_dataset_retention_indexes"),+ ]++ operations = [+ migrations.SeparateDatabaseAndState(+ database_operations=[+ CreateIndexConcurrently(+ index_name="uniq_llma_dataset_v2_live_name",+ table_name="llm_analytics_dataset_v2",+ columns="(team_id, name)",+ unique=True,+ where="WHERE expired_at IS NULL",+ ),+ ],+ state_operations=[+ migrations.AddConstraint(+ model_name="dataset",+ constraint=models.UniqueConstraint(+ condition=models.Q(expired_at__isnull=True),+ fields=("team", "name"),+ name="uniq_llma_dataset_v2_live_name",+ ),+ ),+ ],+ ),+ ]diff --git a/products/ai_observability/backend/migrations/0074_release_expired_dataset_names.py b/products/ai_observability/backend/migrations/0074_release_expired_dataset_names.pynew file mode 100644index 000000000000..818557666c7e--- /dev/null+++ b/products/ai_observability/backend/migrations/0074_release_expired_dataset_names.py@@ -0,0 +1,12 @@+from django.db import migrations+++class Migration(migrations.Migration):+ dependencies = [("ai_observability", "0073_dataset_live_name_uniqueness")]++ operations = [+ migrations.RemoveConstraint(+ model_name="dataset",+ name="uniq_llma_dataset_v2_team_name",+ ),+ ]diff --git a/products/ai_observability/backend/migrations/max_migration.txt b/products/ai_observability/backend/migrations/max_migration.txtindex 0317898d12ac..8ea568e02f5e 100644--- a/products/ai_observability/backend/migrations/max_migration.txt+++ b/products/ai_observability/backend/migrations/max_migration.txt@@ -1 +1 @@-0070_offline_payload_expiry_indexes+0074_release_expired_dataset_namesdiff --git a/products/ai_observability/backend/models/datasets.py b/products/ai_observability/backend/models/datasets.pyindex 36d31484de09..179e0b948d66 100644--- a/products/ai_observability/backend/models/datasets.py+++ b/products/ai_observability/backend/models/datasets.py@@ -5,6 +5,7 @@ from django.db import modelsfrom django.db.models import F, Func, Q, Valuefrom django.db.models.expressions import NegatedExpressionfrom django.db.models.lookups import Exact+from django.utils import timezonefrom posthog.models.scoping.manager import TeamScopedManagerfrom posthog.models.tagged_items_relation import Taggable@@ -34,6 +35,9 @@ class Dataset(Taggable, UUIDModel, CreatedMetaFields, UpdatedMetaFields):description = models.TextField(blank=True, default="")metadata = models.JSONField(blank=True, default=dict)archived = models.BooleanField(default=False)+ last_used_at = models.DateTimeField(default=timezone.now, null=True)+ expired_at = models.DateTimeField(null=True, blank=True)+ retention_checked_at = models.DateTimeField(null=True, blank=True)current_revision = models.ForeignKey("DatasetRevision",on_delete=models.RESTRICT,@@ -46,13 +50,19 @@ class Dataset(Taggable, UUIDModel, CreatedMetaFields, UpdatedMetaFields):db_table = "llm_analytics_dataset_v2"ordering = ["-created_at", "id"]constraints = [- models.UniqueConstraint(fields=["team", "name"], name="uniq_llma_dataset_v2_team_name"),+ models.UniqueConstraint(+ fields=["team", "name"],+ condition=Q(expired_at__isnull=True),+ name="uniq_llma_dataset_v2_live_name",+ ),models.CheckConstraint(condition=_json_type_is("metadata", "object"),name="llma_dataset_v2_metadata_object",),]indexes = [+ models.Index(fields=["last_used_at", "id"], name="llma_dataset_retention_idx"),+ models.Index(fields=["retention_checked_at", "id"], name="llma_dataset_retention_chk_idx"),models.Index(fields=["team", "archived", "-created_at", "id"],name="llma_dataset_v2_team_arch_idx",@@ -80,10 +90,13 @@ class DatasetRevision(UUIDModel, CreatedMetaFields):dataset = models.ForeignKey(Dataset, on_delete=models.CASCADE, related_name="revisions")revision = models.PositiveIntegerField()+ last_used_at = models.DateTimeField(default=timezone.now, null=True)+ expired_at = models.DateTimeField(null=True, blank=True)class Meta:db_table = "llm_analytics_datasetrevision_v2"ordering = ["-revision"]+ indexes = [models.Index(fields=["last_used_at", "id"], name="llma_revision_retention_idx")]constraints = [models.UniqueConstraint(fields=["dataset", "revision"],@@ -156,6 +169,7 @@ class DatasetItemVersion(UUIDModel, CreatedMetaFields):related_name="item_versions",)version = models.PositiveIntegerField()+ last_used_at = models.DateTimeField(default=timezone.now, null=True)archived = models.BooleanField(default=False)input = models.JSONField()expected_output = models.JSONField(null=True, blank=True)diff --git a/products/ai_observability/backend/offline_evaluation_service.py b/products/ai_observability/backend/offline_evaluation_service.pyindex 29f53c73f7f2..66f685e8b9a2 100644--- a/products/ai_observability/backend/offline_evaluation_service.py+++ b/products/ai_observability/backend/offline_evaluation_service.py@@ -17,6 +17,7 @@ from posthog.exceptions import QuotaLimitExceededfrom posthog.models.team import Teamfrom products.ai_observability.backend.dataset_queries import dataset_item_versions_at_revision+from products.ai_observability.backend.dataset_retention import DatasetUsagefrom products.ai_observability.backend.models.datasets import Dataset, DatasetItemVersion, DatasetRevisionfrom products.ai_observability.backend.models.offline_evaluations import (OfflineEvaluationResult,@@ -173,7 +174,11 @@ class _OfflineEvaluationService:def _dataset_revisions(self) -> QuerySet[DatasetRevision]:# Skip parent resolution because hosted dataset references belong to the submitted environment.- revisions = DatasetRevision.objects.for_team(self.team_id, canonical=True)+ revisions = (+ DatasetRevision.objects.for_team(self.team_id, canonical=True)+ .filter(expired_at__isnull=True, dataset__expired_at__isnull=True)+ .select_related("dataset")+ )if self._user_access_control is not None:datasets = self._user_access_control.filter_queryset_by_access_level(Dataset.objects.for_team(self.team_id, canonical=True), include_all_if_admin=True, resource="dataset"@@ -213,6 +218,12 @@ class OfflineExperimentService(_OfflineEvaluationService):errors.update(error.errors)if errors:raise OfflineEvaluationValidationError.from_errors(errors)+ try:+ DatasetUsage.record(revision.dataset, revision)+ except (Dataset.DoesNotExist, DatasetRevision.DoesNotExist) as error:+ raise OfflineEvaluationValidationError(+ "dataset_revision_id", "Select an existing dataset revision."+ ) from errorreturn fieldsdef create(self, submission: ExperimentSubmission, *, created_by_id: int | None = None) -> ExperimentReceipt:@@ -251,6 +262,10 @@ class OfflineExperimentService(_OfflineEvaluationService):return ExperimentReceipt(experiment=experiment, counts=counts)if experiment.status != OfflineExperiment.Status.UPLOADING:raise OfflineEvaluationConflict("experiment_closed", "This experiment is already closed.")+ if experiment.dataset_revision_id is not None:+ revision = self._dataset_revisions().filter(id=experiment.dataset_revision_id).first()+ if revision is not None:+ DatasetUsage.record(revision.dataset, revision)if status == "completed" and (counts.expected_item_count is not Noneand counts.expected_item_count != counts.accepted_item_count@@ -306,6 +321,7 @@ class OfflineEvaluationIngestionService(_OfflineEvaluationService):revision = self._dataset_revisions().filter(id=revision_id).first()if revision is None:raise OfflineEvaluationValidationError("items", "The experiment's dataset revision is no longer available.")+ DatasetUsage.record(revision.dataset, revision)return {version.id: versionfor version in dataset_item_versions_at_revision(@@ -506,6 +522,10 @@ class OfflineEvaluationIngestionService(_OfflineEvaluationService):results[ResultIdentity(item_id=result.item_id, scorer_version_id=result.scorer_version_id)] = rowif errors:raise OfflineEvaluationValidationError.from_errors(errors)+ if new_results and not new_item_submissions and experiment.dataset_revision_id is not None:+ revision = self._dataset_revisions().filter(id=experiment.dataset_revision_id).first()+ if revision is not None:+ DatasetUsage.record(revision.dataset, revision)OfflineExperimentItem.objects.for_team(self.team_id).bulk_create(new_items)new_item_ids = {item.id for item in new_items}OfflineExperimentItemPayload.objects.for_team(self.team_id).bulk_create(diff --git a/products/ai_observability/backend/tasks/__init__.py b/products/ai_observability/backend/tasks/__init__.pyindex b69cd3fdf549..b08fe61941ca 100644--- a/products/ai_observability/backend/tasks/__init__.py+++ b/products/ai_observability/backend/tasks/__init__.py@@ -1,3 +1,4 @@+from .dataset_retention import expire_datasetsfrom .offline_payload_retention import expire_offline_evaluation_payloads-__all__ = ["expire_offline_evaluation_payloads"]+__all__ = ["expire_datasets", "expire_offline_evaluation_payloads"]diff --git a/products/ai_observability/backend/tasks/dataset_retention.py b/products/ai_observability/backend/tasks/dataset_retention.pynew file mode 100644index 000000000000..106c1cee3861--- /dev/null+++ b/products/ai_observability/backend/tasks/dataset_retention.py@@ -0,0 +1,43 @@+import time+from uuid import uuid4++import structlog+from celery import shared_task++from posthog.ph_client import ph_scoped_capture+from posthog.scoping_audit import skip_team_scope_audit+from posthog.utils import get_instance_region++from products.ai_observability.backend.dataset_retention import DatasetRetention++logger = structlog.get_logger(__name__)+++@shared_task(ignore_result=True, soft_time_limit=240, time_limit=300)+@skip_team_scope_audit+def expire_datasets() -> None:+ sweep = DatasetRetention()+ started = time.monotonic()+ status = "failed"+ try:+ sweep.run()+ status = "disabled" if sweep.disabled else "completed_with_errors" if sweep.datasets_failed else "completed"+ finally:+ properties = {+ "run_id": str(uuid4()),+ "status": status,+ "datasets_expired": sweep.datasets_expired,+ "revisions_expired": sweep.revisions_expired,+ "versions_deleted": sweep.versions_deleted,+ "datasets_deleted": sweep.datasets_deleted,+ "datasets_failed": sweep.datasets_failed,+ "budget_exhausted": sweep.budget_exhausted,+ "duration_seconds": time.monotonic() - started,+ "$process_person_profile": False,+ }+ logger.info("dataset_retention_run", **properties)+ try:+ with ph_scoped_capture(region=get_instance_region() or "US") as capture:+ capture(distinct_id="dataset-retention", event="dataset retention run", properties=properties)+ except Exception:+ logger.exception("dataset_retention_capture_failed")diff --git a/products/ai_observability/backend/templates/admin/posthog/organization/ai_observability_dataset_retention.html b/products/ai_observability/backend/templates/admin/posthog/organization/ai_observability_dataset_retention.htmlnew file mode 100644index 000000000000..363b0d412195--- /dev/null+++ b/products/ai_observability/backend/templates/admin/posthog/organization/ai_observability_dataset_retention.html@@ -0,0 +1,24 @@+{% extends "admin/base_site.html" %}+{% block content %}+ <div id="content-main">+ <p>Dataset cleanup is currently <strong>{{ enabled|yesno:"enabled,disabled" }}</strong> in this region.</p>+ <p>+ Cleanup permanently deletes unused active datasets after 90 days and archived datasets and historical+ revisions after 30 days, with a 12-hour usage recording margin. Evaluation results and scores are preserved.+ </p>+ <p>+ Disabling cleanup stops new expiry and deletion, including deletion already in progress. A running sweep+ finishes its current dataset before stopping. Expired datasets remain unavailable, and deleted content is not restored.+ </p>+ <p>Enabling cleanup resumes the hourly schedule. Change this setting separately in each region.</p>+ <form method="post">+ {% csrf_token %}+ <fieldset class="module aligned">+ {{ form.as_div }}+ </fieldset>+ <div class="submit-row">+ <input type="submit" value="Save dataset cleanup setting" class="default">+ </div>+ </form>+ </div>+{% endblock %}diff --git a/products/ai_observability/backend/test/test_dataset_export.py b/products/ai_observability/backend/test/test_dataset_export.pyindex d578a2ce3237..944db5eafc3d 100644--- a/products/ai_observability/backend/test/test_dataset_export.py+++ b/products/ai_observability/backend/test/test_dataset_export.py@@ -1,8 +1,11 @@import json+import time_machinefrom posthog.test.base import APIBaseTestfrom unittest.mock import patch+from django.utils import timezone+from parameterized import parameterizedfrom posthog.storage import object_storage@@ -15,7 +18,7 @@ from products.ai_observability.backend.dataset_service import (create_dataset_item,update_dataset_item,)-from products.ai_observability.backend.models.datasets import Dataset+from products.ai_observability.backend.models.datasets import Dataset, DatasetRevisionfrom products.exports.backend.facade.api import DATASET_EXPORT_KIND, ExportedAsset, RetryableExportErrorfrom products.exports.backend.tasks.failure_handler import EXCEPTIONS_TO_RETRY@@ -204,6 +207,29 @@ class TestDatasetExport(APIBaseTest):asset.refresh_from_db()self.assertFalse(asset.has_content)+ @parameterized.expand([("expired", 1), ("missing", 2)])+ @time_machine.travel("2026-11-03T12:00:00Z", tick=False)+ @patch("products.ai_observability.backend.dataset_export.posthog_feature_flag_value", return_value=True)+ def test_export_rejects_unavailable_revisions(self, _name: str, revision: int, _feature_flag: object) -> None:+ dataset, _result = self._create_dataset_and_item()+ DatasetRevision.objects.for_team(self.team.id).filter(dataset_id=dataset.id).update(expired_at=timezone.now())+ asset = ExportedAsset.objects.create(+ team=self.team,+ created_by=self.user,+ export_format=ExportedAsset.ExportFormat.JSONL,+ export_context={+ "kind": DATASET_EXPORT_KIND,+ "dataset_id": str(dataset.id),+ "dataset_revision": revision,+ },+ )++ with self.assertRaisesRegex(DatasetExportError, "The selected dataset revision is no longer available"):+ export_dataset_jsonl(asset)++ asset.refresh_from_db()+ self.assertFalse(asset.has_content)+@patch("products.ai_observability.backend.dataset_export.posthog_feature_flag_value", return_value=None)def test_export_retries_when_feature_flag_evaluation_is_unavailable(self, _feature_flag: object) -> None:dataset, _result = self._create_dataset_and_item()diff --git a/products/ai_observability/backend/test/test_dataset_retention.py b/products/ai_observability/backend/test/test_dataset_retention.pynew file mode 100644index 000000000000..059fa38fd71b--- /dev/null+++ b/products/ai_observability/backend/test/test_dataset_retention.py@@ -0,0 +1,685 @@+from concurrent.futures import ThreadPoolExecutor+from datetime import timedelta+from uuid import UUID, uuid4++import time_machine+from unittest.mock import patch++from django.db import DatabaseError, IntegrityError, connection, transaction+from django.db.migrations.executor import MigrationExecutor+from django.test import TestCase, TransactionTestCase+from django.utils import timezone++from celery.exceptions import SoftTimeLimitExceeded+from parameterized import parameterized++from posthog.models import Organization, Team+from posthog.models.instance_setting import override_instance_config, set_instance_setting++from products.ai_observability.backend.dataset_queries import dataset_item_versions_at_revision+from products.ai_observability.backend.dataset_retention import (+ ACTIVE_RETENTION,+ HISTORY_RETENTION,+ RETENTION_ENABLED_SETTING,+ USAGE_REFRESH_INTERVAL,+ DatasetRetention,+ DatasetRetentionPolicy,+ DatasetUsage,+)+from products.ai_observability.backend.dataset_service import (+ DatasetItemMutationResult,+ DatasetMutationConflict,+ archive_dataset,+ archive_dataset_item,+ create_dataset,+ create_dataset_item,+ restore_dataset_item,+ update_dataset_item,+)+from products.ai_observability.backend.models.datasets import Dataset, DatasetItemVersion, DatasetRevision+from products.ai_observability.backend.models.offline_evaluations import (+ OfflineEvaluationResult,+ OfflineEvaluationUsageLedger,+ OfflineExperiment,+ OfflineExperimentItem,+)+from products.ai_observability.backend.models.score_definitions import ScoreDefinition+from products.ai_observability.backend.offline_evaluation_service import (+ OfflineEvaluationIngestionService,+ OfflineExperimentService,+)+from products.ai_observability.backend.offline_evaluation_types import (+ ExperimentSubmission,+ ItemSubmission,+ ResultSubmission,+ UploadSubmission,+)+++@time_machine.travel("2026-11-03T12:00:00Z", tick=False)+class TestDatasetRetention(TestCase):+ team: Team++ @classmethod+ def setUpTestData(cls) -> None:+ _, _, cls.team = Organization.objects.bootstrap(None)++ def setUp(self) -> None:+ self.dataset = Dataset.objects.for_team(self.team.id, canonical=True).create(+ team=self.team, name="Retention fixture"+ )+ self.old = timezone.now() - ACTIVE_RETENTION - USAGE_REFRESH_INTERVAL - timedelta(days=1)++ def create_item(self, value: str = "First input") -> DatasetItemMutationResult:+ result = create_dataset_item(team_id=self.team.id, dataset_id=self.dataset.id, created_by=None, input=value)+ self.dataset.refresh_from_db()+ return result++ def update_item(self, result: DatasetItemMutationResult, value: str = "Updated input") -> DatasetItemMutationResult:+ updated = update_dataset_item(+ team_id=self.team.id,+ dataset_id=self.dataset.id,+ item_id=result.item.id,+ created_by=None,+ base_version=result.version.version,+ input=value,+ )+ self.dataset.refresh_from_db()+ return updated++ def age_history(self) -> None:+ DatasetRevision.objects.for_team(self.team.id, canonical=True).filter(dataset=self.dataset).update(+ last_used_at=self.old+ )+ DatasetItemVersion.objects.for_team(self.team.id, canonical=True).filter(dataset=self.dataset).update(+ last_used_at=self.old+ )++ @parameterized.expand([(False, ACTIVE_RETENTION), (True, HISTORY_RETENTION)])+ def test_expiry_includes_the_usage_recording_margin(self, archived: bool, retention: timedelta) -> None:+ now = timezone.now()+ self.dataset.archived = archived+ self.dataset.last_used_at = now - retention+ self.dataset.save()+ DatasetRetention().run()+ self.assertTrue(Dataset.objects.for_team(self.team.id, canonical=True).filter(id=self.dataset.id).exists())+ self.assertEqual(DatasetRetentionPolicy.dataset_expires_at(self.dataset), now + USAGE_REFRESH_INTERVAL)++ self.dataset.last_used_at -= USAGE_REFRESH_INTERVAL + timedelta(seconds=1)+ self.dataset.save()+ DatasetRetention().run()+ self.assertFalse(Dataset.objects.for_team(self.team.id, canonical=True).filter(id=self.dataset.id).exists())++ def test_repeated_usage_needs_no_database_queries(self) -> None:+ item = self.create_item()+ with self.assertNumQueries(0):+ DatasetUsage.record(self.dataset, item.version.dataset_revision)+ self.age_history()+ self.dataset.last_used_at = self.old+ self.dataset.save()+ item.version.dataset_revision.refresh_from_db()+ DatasetUsage.record(self.dataset, item.version.dataset_revision)+ with self.assertNumQueries(0):+ DatasetUsage.record(self.dataset, item.version.dataset_revision)+ assert self.dataset.last_used_at is not None+ assert item.version.dataset_revision.last_used_at is not None+ self.assertGreater(self.dataset.last_used_at, self.old)+ self.assertGreater(item.version.dataset_revision.last_used_at, self.old)++ def test_shared_version_survives_expired_introducing_revision(self) -> None:+ first = self.create_item()+ second = self.create_item("Second item")+ self.update_item(first)+ self.age_history()+ second.version.dataset_revision.last_used_at = timezone.now()+ second.version.dataset_revision.save(update_fields=["last_used_at"])+ snapshot = dataset_item_versions_at_revision(+ team_id=self.team.id,+ dataset_id=self.dataset.id,+ revision=second.version.dataset_revision.revision,+ archived=False,+ )+ before = list(snapshot.values_list("id", flat=True))+ DatasetRetention().run()++ first.version.dataset_revision.refresh_from_db()+ self.assertIsNotNone(first.version.dataset_revision.expired_at)+ self.assertTrue(+ DatasetItemVersion.objects.for_team(self.team.id, canonical=True).filter(id=first.version.id).exists()+ )+ snapshot = dataset_item_versions_at_revision(+ team_id=self.team.id,+ dataset_id=self.dataset.id,+ revision=second.version.dataset_revision.revision,+ archived=False,+ )+ self.assertCountEqual(snapshot.values_list("id", flat=True), [first.version.id, second.version.id])+ self.assertCountEqual(snapshot.values_list("id", flat=True), before)+ with self.assertRaises(DatasetRevision.DoesNotExist):+ DatasetUsage.read_revision(self.dataset, first.version.dataset_revision.revision)++ def test_obsolete_versions_are_deleted_and_numbers_are_not_reused(self) -> None:+ first = self.create_item()+ second = self.update_item(first)+ self.age_history()+ sweep = DatasetRetention()+ sweep.run()+ self.assertEqual(sweep.versions_deleted, 1)+ self.assertFalse(+ DatasetItemVersion.objects.for_team(self.team.id, canonical=True).filter(id=first.version.id).exists()+ )+ third = self.update_item(second, "Third input")+ self.assertEqual(third.version.version, 3)+ self.assertEqual(third.version.dataset_revision.revision, 3)++ def test_archive_marker_prevents_resurrection_in_retained_snapshot(self) -> None:+ first = self.create_item()+ archived = archive_dataset_item(+ team_id=self.team.id, dataset_id=self.dataset.id, item_id=first.item.id, created_by=None, base_version=1+ )+ second = self.create_item("Second item")+ restore_dataset_item(+ team_id=self.team.id, dataset_id=self.dataset.id, item_id=first.item.id, created_by=None, base_version=2+ )+ self.dataset.refresh_from_db()+ self.age_history()+ first.version.dataset_revision.last_used_at = timezone.now()+ first.version.dataset_revision.save(update_fields=["last_used_at"])+ second.version.dataset_revision.last_used_at = timezone.now()+ second.version.dataset_revision.save(update_fields=["last_used_at"])+ before = list(+ dataset_item_versions_at_revision(+ team_id=self.team.id,+ dataset_id=self.dataset.id,+ revision=second.version.dataset_revision.revision,+ archived=False,+ ).values_list("id", flat=True)+ )+ DatasetRetention().run()+ self.assertTrue(+ DatasetItemVersion.objects.for_team(self.team.id, canonical=True).filter(id=archived.version.id).exists()+ )+ snapshot = dataset_item_versions_at_revision(+ team_id=self.team.id,+ dataset_id=self.dataset.id,+ revision=second.version.dataset_revision.revision,+ archived=False,+ )+ self.assertEqual(list(snapshot.values_list("id", flat=True)), [second.version.id])+ self.assertEqual(list(snapshot.values_list("id", flat=True)), before)++ def test_using_history_protects_only_the_requested_revision(self) -> None:+ first = self.create_item()+ second = self.update_item(first)+ self.update_item(second, "Third input")+ self.age_history()+ DatasetUsage.read_revision(self.dataset, first.version.dataset_revision.revision)+ DatasetRetention().run()+ self.assertTrue(+ DatasetItemVersion.objects.for_team(self.team.id, canonical=True).filter(id=first.version.id).exists()+ )+ self.assertFalse(+ DatasetItemVersion.objects.for_team(self.team.id, canonical=True).filter(id=second.version.id).exists()+ )++ def test_reading_item_history_preserves_its_payload(self) -> None:+ first = self.create_item()+ self.update_item(first)+ self.age_history()+ first.version.refresh_from_db()+ DatasetUsage.record_versions(self.dataset, [first.version])+ DatasetRetention().run()+ self.assertTrue(+ DatasetItemVersion.objects.for_team(self.team.id, canonical=True).filter(id=first.version.id).exists()+ )++ def test_archiving_and_superseding_start_a_full_history_period(self) -> None:+ first = self.create_item()+ self.age_history()+ self.update_item(first)+ first.version.dataset_revision.refresh_from_db()+ assert first.version.dataset_revision.last_used_at is not None+ self.assertGreater(first.version.dataset_revision.last_used_at, timezone.now() - timedelta(seconds=5))+ self.dataset.last_used_at = self.old+ self.dataset.save()+ archive_dataset(team_id=self.team.id, dataset_id=self.dataset.id)+ self.dataset.refresh_from_db()+ expires_at = DatasetRetentionPolicy.dataset_expires_at(self.dataset)+ assert expires_at is not None+ self.assertGreater(expires_at, timezone.now() + HISTORY_RETENTION)++ def test_missing_usage_gets_a_full_grace_period(self) -> None:+ self.dataset.last_used_at = None+ self.dataset.save()+ DatasetRetention().run()+ self.dataset.refresh_from_db()+ self.assertIsNotNone(self.dataset.last_used_at)+ self.assertIsNone(self.dataset.expired_at)++ def test_uploading_evaluation_pins_snapshot_then_completed_history_survives_cleanup(self) -> None:+ first = self.create_item()+ lifecycle = OfflineExperimentService(team_id=self.team.id, user_access_control=None)+ experiment = lifecycle.create(+ ExperimentSubmission(+ id=uuid4(),+ name="Retention run",+ started_at=timezone.now(),+ dataset_revision_id=first.version.dataset_revision_id,+ )+ ).experiment+ definition = ScoreDefinition.objects.create(team=self.team, name="Quality", kind="numeric")+ scorer = definition.create_new_version(config={}, created_by=None)+ item_id = uuid4()+ OfflineEvaluationIngestionService(team_id=self.team.id, user_access_control=None).upload(+ experiment.id,+ UploadSubmission(+ items=[ItemSubmission(id=item_id, dataset_item_version_id=first.version.id)],+ results=[ResultSubmission(item_id=item_id, scorer_version_id=scorer.id, status="ok", value=1.0)],+ ),+ )+ self.update_item(first)+ self.age_history()+ first.version.dataset_revision.last_used_at = timezone.now()+ first.version.dataset_revision.save(update_fields=["last_used_at"])+ self.dataset.last_used_at = self.old+ self.dataset.save()+ DatasetRetention().run()+ self.dataset.refresh_from_db()+ self.assertIsNone(self.dataset.expired_at)+ self.assertTrue(+ DatasetItemVersion.objects.for_team(self.team.id, canonical=True).filter(id=first.version.id).exists()+ )+ lifecycle.close(experiment.id, status="completed")+ self.dataset.refresh_from_db()+ self.assertGreater(self.dataset.last_used_at, self.old)+ self.dataset.last_used_at = self.old+ self.dataset.save()+ for _ in range(3):+ DatasetRetention().run()+ self.assertFalse(Dataset.objects.for_team(self.team.id, canonical=True).filter(id=self.dataset.id).exists())+ experiment.refresh_from_db()+ item = OfflineExperimentItem.objects.for_team(self.team.id).get(id=item_id)+ self.assertIsNone(experiment.dataset_revision_id)+ self.assertEqual(experiment.dataset_revision_identifier, str(first.version.dataset_revision_id))+ self.assertIsNone(item.dataset_item_version_id)+ self.assertEqual(item.dataset_item_version_identifier, str(first.version.id))+ self.assertTrue(OfflineEvaluationResult.objects.for_team(self.team.id).filter(item=item).exists())+ self.assertTrue(+ OfflineEvaluationUsageLedger.objects.for_team(self.team.id).filter(experiment_id=experiment.id).exists()+ )+ scorer.refresh_from_db()++ def test_cleanup_is_bounded_and_cannot_be_starved_by_pinned_history(self) -> None:+ first = self.create_item()+ self.update_item(first)+ OfflineExperimentService(team_id=self.team.id, user_access_control=None).create(+ ExperimentSubmission(+ id=uuid4(),+ name="Pinned snapshot",+ started_at=timezone.now(),+ dataset_revision_id=first.version.dataset_revision_id,+ )+ )+ self.age_history()+ self.dataset.retention_checked_at = None+ self.dataset.save()+ other = Dataset.objects.for_team(self.team.id, canonical=True).create(+ team=self.team, name="Other", last_used_at=self.old, retention_checked_at=timezone.now() - timedelta(days=1)+ )+ with patch("products.ai_observability.backend.dataset_retention.MAX_DATASETS_PER_RUN", 1):+ DatasetRetention().run()+ self.assertTrue(Dataset.objects.for_team(self.team.id, canonical=True).filter(id=other.id).exists())+ DatasetRetention().run()+ self.assertFalse(Dataset.objects.for_team(self.team.id, canonical=True).filter(id=other.id).exists())++ def test_expired_shared_history_no_longer_uses_a_cleanup_slot(self) -> None:+ first = self.create_item()+ self.create_item("Second item")+ self.age_history()+ DatasetRetention().run()+ self.dataset.refresh_from_db()+ first.version.dataset_revision.refresh_from_db()+ self.assertIsNotNone(first.version.dataset_revision.expired_at)+ other = Dataset.objects.for_team(self.team.id, canonical=True).create(+ team=self.team, name="Needs cleanup", last_used_at=self.old, retention_checked_at=timezone.now()+ )+ checked_at = self.dataset.retention_checked_at+ with patch("products.ai_observability.backend.dataset_retention.MAX_DATASETS_PER_RUN", 1):+ DatasetRetention().run()+ self.assertFalse(Dataset.objects.for_team(self.team.id, canonical=True).filter(id=other.id).exists())+ self.dataset.refresh_from_db()+ self.assertEqual(self.dataset.retention_checked_at, checked_at)++ @parameterized.expand([("active", False), ("abandoned", True)])+ def test_upload_protection_uses_snapshot_activity(self, _name: str, abandoned: bool) -> None:+ first = self.create_item()+ experiment = (+ OfflineExperimentService(team_id=self.team.id, user_access_control=None)+ .create(+ ExperimentSubmission(+ id=uuid4(),+ name="Upload activity",+ started_at=self.old,+ dataset_revision_id=first.version.dataset_revision_id,+ )+ )+ .experiment+ )+ OfflineExperiment.objects.for_team(self.team.id).filter(id=experiment.id).update(created_at=self.old)+ self.dataset.last_used_at = self.old+ self.dataset.save(update_fields=["last_used_at"])+ if abandoned:+ self.age_history()+ DatasetRetention().run()+ self.dataset.refresh_from_db()+ self.assertEqual(self.dataset.expired_at is not None, abandoned)+ experiment.refresh_from_db()+ self.assertEqual(experiment.status, OfflineExperiment.Status.UPLOADING)++ def test_result_only_upload_refreshes_snapshot_activity(self) -> None:+ first = self.create_item()+ experiment = (+ OfflineExperimentService(team_id=self.team.id, user_access_control=None)+ .create(+ ExperimentSubmission(+ id=uuid4(),+ name="Result batches",+ started_at=self.old,+ dataset_revision_id=first.version.dataset_revision_id,+ )+ )+ .experiment+ )+ ingestion = OfflineEvaluationIngestionService(team_id=self.team.id, user_access_control=None)+ item_id = uuid4()+ ingestion.upload(+ experiment.id,+ UploadSubmission(items=[ItemSubmission(id=item_id, dataset_item_version_id=first.version.id)], results=[]),+ )+ definition = ScoreDefinition.objects.create(team=self.team, name="Quality", kind="numeric")+ scorer = definition.create_new_version(config={}, created_by=None)+ self.age_history()+ self.dataset.last_used_at = self.old+ self.dataset.save(update_fields=["last_used_at"])+ ingestion.upload(+ experiment.id,+ UploadSubmission(+ items=[],+ results=[ResultSubmission(item_id=item_id, scorer_version_id=scorer.id, status="ok", value=1.0)],+ ),+ )+ DatasetRetention().run()+ first.version.dataset_revision.refresh_from_db()+ self.dataset.refresh_from_db()+ self.assertIsNone(self.dataset.expired_at)+ self.assertIsNone(first.version.dataset_revision.expired_at)+ assert first.version.dataset_revision.last_used_at is not None+ self.assertGreater(first.version.dataset_revision.last_used_at, self.old)++ def test_disabled_cleanup_preserves_pending_deletions_and_can_resume(self) -> None:+ self.create_item()+ self.dataset.expired_at = timezone.now()+ self.dataset.save(update_fields=["expired_at"])+ with override_instance_config(RETENTION_ENABLED_SETTING, False):+ sweep = DatasetRetention()+ sweep.run()+ self.assertTrue(sweep.disabled)+ self.assertTrue(DatasetItemVersion.objects.for_team(self.team.id).filter(dataset=self.dataset).exists())+ set_instance_setting(RETENTION_ENABLED_SETTING, True)+ DatasetRetention().run()+ self.assertFalse(DatasetItemVersion.objects.for_team(self.team.id).filter(dataset=self.dataset).exists())++ def test_disabling_during_a_sweep_stops_before_the_next_dataset(self) -> None:+ other = Dataset.objects.for_team(self.team.id, canonical=True).create(+ team=self.team, name="Next dataset", last_used_at=self.old, retention_checked_at=timezone.now()+ )+ self.dataset.last_used_at = self.old+ self.dataset.save(update_fields=["last_used_at"])+ clean_dataset = DatasetRetention._clean_dataset++ def disable_after_cleanup(sweep: DatasetRetention, team_id: int, dataset_id: UUID) -> None:+ clean_dataset(sweep, team_id, dataset_id)+ set_instance_setting(RETENTION_ENABLED_SETTING, False)++ with (+ override_instance_config(RETENTION_ENABLED_SETTING, True),+ patch.object(DatasetRetention, "_clean_dataset", disable_after_cleanup),+ ):+ sweep = DatasetRetention()+ sweep.run()+ self.assertTrue(sweep.disabled)+ self.assertFalse(Dataset.objects.for_team(self.team.id).filter(id=self.dataset.id).exists())+ self.assertTrue(Dataset.objects.for_team(self.team.id).filter(id=other.id).exists())++ @parameterized.expand([("dataset", True), ("history", False)])+ def test_evaluation_references_are_detached_in_batches(self, _name: str, expire_dataset: bool) -> None:+ first = self.create_item()+ lifecycle = OfflineExperimentService(team_id=self.team.id, user_access_control=None)+ ingestion = OfflineEvaluationIngestionService(team_id=self.team.id, user_access_control=None)+ experiment_ids = []+ for index in range(3):+ experiment = lifecycle.create(+ ExperimentSubmission(+ id=uuid4(),+ name=f"Repeated evaluation {index}",+ started_at=timezone.now(),+ dataset_revision_id=first.version.dataset_revision_id,+ )+ ).experiment+ experiment_ids.append(experiment.id)+ ingestion.upload(+ experiment.id,+ UploadSubmission(+ items=[ItemSubmission(id=uuid4(), dataset_item_version_id=first.version.id)], results=[]+ ),+ )+ lifecycle.close(experiment.id, status="completed")+ if not expire_dataset:+ self.update_item(first)+ self.age_history()+ if expire_dataset:+ self.dataset.last_used_at = self.old+ self.dataset.save(update_fields=["last_used_at"])++ items = OfflineExperimentItem.objects.for_team(self.team.id).filter(experiment_id__in=experiment_ids)+ experiments = OfflineExperiment.objects.for_team(self.team.id).filter(id__in=experiment_ids)+ versions = DatasetItemVersion.objects.for_team(self.team.id, canonical=True).filter(id=first.version.id)+ with patch("products.ai_observability.backend.dataset_retention.BATCH_SIZE", 2):+ DatasetRetention().run()+ self.assertEqual(items.filter(dataset_item_version_id=first.version.id).count(), 1)+ self.assertTrue(versions.exists())+ self.assertEqual(experiments.filter(dataset_revision_id=first.version.dataset_revision_id).count(), 3)+ with patch("django.db.models.query.QuerySet.delete", side_effect=DatabaseError("Injected failure")):+ failed_sweep = DatasetRetention()+ failed_sweep.run()+ self.assertEqual(failed_sweep.datasets_failed, 1)+ self.assertEqual(items.filter(dataset_item_version_id=first.version.id).count(), 1)+ self.assertTrue(versions.exists())+ DatasetRetention().run()+ self.assertFalse(versions.exists())+ self.assertEqual(items.filter(dataset_item_version_id__isnull=True).count(), 3)+ DatasetRetention().run()+ self.assertEqual(+ experiments.filter(dataset_revision_id=first.version.dataset_revision_id).count(),+ 1 if expire_dataset else 3,+ )+ self.assertTrue(+ DatasetRevision.objects.for_team(self.team.id, canonical=True)+ .filter(id=first.version.dataset_revision_id)+ .exists()+ )+ DatasetRetention().run()+ self.assertEqual(experiments.filter(dataset_revision_id__isnull=True).count(), 3 if expire_dataset else 0)+ self.assertEqual(items.filter(dataset_item_version_identifier=str(first.version.id)).count(), 3)+ self.assertEqual(+ experiments.filter(dataset_revision_identifier=str(first.version.dataset_revision_id)).count(), 3+ )++ def test_dataset_payload_deletion_is_batched_and_hidden_until_finished(self) -> None:+ first = self.create_item()+ second = self.create_item("Second item")+ self.dataset.last_used_at = self.old+ self.dataset.save()+ with patch("products.ai_observability.backend.dataset_retention.BATCH_SIZE", 1):+ sweep = DatasetRetention()+ with patch("products.ai_observability.backend.dataset_retention.logger.info") as log:+ with self.captureOnCommitCallbacks(execute=True):+ sweep.run()+ log.assert_not_called()+ log.assert_called_once_with(+ "dataset_retention_dataset_cleaned",+ team_id=self.team.id,+ dataset_id=str(self.dataset.id),+ datasets_expired=1,+ revisions_expired=0,+ versions_deleted=1,+ datasets_deleted=0,+ )+ self.assertEqual(sweep.versions_deleted, 1)+ self.dataset.refresh_from_db()+ self.assertIsNotNone(self.dataset.expired_at)+ replacement = create_dataset(team=self.team, created_by=None, name=self.dataset.name)+ self.assertNotEqual(replacement.id, self.dataset.id)+ with self.assertRaises(DatasetMutationConflict):+ create_dataset(team=self.team, created_by=None, name=self.dataset.name)+ self.assertEqual(+ DatasetItemVersion.objects.for_team(self.team.id, canonical=True).filter(dataset=self.dataset).count(),+ 1,+ )+ for _ in range(4):+ DatasetRetention().run()+ self.assertFalse(Dataset.objects.for_team(self.team.id, canonical=True).filter(id=self.dataset.id).exists())+ self.assertFalse(+ DatasetItemVersion.objects.for_team(self.team.id, canonical=True)+ .filter(id__in=[first.version.id, second.version.id])+ .exists()+ )++ @parameterized.expand([(1, RuntimeError), (100, DatabaseError), (100, SoftTimeLimitExceeded)])+ def test_deletion_failure_rolls_back_content_without_starving_cleanup(+ self, max_datasets: int, error_type: type[Exception]+ ) -> None:+ first = self.create_item()+ self.dataset.last_used_at = self.old+ self.dataset.save()+ other = Dataset.objects.for_team(self.team.id, canonical=True).create(+ team=self.team, name="Another eligible dataset", last_used_at=self.old, retention_checked_at=self.old+ )+ sweep = DatasetRetention()+ with (+ patch("products.ai_observability.backend.dataset_retention.MAX_DATASETS_PER_RUN", max_datasets),+ patch("django.db.models.query.QuerySet.delete", side_effect=error_type("Injected failure")),+ patch("products.ai_observability.backend.dataset_retention.logger.info") as log,+ self.captureOnCommitCallbacks(execute=True),+ ):+ if error_type is SoftTimeLimitExceeded:+ with self.assertRaises(SoftTimeLimitExceeded):+ sweep.run()+ else:+ sweep.run()+ self.assertFalse(any(call.kwargs.get("dataset_id") == str(self.dataset.id) for call in log.call_args_list))+ self.assertEqual(sweep.datasets_failed, 1)+ self.assertEqual(sweep.versions_deleted, 0)+ self.dataset.refresh_from_db()+ first.item.refresh_from_db()+ self.assertIsNone(self.dataset.expired_at)+ self.assertEqual(first.item.current_version_id, first.version.id)+ self.assertIsNotNone(self.dataset.retention_checked_at)+ if error_type is DatabaseError:+ self.assertFalse(Dataset.objects.for_team(self.team.id, canonical=True).filter(id=other.id).exists())+ else:+ with patch("products.ai_observability.backend.dataset_retention.MAX_DATASETS_PER_RUN", 1):+ DatasetRetention().run()+ self.assertFalse(Dataset.objects.for_team(self.team.id, canonical=True).filter(id=other.id).exists())++ @parameterized.expand([(8, False), (7, True), (1, True)])+ def test_expiring_soon_window(self, days: int, expected: bool) -> None:+ self.assertEqual(DatasetRetentionPolicy.expiring_soon(timezone.now() + timedelta(days=days)), expected)+++class TestDatasetRetentionConcurrency(TransactionTestCase):+ def setUp(self) -> None:+ _, _, self.team = Organization.objects.bootstrap(None)+ self.dataset = Dataset.objects.for_team(self.team.id, canonical=True).create(+ team=self.team, name="Concurrent retention", last_used_at=timezone.now() - timedelta(days=100)+ )++ @staticmethod+ def run_cleanup() -> None:+ try:+ with connection.cursor() as cursor:+ cursor.execute("SET lock_timeout = '2s'")+ DatasetRetention().run()+ finally:+ connection.close()++ def test_cleanup_skips_a_usage_transaction_then_rechecks_the_new_timestamp(self) -> None:+ with ThreadPoolExecutor(max_workers=1) as executor:+ with transaction.atomic():+ DatasetUsage.record(self.dataset)+ executor.submit(self.run_cleanup).result(timeout=10)+ executor.submit(self.run_cleanup).result(timeout=10)+ self.dataset.refresh_from_db()+ self.assertIsNone(self.dataset.expired_at)++ def test_usage_after_cleanup_cannot_resurrect_an_expired_dataset(self) -> None:+ create_dataset_item(team_id=self.team.id, dataset_id=self.dataset.id, created_by=None, input="Example")+ Dataset.objects.for_team(self.team.id, canonical=True).filter(id=self.dataset.id).update(+ last_used_at=self.dataset.last_used_at+ )+ with ThreadPoolExecutor(max_workers=1) as executor:+ executor.submit(self.run_cleanup).result(timeout=10)+ with self.assertRaises(Dataset.DoesNotExist):+ DatasetUsage.record(self.dataset)+ self.dataset.refresh_from_db()+ self.assertIsNotNone(self.dataset.expired_at)+++@time_machine.travel("2026-11-03T12:00:00Z", tick=False)+class TestDatasetRetentionMigration(TransactionTestCase):+ def test_existing_content_receives_a_full_retention_period(self) -> None:+ _, _, team = Organization.objects.bootstrap(None)+ executor = MigrationExecutor(connection)+ before = [("ai_observability", "0070_offline_payload_expiry_indexes")]+ after = [("ai_observability", "0074_release_expired_dataset_names")]+ executor.migrate(before)+ try:+ apps = executor.loader.project_state(before).apps+ legacy_dataset = apps.get_model("ai_observability", "Dataset").objects.create(+ team_id=team.id, name="Existing dataset"+ )+ legacy_revision = apps.get_model("ai_observability", "DatasetRevision").objects.create(+ team_id=team.id, dataset_id=legacy_dataset.id, revision=1+ )+ legacy_item = apps.get_model("ai_observability", "DatasetItem").objects.create(+ team_id=team.id, dataset_id=legacy_dataset.id+ )+ legacy_version = apps.get_model("ai_observability", "DatasetItemVersion").objects.create(+ team_id=team.id,+ dataset_id=legacy_dataset.id,+ dataset_item_id=legacy_item.id,+ dataset_revision_id=legacy_revision.id,+ version=1,+ input="Existing input",+ )+ started = timezone.now()+ MigrationExecutor(connection).migrate(after)+ dataset = Dataset.objects.for_team(team.id, canonical=True).get(id=legacy_dataset.id)+ revision = DatasetRevision.objects.for_team(team.id, canonical=True).get(id=legacy_revision.id)+ version = DatasetItemVersion.objects.for_team(team.id, canonical=True).get(id=legacy_version.id)+ assert dataset.last_used_at is not None+ assert revision.last_used_at is not None+ assert version.last_used_at is not None+ self.assertGreaterEqual(dataset.last_used_at, started)+ self.assertGreaterEqual(revision.last_used_at, started)+ self.assertGreaterEqual(version.last_used_at, started)+ DatasetRetention().run()+ self.assertTrue(DatasetItemVersion.objects.for_team(team.id, canonical=True).filter(id=version.id).exists())+ with self.assertRaises(IntegrityError), transaction.atomic():+ Dataset.objects.for_team(team.id, canonical=True).create(team_id=team.id, name=dataset.name)+ dataset.expired_at = timezone.now()+ dataset.save(update_fields=["expired_at"])+ replacement = Dataset.objects.for_team(team.id, canonical=True).create(team_id=team.id, name=dataset.name)+ self.assertNotEqual(dataset.id, replacement.id)+ finally:+ MigrationExecutor(connection).migrate(after)diff --git a/products/ai_observability/backend/test/test_dataset_retention_admin.py b/products/ai_observability/backend/test/test_dataset_retention_admin.pynew file mode 100644index 000000000000..2c66d1a71e01--- /dev/null+++ b/products/ai_observability/backend/test/test_dataset_retention_admin.py@@ -0,0 +1,41 @@+from posthog.test.base import BaseTest++from django.urls import reverse++from parameterized import parameterized++from posthog.models.instance_setting import override_instance_config++from products.ai_observability.backend.dataset_retention import RETENTION_ENABLED_SETTING, dataset_retention_enabled+++class TestDatasetRetentionAdmin(BaseTest):+ @parameterized.expand([("enabled", True), ("disabled", False)])+ def test_staff_can_change_cleanup_setting(self, value: str, expected: bool) -> None:+ self.user.is_staff = True+ self.user.save(update_fields=["is_staff"])+ self.client.force_login(self.user)+ url = reverse("admin:ai-observability-dataset-retention")+ with override_instance_config(RETENTION_ENABLED_SETTING, not expected):+ response = self.client.get(url)+ self.assertContains(response, "AI observability dataset retention")+ self.assertEqual(dataset_retention_enabled(), not expected)+ response = self.client.post(url, {"enabled": value}, follow=True)+ self.assertEqual(response.status_code, 200)+ self.assertEqual(dataset_retention_enabled(), expected)++ def test_nonstaff_cannot_change_cleanup_setting(self) -> None:+ self.client.force_login(self.user)+ with override_instance_config(RETENTION_ENABLED_SETTING, True):+ response = self.client.post(reverse("admin:ai-observability-dataset-retention"), {"enabled": "disabled"})+ self.assertEqual(response.status_code, 302)+ self.assertTrue(dataset_retention_enabled())++ def test_invalid_setting_does_not_disable_cleanup(self) -> None:+ self.user.is_staff = True+ self.user.save(update_fields=["is_staff"])+ self.client.force_login(self.user)+ with override_instance_config(RETENTION_ENABLED_SETTING, True):+ response = self.client.post(reverse("admin:ai-observability-dataset-retention"), {"enabled": "invalid"})+ self.assertEqual(response.status_code, 200)+ self.assertTrue(dataset_retention_enabled())diff --git a/products/ai_observability/frontend/datasets/AIObservabilityDatasetScene.stories.tsx b/products/ai_observability/frontend/datasets/AIObservabilityDatasetScene.stories.tsxindex 709afd51fa93..c27d569c58a7 100644--- a/products/ai_observability/frontend/datasets/AIObservabilityDatasetScene.stories.tsx+++ b/products/ai_observability/frontend/datasets/AIObservabilityDatasetScene.stories.tsx@@ -19,6 +19,8 @@ const dataset: DatasetReadApi = {metadata: { owner: 'support', language: 'en' },tags: [],archived: false,+ expires_at: '2024-04-01T12:00:00Z',+ expiring_soon: false,current_revision: 1,current_revision_id: 'example-revision',created_at: '2024-01-01T00:00:00Z',diff --git a/products/ai_observability/frontend/datasets/AIObservabilityDatasetScene.tsx b/products/ai_observability/frontend/datasets/AIObservabilityDatasetScene.tsxindex 45b787a53cea..655e47bca1b4 100644--- a/products/ai_observability/frontend/datasets/AIObservabilityDatasetScene.tsx+++ b/products/ai_observability/frontend/datasets/AIObservabilityDatasetScene.tsx@@ -273,9 +273,9 @@ export function AIObservabilityDatasetScene(): JSX.Element {<h2 className="text-base font-semibold m-0 truncate" title={datasetForm.name}>{isNewDataset ? 'New dataset' : (isDataset(dataset) && dataset.name) || 'Dataset'}</h2>- {isDataset(dataset) && dataset.archived && (- <LemonTag type="muted" size="small">- Archived+ {isDataset(dataset) && (dataset.expiring_soon || dataset.archived) && (+ <LemonTag type={dataset.expiring_soon ? 'warning' : 'muted'} size="small">+ {dataset.expiring_soon ? 'Expiring soon' : 'Archived'}</LemonTag>)}</nav>diff --git a/products/ai_observability/frontend/datasets/AIObservabilityDatasetsScene.tsx b/products/ai_observability/frontend/datasets/AIObservabilityDatasetsScene.tsxindex 77181290a7e4..f98e302c11ec 100644--- a/products/ai_observability/frontend/datasets/AIObservabilityDatasetsScene.tsx+++ b/products/ai_observability/frontend/datasets/AIObservabilityDatasetsScene.tsx@@ -88,10 +88,11 @@ export function AIObservabilityDatasetsScene({ navigation }: { navigation?: Reac{title: 'Status',key: 'status',+ width: 112,render: function renderStatus(_, dataset) {return (- <LemonTag type={dataset.archived ? 'muted' : 'success'}>- {dataset.archived ? 'Archived' : 'Active'}+ <LemonTag type={dataset.expiring_soon ? 'warning' : dataset.archived ? 'muted' : 'success'}>+ {dataset.expiring_soon ? 'Expiring soon' : dataset.archived ? 'Archived' : 'Active'}</LemonTag>)},diff --git a/products/ai_observability/frontend/datasets/DatasetItems.test.tsx b/products/ai_observability/frontend/datasets/DatasetItems.test.tsxindex 067ae3493acb..9706b8fa0e64 100644--- a/products/ai_observability/frontend/datasets/DatasetItems.test.tsx+++ b/products/ai_observability/frontend/datasets/DatasetItems.test.tsx@@ -41,6 +41,8 @@ const DATASET: Dataset = {updated_at: TIMESTAMP,created_by: null,archived: false,+ expires_at: null,+ expiring_soon: false,current_revision: null,current_revision_id: null,user_access_level: 'editor',diff --git a/products/ai_observability/frontend/datasets/aiObservabilityDatasetLogic.test.ts b/products/ai_observability/frontend/datasets/aiObservabilityDatasetLogic.test.tsindex 311bd009e18c..62049623815a 100644--- a/products/ai_observability/frontend/datasets/aiObservabilityDatasetLogic.test.ts+++ b/products/ai_observability/frontend/datasets/aiObservabilityDatasetLogic.test.ts@@ -58,6 +58,8 @@ describe('aiObservabilityDatasetLogic', () => {updated_at: '2024-01-01T00:00:00Z',created_by: null,archived: false,+ expires_at: null,+ expiring_soon: false,current_revision: null,current_revision_id: null,user_access_level: 'editor',diff --git a/products/ai_observability/frontend/datasets/aiObservabilityDatasetLogic.ts b/products/ai_observability/frontend/datasets/aiObservabilityDatasetLogic.tsindex 4e018875b096..0b855c775eb8 100644--- a/products/ai_observability/frontend/datasets/aiObservabilityDatasetLogic.ts+++ b/products/ai_observability/frontend/datasets/aiObservabilityDatasetLogic.ts@@ -986,7 +986,10 @@ export const aiObservabilityDatasetLogic = kea<aiObservabilityDatasetLogicType>({ value: 'latest', label: 'Latest' },...revisionNumbers.sort((a, b) => b - a)- .map((revision) => ({ value: String(revision), label: `Revision ${revision}` })),+ .map((revision) => ({+ value: String(revision),+ label: `Revision ${revision}${revisions.results.find((entry) => entry.revision === revision)?.expiring_soon ? ' · Expiring soon' : ''}`,+ })),]},],diff --git a/products/ai_observability/frontend/datasets/aiObservabilityDatasetsLogic.test.ts b/products/ai_observability/frontend/datasets/aiObservabilityDatasetsLogic.test.tsindex b948a52f776a..fb660000826e 100644--- a/products/ai_observability/frontend/datasets/aiObservabilityDatasetsLogic.test.ts+++ b/products/ai_observability/frontend/datasets/aiObservabilityDatasetsLogic.test.ts@@ -40,6 +40,8 @@ describe('aiObservabilityDatasetsLogic', () => {updated_at: '2024-01-01T00:00:00Z',created_by: null,archived: false,+ expires_at: null,+ expiring_soon: false,current_revision: null,current_revision_id: null,user_access_level: 'editor',@@ -56,6 +58,8 @@ describe('aiObservabilityDatasetsLogic', () => {updated_at: '2024-01-02T00:00:00Z',created_by: null,archived: false,+ expires_at: null,+ expiring_soon: false,current_revision: null,current_revision_id: null,user_access_level: 'editor',diff --git a/products/ai_observability/frontend/datasets/saveToDatasetButtonLogic.test.ts b/products/ai_observability/frontend/datasets/saveToDatasetButtonLogic.test.tsindex cd712c5a2b90..ef43eb8cb079 100644--- a/products/ai_observability/frontend/datasets/saveToDatasetButtonLogic.test.ts+++ b/products/ai_observability/frontend/datasets/saveToDatasetButtonLogic.test.ts@@ -41,6 +41,8 @@ describe('saveToDatasetButtonLogic', () => {updated_at: '2024-01-01T00:00:00Z',created_by: null,archived: false,+ expires_at: null,+ expiring_soon: false,current_revision: null,current_revision_id: null,user_access_level: 'editor',@@ -57,6 +59,8 @@ describe('saveToDatasetButtonLogic', () => {updated_at: '2024-01-02T00:00:00Z',created_by: null,archived: false,+ expires_at: null,+ expiring_soon: false,current_revision: null,current_revision_id: null,user_access_level: 'editor',diff --git a/products/ai_observability/frontend/generated/api.schemas.ts b/products/ai_observability/frontend/generated/api.schemas.tsindex 1d6f1b50ec26..fa2923423782 100644--- a/products/ai_observability/frontend/generated/api.schemas.ts+++ b/products/ai_observability/frontend/generated/api.schemas.ts@@ -2111,6 +2111,13 @@ export interface DatasetReadApi {/** Project tag names assigned to this resource. */readonly tags: readonly string[]readonly archived: boolean+ /**+ * Earliest cleanup time if the dataset remains unused.+ * @nullable+ */+ readonly expires_at: string | null+ /** Whether the dataset is within seven days of expiry. */+ readonly expiring_soon: boolean/*** Latest dataset revision, or null before the first item is added.* @nullable@@ -2237,6 +2244,13 @@ export interface DatasetRevisionReadApi {readonly created_by: UserBasicApi | null/** Project that owns the revision. */readonly team_id: number+ /**+ * Earliest expiry time of this dataset snapshot if unused.+ * @nullable+ */+ readonly expires_at: string | null+ /** Whether this snapshot is within seven days of expiry. */+ readonly expiring_soon: boolean}export interface PaginatedDatasetRevisionReadListApi {diff --git a/products/ai_observability/mcp/tools.yaml b/products/ai_observability/mcp/tools.yamlindex d516dae5b5a3..63845e730f1f 100644--- a/products/ai_observability/mcp/tools.yaml+++ b/products/ai_observability/mcp/tools.yaml@@ -121,8 +121,9 @@ tools:enrich_url: '{id}'title: Archive datasetdescription: >- Archive a dataset by ID. Its revisions and items remain readable, but item mutations are rejected until the- dataset is restored. Archiving an already archived dataset leaves it unchanged.+ Archive a dataset by ID. Its revisions and items remain readable until expiry, but item mutations are+ rejected until the dataset is restored. Archived datasets and historical revisions expire after 30 days of+ inactivity; active datasets expire after 90 days. Archiving an already archived dataset leaves it unchanged.feature_flag: llm-analytics-datasetsurl_prefix: /ai-evals/datasetsresponse:@@ -305,9 +306,10 @@ tools:list: truetitle: List dataset item versionsdescription: >- List every immutable version of a dataset item, newest first. The response includes version metadata only.- To retrieve a version's content, call `llma-dataset-item-get` with the item ID and its `dataset_revision` as- `revision`.+ List retained immutable versions of a dataset item, newest first. Unused history expires after 30 days. The+ response includes version metadata only. To read content, call `llma-dataset-item-get` with a retained+ revision containing the version. Its introducing `dataset_revision` may have expired even when a later+ snapshot still needs the version; expired revisions return 404.feature_flag: llm-analytics-datasetsresponse:include:@@ -337,7 +339,8 @@ tools:description: >List active datasets in the current project by default. Filter by search text, UUIDs, or tags, or set`archived` to list archived datasets. Pass tags as a JSON array of names and tag_match as any (OR) or all- (AND).+ (AND). Active datasets expire after 90 days of inactivity; archived datasets expire after 30 days. Listing+ datasets does not refresh retention. Expired datasets are omitted.feature_flag: llm-analytics-datasetsurl_prefix: /ai-evals/datasetsresponse:@@ -348,6 +351,8 @@ tools:- tags- archived- current_revision+ - expires_at+ - expiring_soon- created_at- updated_at- user_access_level@@ -386,8 +391,9 @@ tools:list: truetitle: List dataset revisionsdescription: >- List immutable revisions for a dataset by UUID, newest first. Use a revision number with the dataset item- list tool to inspect that exact snapshot.+ List available immutable revisions for a dataset by UUID, newest first. Historical revisions expire after 30+ days of inactivity and are omitted. Listing does not refresh retention. Use a revision number with the+ dataset item list tool to inspect that exact snapshot and refresh its retention.feature_flag: llm-analytics-datasetsurl_prefix: /ai-evals/datasetsresponse:@@ -395,6 +401,8 @@ tools:- id- dataset_id- revision+ - expires_at+ - expiring_soon- created_at- created_byinformational_wrapper:diff --git a/products/posthog_ai/skills/querying-posthog-data/references/models-datasets.md.j2 b/products/posthog_ai/skills/querying-posthog-data/references/models-datasets.md.j2index 33afefe5103f..192fa054f49d 100644--- a/products/posthog_ai/skills/querying-posthog-data/references/models-datasets.md.j2+++ b/products/posthog_ai/skills/querying-posthog-data/references/models-datasets.md.j2@@ -4,6 +4,13 @@ Datasets hold curated inputs, expected outputs, and optional trace provenance foDataset items have stable IDs and immutable content versions.Every item mutation creates a dataset revision, which makes prior dataset contents queryable as exact snapshots.+Active datasets expire after 90 days of inactivity; archived datasets and historical revisions expire after 30 days,+with a 12-hour usage recording margin. Expired datasets and revisions may remain in these tables during cleanup.+Only datasets and requested snapshots with `expired_at IS NULL` are available; expired snapshots can be incomplete.+HogQL reads do not refresh retention. Use the dataset API or MCP item-read tools to refresh a snapshot's retention.+Expired snapshots return 404 through those tools. Dataset names can be reused after expiry, so select by UUID or+filter out expired datasets when selecting by name.+## Dataset (`system.datasets`){{ schema_columns('system.datasets') }}@@ -52,8 +59,10 @@ SELECTv.source_output,v.metadataFROM system.dataset_items AS i+INNER JOIN system.datasets AS d ON d.id = i.dataset_idINNER JOIN system.dataset_item_versions AS v ON v.id = i.current_version_idWHERE i.dataset_id = '01234567-89ab-cdef-0123-456789abcdef'+ AND d.expired_at IS NULLAND v.archived = falseORDER BY i.created_at DESCLIMIT 100@@ -61,6 +70,9 @@ LIMIT 100Reconstruct active items at dataset revision 7:+Check the requested snapshot's expiry, not the expiry of each version's introducing revision. A version introduced+by an expired revision can still be needed by an available snapshot.+```sqlSELECTi.id,@@ -71,28 +83,34 @@ SELECTtupleElement(argMax(tuple(v.source_output), r.revision), 1) AS source_output,argMax(v.metadata, r.revision) AS metadataFROM system.dataset_items AS i+INNER JOIN system.datasets AS d ON d.id = i.dataset_id+INNER JOIN system.dataset_revisions AS snapshot ON snapshot.dataset_id = i.dataset_id AND snapshot.revision = 7INNER JOIN system.dataset_item_versions AS v ON v.dataset_item_id = i.idINNER JOIN system.dataset_revisions AS r ON r.id = v.dataset_revision_idWHERE i.dataset_id = '01234567-89ab-cdef-0123-456789abcdef'+ AND d.expired_at IS NULL+ AND snapshot.expired_at IS NULLAND r.revision <= 7GROUP BY i.id, i.client_item_idHAVING argMax(v.archived, r.revision) = falseORDER BY i.id```-List the immutable history of one item:+List the retained immutable history of one item in an available dataset:```sqlSELECT- version,- archived,- input,- expected_output,- metadata,- dataset_revision_id,- created_by_id,- created_at-FROM system.dataset_item_versions-WHERE dataset_item_id = '01234567-89ab-cdef-0123-456789abcdef'-ORDER BY version DESC+ v.version,+ v.archived,+ v.input,+ v.expected_output,+ v.metadata,+ v.dataset_revision_id,+ v.created_by_id,+ v.created_at+FROM system.dataset_item_versions AS v+INNER JOIN system.datasets AS d ON d.id = v.dataset_id+WHERE v.dataset_item_id = '01234567-89ab-cdef-0123-456789abcdef'+ AND d.expired_at IS NULL+ORDER BY v.version DESC```diff --git a/services/mcp/schema/generated-tool-definitions.json b/services/mcp/schema/generated-tool-definitions.jsonindex 50d1d6fee604..8ab0cfbbc73c 100644--- a/services/mcp/schema/generated-tool-definitions.json+++ b/services/mcp/schema/generated-tool-definitions.json@@ -8759,7 +8759,7 @@}},"llma-dataset-archive": {- "description": "Archive a dataset by ID. Its revisions and items remain readable, but item mutations are rejected until the dataset is restored. Archiving an already archived dataset leaves it unchanged.",+ "description": "Archive a dataset by ID. Its revisions and items remain readable until expiry, but item mutations are rejected until the dataset is restored. Archived datasets and historical revisions expire after 30 days of inactivity; active datasets expire after 90 days. Archiving an already archived dataset leaves it unchanged.","category": "AI observability","feature": "llm_analytics","summary": "Archive dataset",@@ -8903,7 +8903,7 @@"feature_flag": "llm-analytics-datasets"},"llma-dataset-item-version-list": {- "description": "List every immutable version of a dataset item, newest first. The response includes version metadata only. To retrieve a version's content, call `llma-dataset-item-get` with the item ID and its `dataset_revision` as `revision`.",+ "description": "List retained immutable versions of a dataset item, newest first. Unused history expires after 30 days. The response includes version metadata only. To read content, call `llma-dataset-item-get` with a retained revision containing the version. Its introducing `dataset_revision` may have expired even when a later snapshot still needs the version; expired revisions return 404.","category": "AI observability","feature": "llm_analytics","summary": "List dataset item versions",@@ -8919,7 +8919,7 @@"feature_flag": "llm-analytics-datasets"},"llma-dataset-list": {- "description": "List active datasets in the current project by default. Filter by search text, UUIDs, or tags, or set `archived` to list archived datasets. Pass tags as a JSON array of names and tag_match as any (OR) or all (AND).",+ "description": "List active datasets in the current project by default. Filter by search text, UUIDs, or tags, or set `archived` to list archived datasets. Pass tags as a JSON array of names and tag_match as any (OR) or all (AND). Active datasets expire after 90 days of inactivity; archived datasets expire after 30 days. Listing datasets does not refresh retention. Expired datasets are omitted.","category": "AI observability","feature": "llm_analytics","summary": "List datasets",@@ -8951,7 +8951,7 @@"feature_flag": "llm-analytics-datasets"},"llma-dataset-revision-list": {- "description": "List immutable revisions for a dataset by UUID, newest first. Use a revision number with the dataset item list tool to inspect that exact snapshot.",+ "description": "List available immutable revisions for a dataset by UUID, newest first. Historical revisions expire after 30 days of inactivity and are omitted. Listing does not refresh retention. Use a revision number with the dataset item list tool to inspect that exact snapshot and refresh its retention.","category": "AI observability","feature": "llm_analytics","summary": "List dataset revisions",diff --git a/services/mcp/schema/tool-definitions-all.json b/services/mcp/schema/tool-definitions-all.jsonindex c6ed92ba23e0..f648462b5f6d 100644--- a/services/mcp/schema/tool-definitions-all.json+++ b/services/mcp/schema/tool-definitions-all.json@@ -9025,7 +9025,7 @@}},"llma-dataset-archive": {- "description": "Archive a dataset by ID. Its revisions and items remain readable, but item mutations are rejected until the dataset is restored. Archiving an already archived dataset leaves it unchanged.",+ "description": "Archive a dataset by ID. Its revisions and items remain readable until expiry, but item mutations are rejected until the dataset is restored. Archived datasets and historical revisions expire after 30 days of inactivity; active datasets expire after 90 days. Archiving an already archived dataset leaves it unchanged.","category": "AI observability","feature": "llm_analytics","summary": "Archive dataset",@@ -9169,7 +9169,7 @@"feature_flag": "llm-analytics-datasets"},"llma-dataset-item-version-list": {- "description": "List every immutable version of a dataset item, newest first. The response includes version metadata only. To retrieve a version's content, call `llma-dataset-item-get` with the item ID and its `dataset_revision` as `revision`.",+ "description": "List retained immutable versions of a dataset item, newest first. Unused history expires after 30 days. The response includes version metadata only. To read content, call `llma-dataset-item-get` with a retained revision containing the version. Its introducing `dataset_revision` may have expired even when a later snapshot still needs the version; expired revisions return 404.","category": "AI observability","feature": "llm_analytics","summary": "List dataset item versions",@@ -9185,7 +9185,7 @@"feature_flag": "llm-analytics-datasets"},"llma-dataset-list": {- "description": "List active datasets in the current project by default. Filter by search text, UUIDs, or tags, or set `archived` to list archived datasets. Pass tags as a JSON array of names and tag_match as any (OR) or all (AND).",+ "description": "List active datasets in the current project by default. Filter by search text, UUIDs, or tags, or set `archived` to list archived datasets. Pass tags as a JSON array of names and tag_match as any (OR) or all (AND). Active datasets expire after 90 days of inactivity; archived datasets expire after 30 days. Listing datasets does not refresh retention. Expired datasets are omitted.","category": "AI observability","feature": "llm_analytics","summary": "List datasets",@@ -9217,7 +9217,7 @@"feature_flag": "llm-analytics-datasets"},"llma-dataset-revision-list": {- "description": "List immutable revisions for a dataset by UUID, newest first. Use a revision number with the dataset item list tool to inspect that exact snapshot.",+ "description": "List available immutable revisions for a dataset by UUID, newest first. Historical revisions expire after 30 days of inactivity and are omitted. Listing does not refresh retention. Use a revision number with the dataset item list tool to inspect that exact snapshot and refresh its retention.","category": "AI observability","feature": "llm_analytics","summary": "List dataset revisions",diff --git a/services/mcp/src/api/generated.ts b/services/mcp/src/api/generated.tsindex 7bb51a70c910..7fb573aabe9f 100644--- a/services/mcp/src/api/generated.ts+++ b/services/mcp/src/api/generated.ts@@ -33948,6 +33948,13 @@ export namespace Schemas {/** Project tag names assigned to this resource. */readonly tags: readonly string[];readonly archived: boolean;+ /**+ * Earliest cleanup time if the dataset remains unused.+ * @nullable+ */+ readonly expires_at: string | null;+ /** Whether the dataset is within seven days of expiry. */+ readonly expiring_soon: boolean;/*** Latest dataset revision, or null before the first item is added.* @nullable@@ -33980,6 +33987,13 @@ export namespace Schemas {readonly created_by: UserBasic | null;/** Project that owns the revision. */readonly team_id: number;+ /**+ * Earliest expiry time of this dataset snapshot if unused.+ * @nullable+ */+ readonly expires_at: string | null;+ /** Whether this snapshot is within seven days of expiry. */+ readonly expiring_soon: boolean;}export interface DayItem {diff --git a/services/mcp/src/tools/generated/ai_observability.ts b/services/mcp/src/tools/generated/ai_observability.tsindex ea3b6f69a3e6..50163a2e3c74 100644--- a/services/mcp/src/tools/generated/ai_observability.ts+++ b/services/mcp/src/tools/generated/ai_observability.ts@@ -610,6 +610,8 @@ const llmaDatasetList = (): ToolBase<'tags','archived','current_revision',+ 'expires_at',+ 'expiring_soon','created_at','updated_at','user_access_level',@@ -674,7 +676,15 @@ const llmaDatasetRevisionList = (): ToolBase<const filtered = {...result,results: (result.results ?? []).map((item: any) =>- pickResponseFields(item, ['id', 'dataset_id', 'revision', 'created_at', 'created_by'])+ pickResponseFields(item, [+ 'id',+ 'dataset_id',+ 'revision',+ 'expires_at',+ 'expiring_soon',+ 'created_at',+ 'created_by',+ ])),} as typeof resultreturn withInformationalResponse(