:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(data-modeling): add model usage engine filters (#122683)
Jovan Sakovic648e2d88e059
diff --git a/docs/published/handbook/engineering/data-warehouse.md b/docs/published/handbook/engineering/data-warehouse.mdindex bb9c49b22e1f..9c70ec1e24a8 100644--- a/docs/published/handbook/engineering/data-warehouse.md+++ b/docs/published/handbook/engineering/data-warehouse.md@@ -18,10 +18,10 @@ This reservation does not add qualified names to existing models or require a na## Model materialization usage-The Models **Usage** tab shows bytes read, CPU time, and run counts for ClickHouse materializations over 7, 30, or 90 complete UTC days.+The Models **Usage** tab shows bytes read, CPU time, and run counts for materializations over a preset or custom range of complete UTC days.The totals and model breakdown follow saved-query access controls. The breakdown sorts by bytes read and includes soft-deleted models while their saved queries and job history exist.Usage includes schema checks, retries, failed runs, and canceled runs, and is attributed to the UTC day each run finished.-It does not include managed warehouse or legacy Duckgres runs.+The date and materialized-view filters apply to the whole tab. The jobs table also has a status filter. Projects with the Trino managed warehouse enabled also get a tab-level engine filter, daily bars stacked by engine, and separate ClickHouse and managed warehouse byte totals per model. Legacy Duckgres runs are not included.A daily partitioned Dagster job, `model_job_usage_job`, collects usage from the regional query log archive at 03:00 UTC for the previous day.It stores totals on `DataModelingJob`, keeping usage reads independent of query-log retention and leaving the streaming materialization path unchanged.diff --git a/frontend/snapshots.yml b/frontend/snapshots.ymlindex 4fe6b6160387..9af5e5c99360 100644--- a/frontend/snapshots.yml+++ b/frontend/snapshots.yml@@ -7469,25 +7469,25 @@ snapshots:products-data-modeling-models-overview--partial-last-page--light:hash: v1.k794b7964.0a510efc77afae6a02c52831583c4c38870a6e9c1a1dcd78c2905ff05933c085.KSDQ61iPt91mJjAozciHJ7_p5ed-dd3IkN5KPYxR204products-data-modeling-models-usage--empty--dark:- hash: v1.k794b7964.9b010812c07fd68c796397e95a852963fe0ed2d076461033570268f068e578fe.ddnt_P7ccX16VE7-PvsLFu_X8SNw7Wqz18Lc0d37Ssw+ hash: v1.k794b7964.18c0ad26660aeabfe409a49668ee77318007a883e8edde9ba4d80338355160c4.9cENDLqBNgbWBwcwOed_5UaQz1HyRlVLupZTqDfYfbEproducts-data-modeling-models-usage--empty--light:- hash: v1.k794b7964.b5eb71d311f22b8f007de3b6515a8e91de2e104d81e99b157a79d4884c480b16.p2xjDEkVhQBCnMsxbNy-wTSZ6UPVCx93_01GLM9SWZg+ hash: v1.k794b7964.be096ff4381492e09c3da8d54206c609c72befd5618a1d3e0c551cb6d0f1e576.qc5ZeMyVOmZZiMW5_TgNNVHznvY3HEW8-dQf-XLxrWQproducts-data-modeling-models-usage--error--dark:- hash: v1.k794b7964.ed14c0b8a35aabc4d8c9b75a98990f69eef5afc0d1ddcdf87f52ef39793c0f1c.6UKjPNvqmc9s2N7e8VFibXKF8409RuD9cZrNLJMb95Y+ hash: v1.k794b7964.fc525bf84886def1051de338491e76c5204dd7ece56dc2b8b322c3e64692e44c.tziVJb9cJG7vFYcFUJbs6uHsNcm4KHJ7idyHWmgu_8Aproducts-data-modeling-models-usage--error--light:- hash: v1.k794b7964.1cd78cc9f4b33d01ac1f08b4af0bc46c50bc06d079de3f1c97aa6c0b53a2654c.m5r_dItuhFq67MNJo1BNrN4KMGwqJROhaY9JAbjFepI+ hash: v1.k794b7964.3eb45cfc31a92bf216e2cac9f1b2afae561ebcbee9212c8c2bb8d262314aef3f.fEKQiIHOXqxZukkfbGgbNuiWFTUjtR7CyNM3WU13p80products-data-modeling-models-usage--narrow--dark:- hash: v1.k794b7964.3737a68af17ed35f92462cd0d965d1a619a613122c38eb11645284ea6f049042.oKe2F_N5SBtfaZrwA-IOu0a1Dv8RbGv4mgSrsf94SF8+ hash: v1.k794b7964.4fad717b5fe2b7729caf0623577cb140fb5566b14137897c7f123a6fa0322489.efpSZxHEV4137a0YaX0t5ZGZm3bvNGQUhgTQ2E3PuCYproducts-data-modeling-models-usage--narrow--light:- hash: v1.k794b7964.1c299bdad8de569e63bb754dce5e3f22e51d03c1c24832f90d75f965ec387b61.zPOZG3zuCRaeZ7IAkDFaPtaTkPUKCCDww6NiB7gbaRk+ hash: v1.k794b7964.3ac8e20a35a9b3dc80eebecaac95e33b09650194596d6d3c1f8bca5bfdf93269.09Su4MCXEdjI5TNoKQcYkbw67_-PHOJ6F4F7xzWoVWEproducts-data-modeling-models-usage--populated--dark:- hash: v1.k794b7964.0efcfeaeddcd34cd480814bd7c05424c7e18576085fbe0b6d5852a48c1f6df5c.6a8Fu4MPsdzKYcZ0TnVT29fMShkXrvd0pwWGeefRObQ+ hash: v1.k794b7964.13bc4856822b1758befe9e138af9e5ccb9a5d0626f67884eaa0e28f0f9b45fc9.-FIl8e5Fei8C-XZp3Fmo8Vz7BVAo1N7Iq1hlbJlDWggproducts-data-modeling-models-usage--populated--light:- hash: v1.k794b7964.4ee0686c5a3af483c0fa81fbcf4e3ccb518e3113faf77df05226f743df5e8ae6.FXPBKh0kEYIsNwqLRBR8EysShkkwZ30JJ05IS-VD0KI+ hash: v1.k794b7964.513c49058b973329230d944342af41d17332f9ec88f6f807f7e7c0184cc233b3.TVTVGPxsNKr7BIIJi--ZCC6rK5bLHiX-FjCWOzef5i0products-data-modeling-models-usage--unmeasured--dark:- hash: v1.k794b7964.faffa961c8ecb1afdcf0f735523f121268cc5426e0db2d8736aac89a5615a074.eCMENKYh5gT9qgM34Cxh4P3341fjFGoGXt3Tmuuqd6M+ hash: v1.k794b7964.fbccec9254c34a6b3a3141c465e6e794f7c0c4b51b15ab3910619da0e1d61df3.gJtdNulD3Zvx-D5KZ2R-YFrfvlCrRMDpPSkZZ0-Gfgoproducts-data-modeling-models-usage--unmeasured--light:- hash: v1.k794b7964.988588eee2a1669d7d3d7647d819bb05e586f8924394d1dad6e841692e3527dc._6MYcT52dQondfg0kK8DSsx3nZtG15LlKFZH67q5-1A+ hash: v1.k794b7964.c53a65db17e9cbb7c5a8c97d200376ecf2be864f797b7c658195f2a2d515e67d.LKM9CH0fIBA1V4aYUqn28Odwukzp6OW04kJUtuLnY-Iproducts-data-modeling-node-detail-scene--deleted-model--dark:hash: v1.k794b7964.bf008eaa19295f4131e58d639e52c574e724973ac897a20d95c905206ccfe333.8CL_nCzrTvLfheyYXNeMHjX8MI8t_lazxmw_8FtPLu4products-data-modeling-node-detail-scene--deleted-model--light:diff --git a/posthog/temporal/data_modeling/activities/materialize_view_managed_warehouse.py b/posthog/temporal/data_modeling/activities/materialize_view_managed_warehouse.pyindex b78ff5c3de93..a66aae92124e 100644--- a/posthog/temporal/data_modeling/activities/materialize_view_managed_warehouse.py+++ b/posthog/temporal/data_modeling/activities/materialize_view_managed_warehouse.py@@ -106,7 +106,7 @@ class ManagedWarehouseShadowResult:file_size_bytes: int = 0file_size_delta_bytes: int = 0run_mode: str = UNKNOWN_RUN_MODE- bytes_read: int | None = None+ read_bytes: int | None = Nonedef _failed_run_mode(plan: WritePlan | None, use_trino: bool) -> str:@@ -244,8 +244,10 @@ def _resolve_managed_warehouse_job(job_id: str, result: "ManagedWarehouseShadowRjob.status = DataModelingJobStatus.FAILEDjob.rows_materialized = 0job.error = result.error- job.bytes_read = result.bytes_read- job.last_run_at = dt.datetime.now(dt.UTC)+ completed_at = dt.datetime.now(dt.UTC)+ job.read_bytes = result.read_bytes+ job.usage_updated_at = completed_at if result.read_bytes is not None else None+ job.last_run_at = completed_atjob.save()return job.engine@@ -351,7 +353,7 @@ async def _materialize_view_managed_warehouse(file_size_bytes=result.file_size_bytes,file_size_delta_bytes=result.file_size_delta_bytes,run_mode=INCREMENTAL_RUN_MODE if inputs.use_trino and result.merged else FULL_REFRESH_RUN_MODE,- bytes_read=_total_bytes_read(processed_bytes),+ read_bytes=_total_read_bytes(processed_bytes),)job_engine = await _resolve_managed_warehouse_job(inputs.job_id, shadow_result)await clear_node_suspension_for_engine(@@ -400,7 +402,7 @@ async def _materialize_view_managed_warehouse(table_name=table_name,error=str(e),run_mode=_failed_run_mode(plan, inputs.use_trino),- bytes_read=_total_bytes_read(processed_bytes),+ read_bytes=_total_read_bytes(processed_bytes),)if plan is not None and plan.incremental:# The merge may have stopped partway or failed on a schema change. Rebuilding next run@@ -427,7 +429,7 @@ async def _materialize_view_managed_warehouse(return shadow_result-def _total_bytes_read(processed_bytes: list[int]) -> int | None:+def _total_read_bytes(processed_bytes: list[int]) -> int | None:# No statement reported its bytes, so the run was not measured. That is not the same as reading zero bytes.return sum(processed_bytes) if processed_bytes else Nonediff --git a/posthog/temporal/tests/data_modeling/test_materialize_view_activities.py b/posthog/temporal/tests/data_modeling/test_materialize_view_activities.pyindex 3bb10fddbb24..8daf14bf2b1a 100644--- a/posthog/temporal/tests/data_modeling/test_materialize_view_activities.py+++ b/posthog/temporal/tests/data_modeling/test_materialize_view_activities.py@@ -236,7 +236,9 @@ class TestMaterializeViewManagedWarehouseActivity:)legacy_execute.assert_not_called()await database_sync_to_async(ajob.refresh_from_db)()- assert ajob.bytes_read == 345+ assert ajob.read_bytes == 345+ assert ajob.read_bytes == 345+ assert ajob.usage_updated_at is not Noneassert capture.called is error_trackedif compile_error is not None:reconcile_aliases.assert_not_awaited()diff --git a/products/data_modeling/backend/logic/usage.py b/products/data_modeling/backend/logic/usage.pyindex e77f358b6fb6..08dbdd9d7dbb 100644--- a/products/data_modeling/backend/logic/usage.py+++ b/products/data_modeling/backend/logic/usage.py@@ -1,4 +1,5 @@from datetime import UTC, date, datetime, time, timedelta+from typing import TypedDict, castfrom uuid import UUIDfrom django.db.models import Count, F, Max, Q, QuerySet, Sum@@ -11,60 +12,120 @@ from products.data_modeling.backend.models.data_modeling_job import ()+class EngineUsage(TypedDict):+ engine: DataModelingJobEngine+ runs: int+ measured_runs: int+ read_bytes: int | None+ read_rows: int | None+ cpu_microseconds: int | None+ query_count: int | None++class ModelUsage:+ USAGE_FIELDS = ("runs", "measured_runs", "read_bytes", "read_rows", "cpu_microseconds", "query_count")+ COUNT_USAGE_FIELDS = ("runs", "measured_runs")+@staticmethod- def jobs(team_id: int, start: datetime, end: datetime) -> QuerySet[DataModelingJob]:+ def jobs(+ team_id: int, start: datetime, end: datetime, engines: tuple[DataModelingJobEngine, ...]+ ) -> QuerySet[DataModelingJob]:return DataModelingJob.objects.filter(team_id=team_id,- engine=DataModelingJobEngine.CLICKHOUSE,+ engine__in=engines,status__in=[DataModelingJobStatus.COMPLETED, DataModelingJobStatus.FAILED, DataModelingJobStatus.CANCELLED],last_run_at__gte=start,last_run_at__lt=end,)@staticmethod- def aggregates() -> dict[str, Count | Sum]:+ def aggregates(*, prefix: str = "", engine: DataModelingJobEngine | None = None) -> dict[str, Count | Sum]:+ engine_filter = Q(engine=engine) if engine else Q()+ measured_filter = Q(usage_updated_at__isnull=False)+ return {+ f"{prefix}runs": Count("id", filter=engine_filter),+ f"{prefix}measured_runs": Count("id", filter=engine_filter & measured_filter),+ f"{prefix}read_bytes": Sum("read_bytes", filter=engine_filter),+ f"{prefix}read_rows": Sum("read_rows", filter=engine_filter),+ f"{prefix}cpu_microseconds": Sum("cpu_microseconds", filter=engine_filter),+ f"{prefix}query_count": Sum("query_count", filter=engine_filter),+ }++ @classmethod+ def engine_aggregates(cls, engines: tuple[DataModelingJobEngine, ...]) -> dict[str, Count | Sum]:return {- "runs": Count("id"),- "measured_runs": Count("id", filter=Q(usage_updated_at__isnull=False)),- "read_bytes": Sum("read_bytes"),- "read_rows": Sum("read_rows"),- "cpu_microseconds": Sum("cpu_microseconds"),- "query_count": Sum("query_count"),+ key: aggregate+ for engine in engines+ for key, aggregate in cls.aggregates(prefix=f"{engine}_", engine=engine).items()}@classmethod- def get(cls, team_id: int, saved_query_ids: list[UUID], days: int, page: int, page_size: int) -> dict[str, object]:- end = datetime.combine(datetime.now(UTC).date(), time.min, tzinfo=UTC)- start = end - timedelta(days=days)- jobs = cls.jobs(team_id, start, end).filter(saved_query_id__in=saved_query_ids)+ def engine_usage(cls, row: dict[str, object], engines: tuple[DataModelingJobEngine, ...]) -> list[EngineUsage]:+ return [+ EngineUsage(+ engine=engine,+ runs=cast(int, row[f"{engine}_runs"]),+ measured_runs=cast(int, row[f"{engine}_measured_runs"]),+ read_bytes=cast(int | None, row[f"{engine}_read_bytes"]),+ read_rows=cast(int | None, row[f"{engine}_read_rows"]),+ cpu_microseconds=cast(int | None, row[f"{engine}_cpu_microseconds"]),+ query_count=cast(int | None, row[f"{engine}_query_count"]),+ )+ for engine in engines+ ]++ @classmethod+ def empty_usage(cls, prefix: str = "") -> dict[str, int | None]:+ return {f"{prefix}{field}": 0 if field in cls.COUNT_USAGE_FIELDS else None for field in cls.USAGE_FIELDS}++ @classmethod+ def get(+ cls,+ team_id: int,+ saved_query_ids: list[UUID],+ page: int,+ page_size: int,+ start_date: date,+ end_date: date,+ engines: tuple[DataModelingJobEngine, ...],+ ) -> dict[str, object]:+ start = datetime.combine(start_date, time.min, tzinfo=UTC)+ end = datetime.combine(end_date, time.min, tzinfo=UTC)+ range_days = (end_date - start_date).days+ jobs = cls.jobs(team_id, start, end, engines).filter(saved_query_id__in=saved_query_ids)overview = jobs.aggregate(**cls.aggregates())usage_updated_at = jobs.aggregate(latest=Max("usage_updated_at"))["latest"]daily = {row["day"]: rowfor row in jobs.annotate(day=TruncDate("last_run_at", tzinfo=UTC)).values("day")- .annotate(**cls.aggregates())+ .annotate(**cls.engine_aggregates(engines), **cls.aggregates()).order_by("day")}trends = []- for offset in range(days):+ for offset in range(range_days):day: date = (start + timedelta(days=offset)).date()- trends.append(- daily.get(- day,- {- "day": day,- "runs": 0,- "measured_runs": 0,- "read_bytes": None,- "read_rows": None,- "cpu_microseconds": None,- "query_count": None,+ row = daily.get(+ day,+ {+ "day": day,+ **cls.empty_usage(),+ **{+ key: value+ for selected_engine in engines+ for key, value in cls.empty_usage(f"{selected_engine}_").items()},- )+ },)- models = jobs.values("saved_query_id", "saved_query__name", "saved_query__deleted").annotate(**cls.aggregates())+ trends.append(+ {+ **{field: row[field] for field in ("day", *cls.USAGE_FIELDS)},+ "engines": cls.engine_usage(row, engines),+ }+ )+ models = jobs.values("saved_query_id", "saved_query__name", "saved_query__deleted").annotate(+ **cls.engine_aggregates(engines), **cls.aggregates()+ )count = models.count()offset = (page - 1) * page_sizerows = list(@@ -80,7 +141,8 @@ class ModelUsage:"saved_query_id": row["saved_query_id"],"name": row["saved_query__name"] or "Deleted model","deleted": row["saved_query__deleted"] is not False,- **{key: row[key] for key in cls.aggregates()},+ **{key: row[key] for key in cls.USAGE_FIELDS},+ "engines": cls.engine_usage(row, engines),}for row in rows],@@ -90,6 +152,20 @@ class ModelUsage:def get_model_usage(- team_id: int, saved_query_ids: list[UUID], days: int, page: int, page_size: int+ team_id: int,+ saved_query_ids: list[UUID],+ page: int,+ page_size: int,+ start_date: date,+ end_date: date,+ engines: tuple[DataModelingJobEngine, ...],) -> dict[str, object]:- return ModelUsage.get(team_id, saved_query_ids, days, page, page_size)+ return ModelUsage.get(+ team_id=team_id,+ saved_query_ids=saved_query_ids,+ page=page,+ page_size=page_size,+ start_date=start_date,+ end_date=end_date,+ engines=engines,+ )diff --git a/products/data_modeling/backend/migrations/0056_datamodelingjob_untrack_bytes_read.py b/products/data_modeling/backend/migrations/0056_datamodelingjob_untrack_bytes_read.pynew file mode 100644index 000000000000..72d50d43f616--- /dev/null+++ b/products/data_modeling/backend/migrations/0056_datamodelingjob_untrack_bytes_read.py@@ -0,0 +1,37 @@+from datetime import UTC, datetime++from django.db import migrations+from django.db.models import F++from posthog.migration_helpers import untrack_field++BATCH_SIZE = 1_000+++def copy_bytes_read(apps, schema_editor):+ DataModelingJob = apps.get_model("data_modeling", "DataModelingJob")+ # Only managed warehouse runs wrote bytes_read, and only after it shipped in October 2026.+ # Those runs set last_run_at in the same save, so the bound uses dmjob_usage_run+ # instead of scanning the whole table.+ pending = DataModelingJob.objects.filter(+ last_run_at__gte=datetime(2026, 10, 1, tzinfo=UTC),+ bytes_read__isnull=False,+ read_bytes__isnull=True,+ )+ ids = list(pending.values_list("id", flat=True))+ for start in range(0, len(ids), BATCH_SIZE):+ DataModelingJob.objects.filter(id__in=ids[start : start + BATCH_SIZE]).update(+ read_bytes=F("bytes_read"),+ usage_updated_at=F("last_run_at"),+ )+++class Migration(migrations.Migration):+ dependencies = [+ ("data_modeling", "0055_backfill_datawarehousesavedquery_query_updated_at"),+ ]++ operations = [+ migrations.RunPython(copy_bytes_read, migrations.RunPython.noop),+ untrack_field("datamodelingjob", "bytes_read"),+ ]diff --git a/products/data_modeling/backend/migrations/max_migration.txt b/products/data_modeling/backend/migrations/max_migration.txtindex 73cbbf372b42..4c81d35595f0 100644--- a/products/data_modeling/backend/migrations/max_migration.txt+++ b/products/data_modeling/backend/migrations/max_migration.txt@@ -1 +1 @@-0055_backfill_datawarehousesavedquery_query_updated_at+0056_datamodelingjob_untrack_bytes_readdiff --git a/products/data_modeling/backend/models/data_modeling_job.py b/products/data_modeling/backend/models/data_modeling_job.pyindex 3bfad784b9c8..22100bfeb108 100644--- a/products/data_modeling/backend/models/data_modeling_job.py+++ b/products/data_modeling/backend/models/data_modeling_job.py@@ -55,11 +55,6 @@ class DataModelingJob(CreatedMetaFields, UpdatedMetaFields, UUIDTModel):last_run_at = models.DateTimeField(default=timezone.now)rows_expected = models.IntegerField(null=True, blank=True, help_text="Total rows expected to be materialized")storage_delta_mib = models.FloatField(null=True, blank=True, default=0)- bytes_read = models.BigIntegerField(- null=True,- blank=True,- help_text="Bytes the engine read to build the model. Null when the run was not measured.",- )read_bytes = models.BigIntegerField(null=True, blank=True, help_text="Bytes read by this run, including probes and retries.")diff --git a/products/data_modeling/backend/presentation/views/usage.py b/products/data_modeling/backend/presentation/views/usage.pyindex 3924b3b39476..6cce7099459c 100644--- a/products/data_modeling/backend/presentation/views/usage.py+++ b/products/data_modeling/backend/presentation/views/usage.py@@ -1,3 +1,7 @@+from datetime import UTC, datetime, timedelta++from django.db import models+from drf_spectacular.helpers import forced_singular_serializerfrom drf_spectacular.utils import OpenApiResponsefrom rest_framework import response, serializers, viewsets@@ -6,23 +10,65 @@ from posthog.api.mixins import ValidatedRequest, validated_requestfrom posthog.api.routing import TeamAndOrgViewSetMixinfrom products.data_modeling.backend.facade.api import get_model_usage-from products.data_modeling.backend.facade.models import DataWarehouseSavedQuery+from products.data_modeling.backend.facade.models import DataModelingJobEngine, DataWarehouseSavedQuery+++class ModelUsageEngine(models.TextChoices):+ ALL = "all", "All"+ CLICKHOUSE = "clickhouse", "ClickHouse"+ MANAGED_WAREHOUSE = "managed_warehouse", "Managed warehouse"class ModelUsageQuerySerializer(serializers.Serializer):days = serializers.IntegerField(- default=30, min_value=1, max_value=90, help_text="Number of complete UTC days, up to 90."+ required=False, min_value=1, max_value=90, help_text="Number of complete UTC days, up to 90."+ )+ date_from = serializers.DateField(+ required=False, help_text="Inclusive UTC start date for a custom range, up to 366 complete days."+ )+ date_to = serializers.DateField(+ required=False, help_text="Inclusive UTC end date for a custom range. Today is always excluded.")page = serializers.IntegerField(default=1, min_value=1, help_text="Model breakdown page, starting at 1.")page_size = serializers.IntegerField(default=20, min_value=1, max_value=100, help_text="Models per page, up to 100.")+ engine = serializers.ChoiceField(+ choices=ModelUsageEngine.choices,+ default=ModelUsageEngine.CLICKHOUSE,+ help_text="Materialization engine. All combines ClickHouse and managed warehouse usage.",+ )+ saved_query_id = serializers.UUIDField(+ required=False, allow_null=True, help_text="Limit usage to one accessible materialized view."+ )++ def validate(self, attrs: dict) -> dict:+ days = attrs.get("days")+ date_from = attrs.get("date_from")+ date_to = attrs.get("date_to")+ if days is not None and date_from is not None:+ raise serializers.ValidationError("Use days or a custom date range, not both.")+ if date_to is not None and date_from is None:+ raise serializers.ValidationError({"date_from": "A start date is required for a custom date range."})+ today = datetime.now(UTC).date()+ if date_from is None:+ end_date = today+ attrs["start_date"] = end_date - timedelta(days=days or 30)+ attrs["end_date"] = end_date+ return attrs++ exclusive_date_to = min(date_to, today - timedelta(days=1)) + timedelta(days=1) if date_to else today+ if date_from >= exclusive_date_to:+ raise serializers.ValidationError({"date_from": "The range must include at least one complete UTC day."})+ if (exclusive_date_to - date_from).days > 366:+ raise serializers.ValidationError({"date_from": "The custom date range cannot exceed 366 days."})+ attrs["start_date"] = date_from+ attrs["end_date"] = exclusive_date_to+ return attrsclass ModelUsageTotalsSerializer(serializers.Serializer):- runs = serializers.IntegerField(- help_text="Finished ClickHouse materialization runs, including failures and cancellations."- )+ runs = serializers.IntegerField(help_text="Finished materialization runs, including failures and cancellations.")measured_runs = serializers.IntegerField(help_text="Runs with recorded usage. Other runs have unknown cost.")read_bytes = serializers.IntegerField(allow_null=True, help_text="Measured bytes read, including probes and retries. Null if unmeasured."@@ -36,14 +82,20 @@ class ModelUsageTotalsSerializer(serializers.Serializer):)+class ModelUsageEngineSerializer(ModelUsageTotalsSerializer):+ engine = serializers.CharField(help_text="Materialization engine: clickhouse or managed_warehouse.")++class ModelUsageDaySerializer(ModelUsageTotalsSerializer):day = serializers.DateField(help_text="UTC day on which the runs finished.")+ engines = ModelUsageEngineSerializer(many=True, help_text="Usage totals for each selected engine.")class ModelUsageRowSerializer(ModelUsageTotalsSerializer):saved_query_id = serializers.UUIDField(allow_null=True, help_text="Model ID, or null if permanently deleted.")name = serializers.CharField(help_text="Model name.")deleted = serializers.BooleanField(help_text="Whether this model has been deleted.")+ engines = ModelUsageEngineSerializer(many=True, help_text="Usage totals for each selected engine.")class ModelUsageSerializer(serializers.Serializer):@@ -69,12 +121,26 @@ class ModelUsageViewSet(TeamAndOrgViewSetMixin, viewsets.GenericViewSet):responses={200: OpenApiResponse(response=forced_singular_serializer(ModelUsageSerializer))},)def list(self, request: ValidatedRequest, **kwargs: object) -> response.Response:+ query_data = request.validated_query_data+ saved_queries = self.get_queryset()+ if saved_query_id := query_data.get("saved_query_id"):+ saved_queries = saved_queries.filter(id=saved_query_id)+ selected_engine = query_data["engine"]+ engines = (+ (DataModelingJobEngine.CLICKHOUSE, DataModelingJobEngine.MANAGED_WAREHOUSE)+ if selected_engine == ModelUsageEngine.ALL+ else (DataModelingJobEngine(selected_engine),)+ )return response.Response(ModelUsageSerializer(get_model_usage(self.team_id,- saved_query_ids=list(self.get_queryset().values_list("id", flat=True)),- **request.validated_query_data,+ saved_query_ids=list(saved_queries.values_list("id", flat=True)),+ page=query_data["page"],+ page_size=query_data["page_size"],+ start_date=query_data["start_date"],+ end_date=query_data["end_date"],+ engines=engines,)).data)diff --git a/products/data_modeling/backend/test/test_job_usage.py b/products/data_modeling/backend/test/test_job_usage.pyindex cb4b86cc27e0..3331446027b7 100644--- a/products/data_modeling/backend/test/test_job_usage.py+++ b/products/data_modeling/backend/test/test_job_usage.py@@ -8,7 +8,7 @@ from posthog.clickhouse.client import sync_executefrom posthog.clickhouse.cluster import ClickhouseCluster, ConnectionInfofrom posthog.clickhouse.workload import Workload-from products.data_modeling.backend.models.data_modeling_job import DataModelingJob+from products.data_modeling.backend.models.data_modeling_job import DataModelingJob, DataModelingJobEnginefrom products.data_modeling.dags.job_usage import USAGE_SQL, JobUsageCollector@@ -77,6 +77,15 @@ class TestJobUsageCollector(BaseTest):DataModelingJob.objects.create(**{**base, "workflow_run_id": "duplicate", "last_run_at": finished - timedelta(days=1)})+ # Managed warehouse runs record their own read_bytes, so the archive must not overwrite them.+ managed = DataModelingJob.objects.create(+ **{+ **base,+ "workflow_run_id": "managed",+ "engine": DataModelingJobEngine.MANAGED_WAREHOUSE,+ "read_bytes": 345,+ }+ )DataModelingJob.objects.filter(team=self.team).update(created_at=finished - timedelta(hours=1))bootstrap_client = MagicMock()bootstrap_client.execute.return_value = [("ops.example.com", 9000, 1, 1, "online", "ops")]@@ -87,6 +96,10 @@ class TestJobUsageCollector(BaseTest):make_pool.return_value.get_client.return_value.__enter__.return_value = clientJobUsageCollector.collect(cluster, finished.date(), {})JobUsageCollector.collect(cluster, finished.date(), {})+ sent = {key for call in client.execute.call_args_list for key in call.kwargs["external_tables"][0]["data"]}+ self.assertNotIn((self.team.id, "example-refresh", "managed"), sent)+ managed.refresh_from_db()+ self.assertEqual(managed.read_bytes, 345)measured.refresh_from_db()missing.refresh_from_db()duplicate.refresh_from_db()diff --git a/products/data_modeling/backend/tests/api/test_usage.py b/products/data_modeling/backend/tests/api/test_usage.pyindex 7e02da75ada3..df5083fb2513 100644--- a/products/data_modeling/backend/tests/api/test_usage.py+++ b/products/data_modeling/backend/tests/api/test_usage.py@@ -22,6 +22,43 @@ class TestModelUsageValidation(SimpleTestCase):self.assertFalse(serializer.is_valid())self.assertIn(field, serializer.errors)+ def test_rejects_unknown_engine(self) -> None:+ serializer = ModelUsageQuerySerializer(data={"engine": "duckgres"})+ self.assertFalse(serializer.is_valid())+ self.assertIn("engine", serializer.errors)++ @parameterized.expand(+ [+ ({"days": 7, "date_from": "2026-10-01"},),+ ({"date_to": "2026-10-10"},),+ ({"date_from": "2026-10-10", "date_to": "2026-10-01"},),+ ({"date_from": "2025-10-01", "date_to": "2026-10-02"},),+ ]+ )+ @time_machine.travel("2026-10-16T12:00:00Z", tick=False)+ def test_rejects_invalid_custom_date_range(self, query: dict[str, object]) -> None:+ serializer = ModelUsageQuerySerializer(data=query)+ self.assertFalse(serializer.is_valid())++ @time_machine.travel("2026-10-16T12:00:00Z", tick=False)+ def test_accepts_custom_date_range(self) -> None:+ serializer = ModelUsageQuerySerializer(data={"date_from": "2026-10-01", "date_to": "2026-10-15"})+ self.assertTrue(serializer.is_valid(), serializer.errors)+ self.assertEqual(serializer.validated_data["start_date"].isoformat(), "2026-10-01")+ self.assertEqual(serializer.validated_data["end_date"].isoformat(), "2026-10-16")++ @time_machine.travel("2026-10-16T12:00:00Z", tick=False)+ def test_resolves_relative_date_range(self) -> None:+ serializer = ModelUsageQuerySerializer(data={"days": 7})+ self.assertTrue(serializer.is_valid(), serializer.errors)+ self.assertEqual(serializer.validated_data["start_date"].isoformat(), "2026-10-09")+ self.assertEqual(serializer.validated_data["end_date"].isoformat(), "2026-10-16")++ @time_machine.travel("2026-10-16T12:00:00Z", tick=False)+ def test_accepts_maximum_end_date_without_overflow(self) -> None:+ serializer = ModelUsageQuerySerializer(data={"date_from": "2026-10-01", "date_to": "9999-12-31"})+ self.assertTrue(serializer.is_valid(), serializer.errors)+@time_machine.travel("2026-10-16T12:00:00Z", tick=False)class TestModelUsage(WarehouseAccessControlTestMixin):@@ -43,7 +80,9 @@ class TestModelUsage(WarehouseAccessControlTestMixin):DataModelingJob.objects.create(**base, **measured)DataModelingJob.objects.create(**{**base, "status": "Failed"})DataModelingJob.objects.create(**{**base, "status": "Running"}, **measured)- DataModelingJob.objects.create(**{**base, "engine": "managed_warehouse"}, **measured)+ managed = DataModelingJob.objects.create(+ **{**base, "engine": "managed_warehouse"}, read_bytes=4096, usage_updated_at=finished+ )DataModelingJob.objects.create(**{**base, "last_run_at": finished + timedelta(days=1)}, **measured)other_team = Team.objects.create(organization=self.organization)DataModelingJob.objects.create(team=other_team, status="Completed", last_run_at=finished, **measured)@@ -64,8 +103,37 @@ class TestModelUsage(WarehouseAccessControlTestMixin):self.assertEqual(len(data["trends"]), 7)self.assertIsNone(data["trends"][0]["read_bytes"])self.assertEqual(data["trends"][-1]["runs"], 2)+ self.assertEqual(data["trends"][-1]["engines"][0]["engine"], "clickhouse")self.assertEqual(data["models"][0]["name"], "daily_orders")self.assertEqual(data["count"], 1)++ combined = self.client.get(+ f"/api/projects/{self.team.id}/data_modeling_usage/", {"days": 7, "engine": "all"}+ ).json()+ self.assertEqual(combined["overview"]["runs"], 3)+ self.assertEqual(combined["overview"]["measured_runs"], 2)+ self.assertEqual(combined["overview"]["read_bytes"], 5120)+ self.assertEqual(+ [(item["engine"], item["runs"]) for item in combined["trends"][-1]["engines"]],+ [("clickhouse", 2), ("managed_warehouse", 1)],+ )+ self.assertEqual(+ [(item["engine"], item["read_bytes"]) for item in combined["models"][0]["engines"]],+ [("clickhouse", 1024), ("managed_warehouse", managed.read_bytes)],+ )++ managed_only = self.client.get(+ f"/api/projects/{self.team.id}/data_modeling_usage/", {"days": 7, "engine": "managed_warehouse"}+ ).json()+ self.assertEqual(managed_only["overview"]["runs"], 1)+ self.assertEqual(managed_only["overview"]["read_bytes"], 4096)+ self.assertEqual([item["engine"] for item in managed_only["models"][0]["engines"]], ["managed_warehouse"])++ filtered = self.client.get(+ f"/api/projects/{self.team.id}/data_modeling_usage/",+ {"days": 7, "engine": "all", "saved_query_id": model.id},+ ).json()+ self.assertEqual(filtered["count"], 1)invalid = self.client.get(f"/api/projects/{self.team.id}/data_modeling_usage/", {"days": 91})self.assertEqual(invalid.status_code, 400)@@ -102,6 +170,28 @@ class TestModelUsage(WarehouseAccessControlTestMixin):self.assertEqual([data["overview"]["read_bytes"] for data in results], [2000] * 4)self.assertEqual([data["count"] for data in results], [4] * 4)+ def test_custom_date_range_is_inclusive_and_excludes_today(self) -> None:+ model = DataWarehouseSavedQuery.objects.create(team=self.team, name="custom_range")+ for day in [9, 10, 12, 13, 16]:+ DataModelingJob.objects.create(+ team=self.team,+ saved_query=model,+ status="Completed",+ last_run_at=datetime(2026, 10, day, 12, tzinfo=UTC),+ )++ result = self.client.get(+ f"/api/projects/{self.team.id}/data_modeling_usage/",+ {"date_from": "2026-10-10", "date_to": "2026-10-12"},+ )++ self.assertEqual(result.status_code, 200, result.content)+ data = result.json()+ self.assertEqual(data["date_from"], "2026-10-10")+ self.assertEqual(data["date_to"], "2026-10-13")+ self.assertEqual(data["overview"]["runs"], 2)+ self.assertEqual([day["runs"] for day in data["trends"]], [1, 0, 1])+def test_denies_warehouse_access(self) -> None:self._create_project_default(access_level="none")self.client.force_login(self.no_access_user)diff --git a/products/data_modeling/frontend/MaterializationJobsTable.tsx b/products/data_modeling/frontend/MaterializationJobsTable.tsxindex b658b27d6c86..a63606e22681 100644--- a/products/data_modeling/frontend/MaterializationJobsTable.tsx+++ b/products/data_modeling/frontend/MaterializationJobsTable.tsx@@ -1,5 +1,4 @@import { useActions, useValues } from 'kea'-import { useMemo, useRef } from 'react'import { TZLabel } from 'lib/components/TZLabel'import { LinkPrimitive } from 'lib/lemon-ui/Link/Link'@@ -12,13 +11,6 @@ import {CardFooter,CardHeader,CardTitle,- Combobox,- ComboboxContent,- ComboboxEmpty,- ComboboxInput,- ComboboxItem,- ComboboxList,- ComboboxTrigger,Pagination,PaginationButton,PaginationContent,@@ -26,6 +18,11 @@ import {PaginationItem,PaginationNext,PaginationPrevious,+ Select,+ SelectContent,+ SelectItem,+ SelectTrigger,+ SelectValue,Spinner,Table,TableBody,@@ -42,11 +39,9 @@ import { pluralize } from 'lib/utils/strings'import { computeJobDuration } from 'scenes/data-warehouse/saved_queries/materializationJobUtils'import { urls } from 'scenes/urls'-import type { DataModelingJobApi } from 'products/data_warehouse/frontend/generated/api.schemas'+import { DataModelingJobsListStatus } from 'products/data_warehouse/frontend/generated/api.schemas'-import { MATERIALIZATION_JOBS_PAGE_SIZE, MaterializedViewOption, modelsUsageLogic } from './modelsUsageLogic'--const ALL_VIEWS: MaterializedViewOption = { id: 'all', name: 'All materialized views' }+import { MATERIALIZATION_ENGINE_LABELS, MATERIALIZATION_JOBS_PAGE_SIZE, modelsUsageLogic } from './modelsUsageLogic'const STATUS_VARIANTS: Record<string, 'success' | 'destructive' | 'warning' | 'default'> = {Completed: 'success',@@ -56,38 +51,14 @@ const STATUS_VARIANTS: Record<string, 'success' | 'destructive' | 'warning' | 'dSkipped: 'default',}-function engineLabel(engine: DataModelingJobApi['engine']): string {- if (engine === 'managed_warehouse') {- return 'Managed warehouse'- }- if (engine === 'clickhouse') {- return 'ClickHouse'- }- if (engine === 'duckgres') {- return 'Duckgres'- }- return engine-}--// Jobs can belong to models the saved-query list omits, such as endpoint materializations-// and soft-deleted models, so an unresolved ID is not necessarily a deleted model.-const UNAVAILABLE_MODEL = 'Unavailable model'--export function MaterializationJobsTable(): JSX.Element {- const {- materializationJobs,- jobsLoading,- jobsFailed,- jobsPage,- selectedSavedQueryId,- materializedViewOptions,- savedQueryNameById,- managedWarehouseEnabled,- } = useValues(modelsUsageLogic)- const { refreshJobs, setJobsPage, setSelectedSavedQueryId, openModel } = useActions(modelsUsageLogic)- const triggerRef = useRef<HTMLButtonElement>(null)- const options = useMemo(() => [ALL_VIEWS, ...materializedViewOptions], [materializedViewOptions])- const selected = options.find((option) => option.id === (selectedSavedQueryId ?? 'all')) ?? ALL_VIEWS+export function MaterializationJobsTable({+ savedQueryIdToNodeId,+}: {+ savedQueryIdToNodeId: Record<string, string>+}): JSX.Element {+ const { materializationJobs, jobsLoading, jobsFailed, jobsPage, managedWarehouseEnabled, selectedJobStatus } =+ useValues(modelsUsageLogic)+ const { refreshJobs, setJobsPage, setSelectedJobStatus, openModel } = useActions(modelsUsageLogic)const count = materializationJobs?.count ?? 0const jobs = materializationJobs?.results ?? []const pageCount = Math.max(Math.ceil(count / MATERIALIZATION_JOBS_PAGE_SIZE), 1)@@ -98,41 +69,35 @@ export function MaterializationJobsTable(): JSX.Element {return (<Card size="sm" className="min-w-0 gap-0">- <CardHeader className="flex flex-row flex-wrap items-start justify-between gap-3 border-b border-border pb-3">+ <CardHeader className="flex-row flex-wrap items-center justify-between gap-3 border-b border-border pb-3"><div className="min-w-0"><CardTitle role="heading" aria-level={3}>Materialization jobs</CardTitle>- <CardDescription>Every model refresh, newest first.</CardDescription>+ <CardDescription>Model refreshes since the selected start date, newest first.</CardDescription></div>- <Combobox- items={options}- value={selected}- itemToStringValue={(option: MaterializedViewOption) => option.name}- isItemEqualToValue={(option, value) => option.id === value.id}- onValueChange={(option: MaterializedViewOption | null) =>- setSelectedSavedQueryId(!option || option.id === 'all' ? null : option.id)+ <Select+ value={selectedJobStatus ?? 'all'}+ onValueChange={(value) =>+ value && setSelectedJobStatus(value === 'all' ? null : (value as DataModelingJobsListStatus))}>- <ComboboxTrigger- ref={triggerRef}- render={<Button variant="outline" className="max-w-72 justify-between" />}- data-attr="models-usage-jobs-view-filter"- >- <span className="truncate">{selected.name}</span>- </ComboboxTrigger>- <ComboboxContent anchor={triggerRef} align="end" className="w-80 max-w-[calc(100vw-2rem)]">- <ComboboxInput placeholder="Search materialized views" showTrigger={false} />- <ComboboxEmpty>No materialized views found.</ComboboxEmpty>- <ComboboxList>- {(option: MaterializedViewOption) => (- <ComboboxItem key={option.id} value={option} title={option.name}>- <span className="truncate">{option.name}</span>- </ComboboxItem>- )}- </ComboboxList>- </ComboboxContent>- </Combobox>+ <SelectTrigger data-attr="models-usage-job-status-filter" disabled={jobsLoading}>+ <SelectValue>+ {(value: string) =>+ value === 'all' ? 'All statuses' : value === 'Cancelled' ? 'Canceled' : value+ }+ </SelectValue>+ </SelectTrigger>+ <SelectContent align="end" alignItemWithTrigger={false}>+ <SelectItem value="all">All statuses</SelectItem>+ <SelectItem value={DataModelingJobsListStatus.Completed}>Completed</SelectItem>+ <SelectItem value={DataModelingJobsListStatus.Running}>Running</SelectItem>+ <SelectItem value={DataModelingJobsListStatus.Failed}>Failed</SelectItem>+ <SelectItem value={DataModelingJobsListStatus.Cancelled}>Canceled</SelectItem>+ <SelectItem value={DataModelingJobsListStatus.Skipped}>Skipped</SelectItem>+ </SelectContent>+ </Select></CardHeader>{jobsFailed ? (<CardContent className="flex items-center justify-between gap-3 py-6">@@ -162,16 +127,17 @@ export function MaterializationJobsTable(): JSX.Element {{jobs.length > 0 ? (<TableBody>{jobs.map((job) => {- const modelName = job.saved_query_id- ? (savedQueryNameById[job.saved_query_id] ?? UNAVAILABLE_MODEL)- : 'Deleted model'+ const modelName = job.saved_query_name ?? 'Deleted model'+ const nodeId = job.saved_query_id+ ? savedQueryIdToNodeId[job.saved_query_id]+ : undefinedreturn (<TableRow key={job.id}><TableCell expand className="min-w-40 max-w-80">- {job.saved_query_id && savedQueryNameById[job.saved_query_id] ? (+ {nodeId && !job.saved_query_deleted ? (<LinkPrimitiveclassName="block truncate font-medium text-link-primary hover:underline"- to={urls.sqlEditor({ view_id: job.saved_query_id })}+ to={urls.nodeDetail(nodeId)}onClick={openModel}title={modelName}>@@ -185,6 +151,7 @@ export function MaterializationJobsTable(): JSX.Element {title={modelName}>{modelName}+ {job.saved_query_deleted ? ' (deleted)' : ''}</Text>)}</TableCell>@@ -195,7 +162,7 @@ export function MaterializationJobsTable(): JSX.Element {</TableCell>{managedWarehouseEnabled ? (<TableCell className="whitespace-nowrap">- {engineLabel(job.engine)}+ {MATERIALIZATION_ENGINE_LABELS[job.engine] ?? job.engine}</TableCell>) : null}<TableCell className="hidden whitespace-nowrap @min-[48rem]/main-content:table-cell">@@ -234,7 +201,7 @@ export function MaterializationJobsTable(): JSX.Element {</TableBody>) : (<TableEmpty className="py-8 text-secondary">- No materialization jobs match this view.+ No materialization jobs match these filters.</TableEmpty>)}</Table>diff --git a/products/data_modeling/frontend/ModelsScene.tsx b/products/data_modeling/frontend/ModelsScene.tsxindex aed0700b5da2..6f3674da63c7 100644--- a/products/data_modeling/frontend/ModelsScene.tsx+++ b/products/data_modeling/frontend/ModelsScene.tsx@@ -74,7 +74,7 @@ export function ModelsScene(): JSX.Element {key: 'usage' as const,label: 'Usage',link: urls.models('usage'),- content: <ModelsUsageTab />,+ content: <ModelsUsageTab savedQueryIdToNodeId={savedQueryIdToNodeId} />,'data-attr': 'models-tab-usage',},]diff --git a/products/data_modeling/frontend/ModelsUsageByModelTable.tsx b/products/data_modeling/frontend/ModelsUsageByModelTable.tsxindex 9cc56e719a26..1ad558142697 100644--- a/products/data_modeling/frontend/ModelsUsageByModelTable.tsx+++ b/products/data_modeling/frontend/ModelsUsageByModelTable.tsx@@ -29,12 +29,21 @@ import { humanFriendlyNumber, humanizeBytes } from 'lib/utils/numbers'import { pluralize } from 'lib/utils/strings'import { urls } from 'scenes/urls'+import { DataModelingUsageListEngine, type ModelUsageRowApi } from './generated/api.schemas'import { modelsUsageLogic } from './modelsUsageLogic'const PAGE_SIZE = 20-export function ModelsUsageByModelTable(): JSX.Element | null {- const { usage, usageLoading, page } = useValues(modelsUsageLogic)+function engineBytes(row: ModelUsageRowApi, engine: string): number | null {+ return row.engines.find((item) => item.engine === engine)?.read_bytes ?? null+}++export function ModelsUsageByModelTable({+ savedQueryIdToNodeId,+}: {+ savedQueryIdToNodeId: Record<string, string>+}): JSX.Element | null {+ const { usage, usageLoading, page, managedWarehouseEnabled, selectedEngine } = useValues(modelsUsageLogic)const { setPage, openModel } = useActions(modelsUsageLogic)if (!usage || usage.overview.runs === 0) {@@ -46,6 +55,7 @@ export function ModelsUsageByModelTable(): JSX.Element | null {const pageRange = getPaginationRange(pageCount, pageIndex)const firstRow = usage.count === 0 ? 0 : pageIndex * PAGE_SIZE + 1const lastRow = Math.min(firstRow + usage.models.length - 1, usage.count)+ const showEngineColumns = managedWarehouseEnabled && selectedEngine === DataModelingUsageListEngine.Allreturn (<Card size="sm" className="min-w-0 gap-0">@@ -62,7 +72,14 @@ export function ModelsUsageByModelTable(): JSX.Element | null {<TableHeader><TableRow><TableHead expand>Model</TableHead>- <TableHead align="right">Bytes read</TableHead>+ {showEngineColumns ? (+ <>+ <TableHead align="right">ClickHouse bytes</TableHead>+ <TableHead align="right">Managed warehouse bytes</TableHead>+ </>+ ) : (+ <TableHead align="right">Bytes read</TableHead>+ )}<TableHead align="right" className="hidden @min-[48rem]/main-content:table-cell">CPU time</TableHead>@@ -70,50 +87,78 @@ export function ModelsUsageByModelTable(): JSX.Element | null {</TableRow></TableHeader><TableBody>- {usage.models.map((row) => (- <TableRow key={row.saved_query_id ?? 'deleted'}>- <TableCell expand className="min-w-40 max-w-96 break-all">- {row.deleted || !row.saved_query_id ? (- <Text variant="muted" render={<span />}>- {row.name} (deleted)- </Text>- ) : (- <LinkPrimitive- className="font-medium text-link-primary hover:underline"- to={urls.sqlEditor({ view_id: row.saved_query_id })}- onClick={openModel}- data-attr="models-usage-open-model"- >- {row.name}- </LinkPrimitive>- )}- </TableCell>- <TableCell align="right" className="whitespace-nowrap tabular-nums">- {row.read_bytes === null ? (- <Text variant="muted" render={<span />}>- Not measured- </Text>- ) : (- humanizeBytes(row.read_bytes)- )}- </TableCell>- <TableCell- align="right"- className="hidden whitespace-nowrap tabular-nums @min-[48rem]/main-content:table-cell"- >- {row.cpu_microseconds === null ? (- <Text variant="muted" render={<span />}>- Not measured- </Text>+ {usage.models.map((row) => {+ const nodeId = row.saved_query_id ? savedQueryIdToNodeId[row.saved_query_id] : undefined+ return (+ <TableRow key={row.saved_query_id ?? 'deleted'}>+ <TableCell expand className="min-w-40 max-w-96 break-all">+ {row.deleted || !nodeId ? (+ <Text variant="muted" render={<span />}>+ {row.name}+ {row.deleted ? ' (deleted)' : ''}+ </Text>+ ) : (+ <LinkPrimitive+ className="font-medium text-link-primary hover:underline"+ to={urls.nodeDetail(nodeId)}+ onClick={openModel}+ data-attr="models-usage-open-model"+ >+ {row.name}+ </LinkPrimitive>+ )}+ </TableCell>+ {showEngineColumns ? (+ [+ DataModelingUsageListEngine.Clickhouse,+ DataModelingUsageListEngine.ManagedWarehouse,+ ].map((engine) => {+ const readBytes = engineBytes(row, engine)+ return (+ <TableCell+ key={engine}+ align="right"+ className="whitespace-nowrap tabular-nums"+ >+ {readBytes === null ? (+ <Text variant="muted" render={<span />}>+ Not measured+ </Text>+ ) : (+ humanizeBytes(readBytes)+ )}+ </TableCell>+ )+ })) : (- humanFriendlyDuration(row.cpu_microseconds / 1_000_000)+ <TableCell align="right" className="whitespace-nowrap tabular-nums">+ {row.read_bytes === null ? (+ <Text variant="muted" render={<span />}>+ Not measured+ </Text>+ ) : (+ humanizeBytes(row.read_bytes)+ )}+ </TableCell>)}- </TableCell>- <TableCell align="right" className="whitespace-nowrap tabular-nums">- {humanFriendlyNumber(row.runs)}- </TableCell>- </TableRow>- ))}+ <TableCell+ align="right"+ className="hidden whitespace-nowrap tabular-nums @min-[48rem]/main-content:table-cell"+ >+ {row.cpu_microseconds === null ? (+ <Text variant="muted" render={<span />}>+ Not measured+ </Text>+ ) : (+ humanFriendlyDuration(row.cpu_microseconds / 1_000_000)+ )}+ </TableCell>+ <TableCell align="right" className="whitespace-nowrap tabular-nums">+ {humanFriendlyNumber(row.runs)}+ </TableCell>+ </TableRow>+ )+ })}</TableBody></Table></CardContent>diff --git a/products/data_modeling/frontend/ModelsUsageTab.stories.tsx b/products/data_modeling/frontend/ModelsUsageTab.stories.tsxindex 5fb904f910ee..0ff2870b6e54 100644--- a/products/data_modeling/frontend/ModelsUsageTab.stories.tsx+++ b/products/data_modeling/frontend/ModelsUsageTab.stories.tsx@@ -30,6 +30,10 @@ const models: ModelUsageRowApi[] = [query_count: 240,runs: 120,measured_runs: 118,+ engines: [+ { ...totals, engine: 'clickhouse', read_bytes: 3.8 * 1024 ** 4 },+ { ...totals, engine: 'managed_warehouse', read_bytes: 1.3 * 1024 ** 4 },+ ],},{...totals,@@ -42,6 +46,10 @@ const models: ModelUsageRowApi[] = [query_count: 288,runs: 144,measured_runs: 140,+ engines: [+ { ...totals, engine: 'clickhouse', read_bytes: 1.7 * 1024 ** 4 },+ { ...totals, engine: 'managed_warehouse', read_bytes: 0.7 * 1024 ** 4 },+ ],},{...totals,@@ -54,6 +62,10 @@ const models: ModelUsageRowApi[] = [query_count: 112,runs: 56,measured_runs: 54,+ engines: [+ { ...totals, engine: 'clickhouse', read_bytes: 0.6 * 1024 ** 4 },+ { ...totals, engine: 'managed_warehouse', read_bytes: 0.3 * 1024 ** 4 },+ ],},]const populated: ModelUsageApi = {@@ -63,11 +75,18 @@ const populated: ModelUsageApi = {models,count: 3,usage_updated_at: '2026-10-16T03:00:00Z',- trends: Array.from({ length: 30 }, (_, index) => ({- ...totals,- day: new Date(Date.UTC(2026, 8, 16 + index)).toISOString().slice(0, 10),- read_bytes: (150 + ((index * 71) % 300)) * 1024 ** 3,- })),+ trends: Array.from({ length: 30 }, (_, index) => {+ const readBytes = (150 + ((index * 71) % 300)) * 1024 ** 3+ return {+ ...totals,+ day: new Date(Date.UTC(2026, 8, 16 + index)).toISOString().slice(0, 10),+ read_bytes: readBytes,+ engines: [+ { ...totals, engine: 'clickhouse', read_bytes: readBytes * 0.7 },+ { ...totals, engine: 'managed_warehouse', read_bytes: readBytes * 0.3 },+ ],+ }+ }),}const unknown: ModelUsageTotalsApi = {runs: 8,@@ -83,20 +102,28 @@ const materializedViews = models.map((model) => ({name: model.name,is_materialized: true,}))+const savedQueryIdToNodeId = Object.fromEntries(+ models.map((model, index) => [+ model.saved_query_id!,+ `00000000-0000-4000-9000-${String(index + 1).padStart(12, '0')}`,+ ])+)const jobTemplates: DataModelingJobApi[] = [{id: '10000000-0000-4000-8000-000000000001',saved_query_id: models[0].saved_query_id,+ saved_query_name: models[0].name,+ saved_query_deleted: false,status: 'Completed',engine: 'clickhouse',run_mode: 'incremental',full_refresh_reason: null,rows_materialized: 1_250_000,error: null,- created_at: '2026-10-16T02:00:00Z',- last_run_at: '2026-10-16T02:04:12Z',- updated_at: '2026-10-16T02:04:12Z',+ created_at: '2026-10-15T22:00:00Z',+ last_run_at: '2026-10-15T22:04:12Z',+ updated_at: '2026-10-15T22:04:12Z',workflow_id: 'materialize-example-1',workflow_run_id: 'run-example-1',rows_expected: 1_250_000,@@ -109,15 +136,17 @@ const jobTemplates: DataModelingJobApi[] = [{id: '10000000-0000-4000-8000-000000000002',saved_query_id: models[1].saved_query_id,+ saved_query_name: models[1].name,+ saved_query_deleted: false,status: 'Running',engine: 'managed_warehouse',run_mode: 'full_refresh',full_refresh_reason: 'definition_changed',rows_materialized: 840_000,error: null,- created_at: '2026-10-16T01:30:00Z',- last_run_at: '2026-10-16T01:30:00Z',- updated_at: '2026-10-16T01:30:00Z',+ created_at: '2026-10-15T21:30:00Z',+ last_run_at: '2026-10-15T21:30:00Z',+ updated_at: '2026-10-15T21:30:00Z',workflow_id: 'materialize-example-2',workflow_run_id: 'run-example-2',rows_expected: 2_100_000,@@ -130,15 +159,17 @@ const jobTemplates: DataModelingJobApi[] = [{id: '10000000-0000-4000-8000-000000000003',saved_query_id: models[2].saved_query_id,+ saved_query_name: models[2].name,+ saved_query_deleted: false,status: 'Failed',engine: 'clickhouse',run_mode: null,full_refresh_reason: null,rows_materialized: 0,error: 'Example materialization failure',- created_at: '2026-10-15T23:00:00Z',- last_run_at: '2026-10-15T23:01:30Z',- updated_at: '2026-10-15T23:01:30Z',+ created_at: '2026-10-15T20:00:00Z',+ last_run_at: '2026-10-15T20:01:30Z',+ updated_at: '2026-10-15T20:01:30Z',workflow_id: 'materialize-example-3',workflow_run_id: 'run-example-3',rows_expected: null,@@ -153,7 +184,7 @@ const jobTemplates: DataModelingJobApi[] = [const jobs: DataModelingJobApi[] = Array.from({ length: 23 }, (_, index) => ({...jobTemplates[index % jobTemplates.length],id: `10000000-0000-4000-8000-${String(index + 1).padStart(12, '0')}`,- created_at: new Date(Date.UTC(2026, 9, 16, 2, -index * 15)).toISOString(),+ created_at: new Date(Date.UTC(2026, 9, 15, 22, -index * 15)).toISOString(),}))const inMainContent =@@ -167,6 +198,7 @@ const inMainContent =const meta: Meta<typeof ModelsUsageTab> = {title: 'Products/Data modeling/Models usage',component: ModelsUsageTab,+ args: { savedQueryIdToNodeId },decorators: [mswDecorator({})],parameters: {featureFlags: [FEATURE_FLAGS.DATA_MODELING_USAGE, FEATURE_FLAGS.MANAGED_TRINO_QUERY],@@ -180,9 +212,20 @@ const meta: Meta<typeof ModelsUsageTab> = {const offset = Number(params.get('offset') ?? 0)const limit = Number(params.get('limit') ?? 20)const savedQueryId = params.get('saved_query_id')- const filteredJobs = savedQueryId- ? jobs.filter((job) => job.saved_query_id === savedQueryId)- : jobs+ const engine = params.get('engine')+ const status = params.get('status')+ const after = params.get('after')+ const before = params.get('before')+ const filteredJobs = jobs.filter((job) => {+ const startedAt = new Date(job.created_at)+ return (+ (!savedQueryId || job.saved_query_id === savedQueryId) &&+ (!engine || job.engine === engine) &&+ (!status || job.status === status) &&+ (!after || startedAt >= new Date(after)) &&+ (!before || startedAt < new Date(before))+ )+ })return [200,@@ -225,7 +268,16 @@ export const Unmeasured: Story = {usage_updated_at: null,trends: [],count: 1,- models: [{ ...models[0], ...unknown }],+ models: [+ {+ ...models[0],+ ...unknown,+ engines: [+ { ...unknown, engine: 'clickhouse' },+ { ...unknown, engine: 'managed_warehouse' },+ ],+ },+ ],},},},diff --git a/products/data_modeling/frontend/ModelsUsageTab.tsx b/products/data_modeling/frontend/ModelsUsageTab.tsxindex 07b9965311a8..b4bf344ed149 100644--- a/products/data_modeling/frontend/ModelsUsageTab.tsx+++ b/products/data_modeling/frontend/ModelsUsageTab.tsx@@ -1,9 +1,12 @@import { useActions, useValues } from 'kea'+import { useMemo, useRef } from 'react'import { IconRefresh } from '@posthog/icons'import { TimeSeriesBarChart } from '@posthog/quill-charts'import { useChartConfig, useChartTheme } from 'lib/charts/hooks'+import { DateFilter } from 'lib/components/DateFilter/DateFilter'+import { dayjs } from 'lib/dayjs'import {Button,Card,@@ -11,6 +14,13 @@ import {CardDescription,CardHeader,CardTitle,+ Combobox,+ ComboboxContent,+ ComboboxEmpty,+ ComboboxInput,+ ComboboxItem,+ ComboboxList,+ ComboboxTrigger,Heading,Select,SelectContent,@@ -23,15 +33,22 @@ import {import { humanFriendlyDuration } from 'lib/utils/durations'import { humanFriendlyNumber, humanizeBytes } from 'lib/utils/numbers'+import type { DateMappingOption } from '~/types'++import { DataModelingUsageListEngine } from './generated/api.schemas'import { MaterializationJobsTable } from './MaterializationJobsTable'import { ModelsUsageByModelTable } from './ModelsUsageByModelTable'-import { modelsUsageLogic } from './modelsUsageLogic'+import { MATERIALIZATION_ENGINE_LABELS, MaterializedViewOption, modelsUsageLogic } from './modelsUsageLogic'-const DATE_RANGE_LABELS: Record<string, string> = {- '7': 'Last 7 complete days',- '30': 'Last 30 complete days',- '90': 'Last 90 complete days',-}+const DATE_OPTIONS: DateMappingOption[] = [7, 30, 90].map((days) => ({+ key: `Last ${days} complete days`,+ values: [`-${days}d`],+ getFormattedDate: (date: dayjs.Dayjs): string =>+ `${date.subtract(days, 'day').format('MMM D, YYYY')} – ${date.subtract(1, 'day').format('MMM D, YYYY')}`,+ defaultInterval: 'day',+}))++const ALL_VIEWS: MaterializedViewOption = { id: 'all', name: 'All materialized views' }function MetricTile({ title, value }: { title: string; value: string }): JSX.Element {return (@@ -46,17 +63,40 @@ function MetricTile({ title, value }: { title: string; value: string }): JSX.Ele)}-export function ModelsUsageTab(): JSX.Element {- const { usage, usageLoading, failed, days, chartLabels, chartSeries, jobsLoading } = useValues(modelsUsageLogic)- const { refresh, setDays, loadUsage } = useActions(modelsUsageLogic)+export function ModelsUsageTab({+ savedQueryIdToNodeId,+}: {+ savedQueryIdToNodeId: Record<string, string>+}): JSX.Element {+ const {+ usage,+ usageLoading,+ failed,+ dateFrom,+ dateTo,+ chartLabels,+ chartSeries,+ jobsLoading,+ managedWarehouseEnabled,+ materializedViewOptions,+ selectedEngine,+ selectedSavedQueryId,+ } = useValues(modelsUsageLogic)+ const { refresh, setDateRange, loadUsage, setSelectedEngine, setSelectedSavedQueryId } =+ useActions(modelsUsageLogic)+ const viewFilterRef = useRef<HTMLButtonElement>(null)+ const viewOptions = useMemo(() => [ALL_VIEWS, ...materializedViewOptions], [materializedViewOptions])+ const selectedView = viewOptions.find((option) => option.id === (selectedSavedQueryId ?? ALL_VIEWS.id)) ?? ALL_VIEWSconst theme = useChartTheme()const chartConfig = useChartConfig(() => ({yAxis: { showGrid: true, tickFormatter: (value: number) => humanizeBytes(value) },xAxis: { timezone: 'UTC', interval: 'day' as const },tooltip: { valueFormatter: (value: number) => humanizeBytes(value) },+ barLayout: 'stacked' as const,+ legend: { show: chartSeries.length > 1, position: 'top' as const, interactive: true },}),- []+ [chartSeries.length])return (@@ -69,16 +109,70 @@ export function ModelsUsageTab(): JSX.Element {<Text variant="muted">Find the models and jobs that read the most data when they refresh.</Text></div><div className="flex flex-wrap gap-2">- <Select value={String(days)} onValueChange={(value) => value && setDays(Number(value))}>- <SelectTrigger data-attr="models-usage-date-range" disabled={usageLoading}>- <SelectValue>{(value: string) => DATE_RANGE_LABELS[value] ?? value}</SelectValue>- </SelectTrigger>- <SelectContent align="end" alignItemWithTrigger={false}>- <SelectItem value="7">Last 7 complete days</SelectItem>- <SelectItem value="30">Last 30 complete days</SelectItem>- <SelectItem value="90">Last 90 complete days</SelectItem>- </SelectContent>- </Select>+ <Combobox+ items={viewOptions}+ value={selectedView}+ itemToStringValue={(option: MaterializedViewOption) => option.name}+ isItemEqualToValue={(option, value) => option.id === value.id}+ onValueChange={(option: MaterializedViewOption | null) =>+ setSelectedSavedQueryId(!option || option.id === ALL_VIEWS.id ? null : option.id)+ }+ >+ <ComboboxTrigger+ ref={viewFilterRef}+ render={<Button variant="outline" className="max-w-72 justify-between" />}+ data-attr="models-usage-view-filter"+ disabled={usageLoading || jobsLoading}+ >+ <span className="truncate">{selectedView.name}</span>+ </ComboboxTrigger>+ <ComboboxContent anchor={viewFilterRef} align="end" className="w-80 max-w-[calc(100vw-2rem)]">+ <ComboboxInput placeholder="Search materialized views" showTrigger={false} />+ <ComboboxEmpty>No materialized views found.</ComboboxEmpty>+ <ComboboxList>+ {(option: MaterializedViewOption) => (+ <ComboboxItem key={option.id} value={option} title={option.name}>+ <span className="truncate">{option.name}</span>+ </ComboboxItem>+ )}+ </ComboboxList>+ </ComboboxContent>+ </Combobox>+ {managedWarehouseEnabled ? (+ <Select+ value={selectedEngine}+ onValueChange={(value) => value && setSelectedEngine(value as DataModelingUsageListEngine)}+ >+ <SelectTrigger+ data-attr="models-usage-engine-filter"+ disabled={usageLoading || jobsLoading}+ >+ <SelectValue>+ {(value: DataModelingUsageListEngine) =>+ MATERIALIZATION_ENGINE_LABELS[value] ?? value+ }+ </SelectValue>+ </SelectTrigger>+ <SelectContent align="end" alignItemWithTrigger={false}>+ <SelectItem value={DataModelingUsageListEngine.All}>All engines</SelectItem>+ <SelectItem value={DataModelingUsageListEngine.Clickhouse}>ClickHouse</SelectItem>+ <SelectItem value={DataModelingUsageListEngine.ManagedWarehouse}>+ Managed warehouse+ </SelectItem>+ </SelectContent>+ </Select>+ ) : null}+ <div data-attr="models-usage-date-range">+ <DateFilter+ dateFrom={dateFrom}+ dateTo={dateTo}+ onChange={setDateRange}+ dateOptions={DATE_OPTIONS}+ showRollingRangePicker={false}+ forceGranularity="day"+ disabled={usageLoading || jobsLoading}+ />+ </div><Buttonvariant="outline"onClick={refresh}@@ -91,8 +185,8 @@ export function ModelsUsageTab(): JSX.Element {</div></div><Text size="sm" variant="muted">- Usage updates daily and excludes today. All dates are UTC. Includes failed runs, canceled runs, schema- checks, and retries.+ Usage updates daily and excludes today. Job history starts on the selected start date and includes+ current jobs. All dates are UTC. Usage includes failed runs, canceled runs, schema checks, and retries.</Text>{failed ? (@@ -112,8 +206,7 @@ export function ModelsUsageTab(): JSX.Element {<Card size="sm"><CardContent><Text variant="muted">- No ClickHouse materialization runs finished in this period. Choose a longer date range or- materialize a model to see its usage.+ No materialization runs match these filters. Choose a longer date range or change a filter.</Text></CardContent></Card>@@ -158,6 +251,7 @@ export function ModelsUsageTab(): JSX.Element {</CardTitle><CardDescription>Usage is attributed to the day each run finished. Gaps have no measured usage.+ {chartSeries.length > 1 ? ' Bars are stacked by engine.' : ''}</CardDescription></CardHeader><CardContent>@@ -172,11 +266,11 @@ export function ModelsUsageTab(): JSX.Element {</CardContent></Card>) : null}- <ModelsUsageByModelTable />+ <ModelsUsageByModelTable savedQueryIdToNodeId={savedQueryIdToNodeId} /></>)}- <MaterializationJobsTable />+ <MaterializationJobsTable savedQueryIdToNodeId={savedQueryIdToNodeId} /></div>)}diff --git a/products/data_modeling/frontend/generated/api.schemas.ts b/products/data_modeling/frontend/generated/api.schemas.tsindex 3345ad02139d..d4cb5b9a800f 100644--- a/products/data_modeling/frontend/generated/api.schemas.ts+++ b/products/data_modeling/frontend/generated/api.schemas.ts@@ -267,7 +267,7 @@ export interface LineageResponseApi {}export interface ModelUsageTotalsApi {- /** Finished ClickHouse materialization runs, including failures and cancellations. */+ /** Finished materialization runs, including failures and cancellations. */runs: number/** Runs with recorded usage. Other runs have unknown cost. */measured_runs: number@@ -293,8 +293,37 @@ export interface ModelUsageTotalsApi {query_count: number | null}+export interface ModelUsageEngineApi {+ /** Finished materialization runs, including failures and cancellations. */+ runs: number+ /** Runs with recorded usage. Other runs have unknown cost. */+ measured_runs: number+ /**+ * Measured bytes read, including probes and retries. Null if unmeasured.+ * @nullable+ */+ read_bytes: number | null+ /**+ * Measured rows read. Null if unmeasured.+ * @nullable+ */+ read_rows: number | null+ /**+ * Measured CPU microseconds. Null if unmeasured.+ * @nullable+ */+ cpu_microseconds: number | null+ /**+ * Measured ClickHouse query count. Null if unmeasured.+ * @nullable+ */+ query_count: number | null+ /** Materialization engine: clickhouse or managed_warehouse. */+ engine: string+}+export interface ModelUsageDayApi {- /** Finished ClickHouse materialization runs, including failures and cancellations. */+ /** Finished materialization runs, including failures and cancellations. */runs: number/** Runs with recorded usage. Other runs have unknown cost. */measured_runs: number@@ -320,10 +349,12 @@ export interface ModelUsageDayApi {query_count: number | null/** UTC day on which the runs finished. */day: string+ /** Usage totals for each selected engine. */+ engines: ModelUsageEngineApi[]}export interface ModelUsageRowApi {- /** Finished ClickHouse materialization runs, including failures and cancellations. */+ /** Finished materialization runs, including failures and cancellations. */runs: number/** Runs with recorded usage. Other runs have unknown cost. */measured_runs: number@@ -356,6 +387,8 @@ export interface ModelUsageRowApi {name: string/** Whether this model has been deleted. */deleted: boolean+ /** Usage totals for each selected engine. */+ engines: ModelUsageEngineApi[]}export interface ModelUsageApi {@@ -429,12 +462,29 @@ export type DataModelingNodesLineageRetrieveParams = {}export type DataModelingUsageListParams = {+ /**+ * Inclusive UTC start date for a custom range, up to 366 complete days.+ */+ date_from?: string+ /**+ * Inclusive UTC end date for a custom range. Today is always excluded.+ */+ date_to?: string/*** Number of complete UTC days, up to 90.* @minimum 1* @maximum 90*/days?: number+ /**+ * Materialization engine. All combines ClickHouse and managed warehouse usage.+ *+ * * `all` - All+ * * `clickhouse` - ClickHouse+ * * `managed_warehouse` - Managed warehouse+ * @minLength 1+ */+ engine?: DataModelingUsageListEngine/*** Model breakdown page, starting at 1.* @minimum 1@@ -446,4 +496,17 @@ export type DataModelingUsageListParams = {* @maximum 100*/page_size?: number+ /**+ * Limit usage to one accessible materialized view.+ * @nullable+ */+ saved_query_id?: string | null}++export type DataModelingUsageListEngine = (typeof DataModelingUsageListEngine)[keyof typeof DataModelingUsageListEngine]++export const DataModelingUsageListEngine = {+ All: 'all',+ Clickhouse: 'clickhouse',+ ManagedWarehouse: 'managed_warehouse',+} as constdiff --git a/products/data_modeling/frontend/modelsUsageLogic.test.ts b/products/data_modeling/frontend/modelsUsageLogic.test.tsindex acaa166fea30..65534a61a7a3 100644--- a/products/data_modeling/frontend/modelsUsageLogic.test.ts+++ b/products/data_modeling/frontend/modelsUsageLogic.test.ts@@ -1,12 +1,14 @@import { expectLogic } from 'kea-test-utils'import { FEATURE_FLAGS } from 'lib/constants'+import { dayjs } from 'lib/dayjs'import { featureFlagLogic } from 'lib/logic/featureFlagLogic'import { dataWarehouseViewsLogic } from 'scenes/data-warehouse/saved_queries/dataWarehouseViewsLogic'import { useMocks } from '~/mocks/jest'import { initKeaTests } from '~/test/init'+import { DataModelingUsageListEngine } from './generated/api.schemas'import { modelsUsageLogic } from './modelsUsageLogic'const emptyUsage = {@@ -29,12 +31,17 @@ const emptyUsage = {describe('modelsUsageLogic', () => {let logic: ReturnType<typeof modelsUsageLogic.build>let jobRequests: URLSearchParams[]+ let usageRequests: URLSearchParams[]beforeEach(() => {jobRequests = []+ usageRequests = []useMocks({get: {- '/api/projects/:team_id/data_modeling_usage/': emptyUsage,+ '/api/projects/:team_id/data_modeling_usage/': ({ request }) => {+ usageRequests.push(new URL(request.url).searchParams)+ return [200, emptyUsage]+ },'/api/projects/:team_id/data_modeling_jobs/': ({ request }) => {jobRequests.push(new URL(request.url).searchParams)return [200, { count: 0, next: null, previous: null, results: [] }]@@ -60,17 +67,22 @@ describe('modelsUsageLogic', () => {afterEach(() => logic.unmount())- it('loads all engines and independently filters and pages jobs', async () => {+ it('filters usage and jobs by model across the whole tab', async () => {await expectLogic(logic).toDispatchActions(['loadUsageSuccess', 'loadMaterializationJobsSuccess'])+ expect(usageRequests[0].get('engine')).toBe('all')expect(jobRequests[0].get('all_engines')).toBe('true')+ expect(jobRequests[0].get('engine')).toBeNull()expect(jobRequests[0].get('saved_query_id')).toBeNull()+ expect(jobRequests[0].get('after')).not.toBeNull()+ expect(jobRequests[0].get('before')).toBeNull()expect(logic.values.materializedViewOptions.map((option) => option.name)).toEqual(['Alpha view', 'Beta view'])- expect(logic.values.savedQueryNameById['view-c']).toBe('Former materialized view')await expectLogic(logic, () => logic.actions.setSelectedSavedQueryId('view-a')).toDispatchActions([+ 'loadUsageSuccess','loadMaterializationJobsSuccess',])+ expect(usageRequests.at(-1)?.get('saved_query_id')).toBe('view-a')expect(jobRequests.at(-1)?.get('saved_query_id')).toBe('view-a')expect(jobRequests.at(-1)?.get('offset')).toBe('0')@@ -82,6 +94,64 @@ describe('modelsUsageLogic', () => {expect(logic.values.page).toBe(1)})+ it('filters all tab requests by engine', async () => {+ await expectLogic(logic).toDispatchActions(['loadUsageSuccess', 'loadMaterializationJobsSuccess'])++ await expectLogic(logic, () =>+ logic.actions.setSelectedEngine(DataModelingUsageListEngine.ManagedWarehouse)+ ).toDispatchActions(['loadUsageSuccess', 'loadMaterializationJobsSuccess'])++ expect(usageRequests.at(-1)?.get('engine')).toBe('managed_warehouse')+ expect(jobRequests.at(-1)?.get('all_engines')).toBe('true')+ expect(jobRequests.at(-1)?.get('engine')).toBe('managed_warehouse')+ expect(logic.values.page).toBe(1)+ expect(logic.values.jobsPage).toBe(1)+ })++ it('filters usage and jobs by a custom date range', async () => {+ await expectLogic(logic).toDispatchActions(['loadUsageSuccess', 'loadMaterializationJobsSuccess'])++ await expectLogic(logic, () =>+ logic.actions.setDateRange('2026-10-01T00:00:00', '2026-10-07T23:59:59')+ ).toDispatchActions(['loadUsageSuccess', 'loadMaterializationJobsSuccess'])++ expect(usageRequests.at(-1)?.get('days')).toBeNull()+ expect(usageRequests.at(-1)?.get('date_from')).toBe('2026-10-01')+ expect(usageRequests.at(-1)?.get('date_to')).toBe('2026-10-07')+ expect(jobRequests.at(-1)?.get('after')).toBe('2026-10-01T00:00:00.000Z')+ expect(jobRequests.at(-1)?.get('before')).toBeNull()+ expect(logic.values.page).toBe(1)+ expect(logic.values.jobsPage).toBe(1)+ })++ it('normalizes picker timestamps and limits custom ranges to complete days', async () => {+ await expectLogic(logic).toDispatchActions(['loadUsageSuccess', 'loadMaterializationJobsSuccess'])++ await expectLogic(logic, () =>+ logic.actions.setDateRange('2024-01-01T00:00:00.000Z', '9999-12-31T23:59:59.999Z')+ ).toDispatchActions(['loadUsageSuccess', 'loadMaterializationJobsSuccess'])++ const yesterday = dayjs.utc().subtract(1, 'day').format('YYYY-MM-DD')+ const earliestDate = dayjs.utc().startOf('day').subtract(366, 'day').format('YYYY-MM-DD')+ expect(logic.values.dateFrom).toBe(earliestDate)+ expect(logic.values.dateTo).toBe(yesterday)+ expect(usageRequests.at(-1)?.get('date_from')).toBe(earliestDate)+ expect(usageRequests.at(-1)?.get('date_to')).toBe(yesterday)+ })++ it('filters materialization jobs by status', async () => {+ await expectLogic(logic).toDispatchActions(['loadUsageSuccess', 'loadMaterializationJobsSuccess'])+ const usageRequestCount = usageRequests.length++ await expectLogic(logic, () => logic.actions.setSelectedJobStatus('Failed')).toDispatchActions([+ 'loadMaterializationJobsSuccess',+ ])++ expect(jobRequests.at(-1)?.get('status')).toBe('Failed')+ expect(usageRequests).toHaveLength(usageRequestCount)+ expect(logic.values.jobsPage).toBe(1)+ })+it('keeps the jobs query ClickHouse-only when managed warehouse is disabled', async () => {await expectLogic(logic).toDispatchActions(['loadUsageSuccess', 'loadMaterializationJobsSuccess'])@@ -94,7 +164,9 @@ describe('modelsUsageLogic', () => {await expectLogic(logic, () => logic.actions.refresh()).toDispatchActions(['loadMaterializationJobsSuccess'])expect(logic.values.managedWarehouseEnabled).toBe(false)+ expect(usageRequests.at(-1)?.get('engine')).toBe('clickhouse')expect(jobRequests.at(-1)?.has('all_engines')).toBe(false)+ expect(jobRequests.at(-1)?.has('engine')).toBe(false)})it('surfaces model-name lookup failures and retries both job dependencies', async () => {diff --git a/products/data_modeling/frontend/modelsUsageLogic.ts b/products/data_modeling/frontend/modelsUsageLogic.tsindex c0a596477418..1436ff169206 100644--- a/products/data_modeling/frontend/modelsUsageLogic.ts+++ b/products/data_modeling/frontend/modelsUsageLogic.ts@@ -5,6 +5,7 @@ import posthog from 'posthog-js'import type { Series } from '@posthog/quill-charts'import { FEATURE_FLAGS } from 'lib/constants'+import { dayjs } from 'lib/dayjs'import { featureFlagLogic, FeatureFlagsSet } from 'lib/logic/featureFlagLogic'import { getCurrentTeamId } from 'lib/utils/getAppContext'import {@@ -13,12 +14,90 @@ import {} from 'scenes/data-warehouse/saved_queries/dataWarehouseViewsLogic'import { dataModelingJobsList } from 'products/data_warehouse/frontend/generated/api'-import type { PaginatedDataModelingJobListApi } from 'products/data_warehouse/frontend/generated/api.schemas'+import {+ DataModelingJobsListEngine,+ DataModelingJobsListStatus,+ type PaginatedDataModelingJobListApi,+} from 'products/data_warehouse/frontend/generated/api.schemas'import * as api from './generated/api'-import type { ModelUsageApi } from './generated/api.schemas'+import { DataModelingUsageListEngine, type ModelUsageApi } from './generated/api.schemas'export const MATERIALIZATION_JOBS_PAGE_SIZE = 20+export const DEFAULT_MODELS_USAGE_DATE_FROM = '-30d'+export const MATERIALIZATION_ENGINE_LABELS: Record<string, string> = {+ all: 'All engines',+ clickhouse: 'ClickHouse',+ duckgres: 'Duckgres',+ managed_warehouse: 'Managed warehouse',+}++const JOB_ENGINE_BY_USAGE_ENGINE: Partial<Record<DataModelingUsageListEngine, DataModelingJobsListEngine>> = {+ [DataModelingUsageListEngine.Clickhouse]: DataModelingJobsListEngine.Clickhouse,+ [DataModelingUsageListEngine.ManagedWarehouse]: DataModelingJobsListEngine.ManagedWarehouse,+}++function normalizeModelsUsageDateRange(+ dateFrom: string | null,+ dateTo: string | null+): { dateFrom: string; dateTo: string | null } {+ const normalizedDateFrom = dateFrom ?? DEFAULT_MODELS_USAGE_DATE_FROM+ if (dateTo === null && /^-(\d+)d$/.test(normalizedDateFrom)) {+ return { dateFrom: normalizedDateFrom, dateTo }+ }++ const today = dayjs.utc().startOf('day')+ const lastCompleteDay = today.subtract(1, 'day')+ const parsedDateFrom = dayjs.utc(normalizedDateFrom).startOf('day')+ const parsedDateTo = dateTo ? dayjs.utc(dateTo).startOf('day') : null+ const effectiveDateTo = parsedDateTo?.isBefore(today) ? parsedDateTo : lastCompleteDay+ const exclusiveDateTo = effectiveDateTo.add(1, 'day')+ const earliestDateFrom = exclusiveDateTo.subtract(366, 'day')+ const latestDateFrom = exclusiveDateTo.subtract(1, 'day')+ const effectiveDateFrom = parsedDateFrom.isBefore(earliestDateFrom)+ ? earliestDateFrom+ : parsedDateFrom.isAfter(latestDateFrom)+ ? latestDateFrom+ : parsedDateFrom++ return {+ dateFrom: effectiveDateFrom.format('YYYY-MM-DD'),+ dateTo: dateTo === null ? null : effectiveDateTo.format('YYYY-MM-DD'),+ }+}++function completeUtcUsageDateRange(+ dateFrom: string,+ dateTo: string | null+): {+ days?: number+ date_from?: string+ date_to?: string+} {+ const normalizedRange = normalizeModelsUsageDateRange(dateFrom, dateTo)+ dateFrom = normalizedRange.dateFrom+ dateTo = normalizedRange.dateTo+ const relativeDays = dateTo === null ? /^-(\d+)d$/.exec(dateFrom) : null+ if (relativeDays) {+ return { days: Number(relativeDays[1]) }+ }++ const start = dayjs.utc(dateFrom).startOf('day')+ return {+ date_from: start.format('YYYY-MM-DD'),+ date_to: dateTo ? dayjs.utc(dateTo).format('YYYY-MM-DD') : undefined,+ }+}++function materializationJobsDateFrom(dateFrom: string, dateTo: string | null): string {+ const normalizedRange = normalizeModelsUsageDateRange(dateFrom, dateTo)+ const relativeDays = normalizedRange.dateTo === null ? /^-(\d+)d$/.exec(normalizedRange.dateFrom) : null+ const start = relativeDays+ ? dayjs.utc().startOf('day').subtract(Number(relativeDays[1]), 'day')+ : dayjs.utc(normalizedRange.dateFrom).startOf('day')++ return start.toISOString()+}export interface MaterializedViewOption {id: string@@ -33,7 +112,8 @@ export interface modelsUsageLogicValues {featureFlags: FeatureFlagsSet // featureFlagLogicchartLabels: string[]chartSeries: Series[]- days: number+ dateFrom: string+ dateTo: string | nullfailed: booleanjobsFailed: booleanjobsLoading: boolean@@ -44,7 +124,8 @@ export interface modelsUsageLogicValues {materializationJobsLoading: booleanmaterializedViewOptions: MaterializedViewOption[]page: number- savedQueryNameById: Record<string, string>+ selectedEngine: DataModelingUsageListEngine+ selectedJobStatus: DataModelingJobsListStatus | nullselectedSavedQueryId: string | nullusage: ModelUsageApi | nullusageLoading: boolean@@ -92,8 +173,12 @@ export interface modelsUsageLogicActions {refreshJobs: () => {value: true}- setDays: (days: number) => {- days: number+ setDateRange: (+ dateFrom: string | null,+ dateTo: string | null+ ) => {+ dateFrom: string+ dateTo: string | null}setJobsPage: (page: number) => {page: number@@ -101,6 +186,12 @@ export interface modelsUsageLogicActions {setPage: (page: number) => {page: number}+ setSelectedEngine: (engine: DataModelingUsageListEngine) => {+ engine: DataModelingUsageListEngine+ }+ setSelectedJobStatus: (status: DataModelingJobsListStatus | null) => {+ status: DataModelingJobsListStatus | null+ }setSelectedSavedQueryId: (savedQueryId: string | null) => {savedQueryId: string | null}@@ -113,7 +204,6 @@ export interface modelsUsageLogicMeta {materializedViewOptions: (dataWarehouseSavedQueries: DataWarehouseSavedQuerySummary[]) => MaterializedViewOption[]- savedQueryNameById: (dataWarehouseSavedQueries: DataWarehouseSavedQuerySummary[]) => Record<string, string>jobsFailed: (materializationJobsFailed: boolean, dataWarehouseSavedQueriesFailed: boolean) => booleanjobsLoading: (materializationJobsLoading: boolean, dataWarehouseSavedQueriesLoading: boolean) => booleanchartLabels: (usage: ModelUsageApi | null) => string[]@@ -140,24 +230,50 @@ export const modelsUsageLogic = kea<modelsUsageLogicType>([actions: [dataWarehouseViewsLogic, ['loadDataWarehouseSavedQueries']],})),actions({- setDays: (days: number) => ({ days }),+ setDateRange: (dateFrom: string | null, dateTo: string | null) =>+ normalizeModelsUsageDateRange(dateFrom, dateTo),setPage: (page: number) => ({ page }),setJobsPage: (page: number) => ({ page }),+ setSelectedEngine: (engine: DataModelingUsageListEngine) => ({ engine }),+ setSelectedJobStatus: (status: DataModelingJobsListStatus | null) => ({ status }),setSelectedSavedQueryId: (savedQueryId: string | null) => ({ savedQueryId }),openModel: true,refresh: true,refreshJobs: true,}),reducers({- days: [30, { setDays: (_, { days }) => days }],- page: [1, { setPage: (_, { page }) => page, setDays: () => 1 }],+ dateFrom: [+ DEFAULT_MODELS_USAGE_DATE_FROM as string,+ { setDateRange: (_, { dateFrom }) => dateFrom ?? DEFAULT_MODELS_USAGE_DATE_FROM },+ ],+ dateTo: [null as string | null, { setDateRange: (_, { dateTo }) => dateTo }],+ page: [+ 1,+ {+ setPage: (_, { page }) => page,+ setDateRange: () => 1,+ setSelectedEngine: () => 1,+ setSelectedSavedQueryId: () => 1,+ },+ ],jobsPage: [1,{setJobsPage: (_, { page }) => page,+ setDateRange: () => 1,+ setSelectedEngine: () => 1,+ setSelectedJobStatus: () => 1,setSelectedSavedQueryId: () => 1,},],+ selectedEngine: [+ DataModelingUsageListEngine.All as DataModelingUsageListEngine,+ { setSelectedEngine: (_, { engine }) => engine },+ ],+ selectedJobStatus: [+ null as DataModelingJobsListStatus | null,+ { setSelectedJobStatus: (_, { status }) => status },+ ],selectedSavedQueryId: [null as string | null,{ setSelectedSavedQueryId: (_, { savedQueryId }) => savedQueryId },@@ -173,10 +289,17 @@ export const modelsUsageLogic = kea<modelsUsageLogicType>([null as ModelUsageApi | null,{loadUsage: async (_: void, breakpoint): Promise<ModelUsageApi> => {+ const dateRange = completeUtcUsageDateRange(values.dateFrom, values.dateTo)const result = await api.dataModelingUsageList(String(getCurrentTeamId()), {- days: values.days,+ days: dateRange.days,+ date_from: dateRange.date_from,+ date_to: dateRange.date_to,+ engine: values.managedWarehouseEnabled+ ? values.selectedEngine+ : DataModelingUsageListEngine.Clickhouse,page: values.page,page_size: 20,+ saved_query_id: values.selectedSavedQueryId ?? undefined,})breakpoint()return result@@ -188,10 +311,16 @@ export const modelsUsageLogic = kea<modelsUsageLogicType>([{loadMaterializationJobs: async (_: void, breakpoint): Promise<PaginatedDataModelingJobListApi> => {const result = await dataModelingJobsList(String(getCurrentTeamId()), {+ after: materializationJobsDateFrom(values.dateFrom, values.dateTo),limit: MATERIALIZATION_JOBS_PAGE_SIZE,offset: (values.jobsPage - 1) * MATERIALIZATION_JOBS_PAGE_SIZE,saved_query_id: values.selectedSavedQueryId ?? undefined,all_engines: values.managedWarehouseEnabled || undefined,+ engine:+ values.managedWarehouseEnabled && values.selectedEngine !== DataModelingUsageListEngine.All+ ? JOB_ENGINE_BY_USAGE_ENGINE[values.selectedEngine]+ : undefined,+ status: values.selectedJobStatus ?? undefined,})breakpoint()return result@@ -202,8 +331,7 @@ export const modelsUsageLogic = kea<modelsUsageLogicType>([selectors({managedWarehouseEnabled: [(s) => [s.featureFlags],- (featureFlags: FeatureFlagsSet): boolean =>- !!featureFlags[FEATURE_FLAGS.DATA_WAREHOUSE_SCENE] || !!featureFlags[FEATURE_FLAGS.MANAGED_TRINO_QUERY],+ (featureFlags: FeatureFlagsSet): boolean => !!featureFlags[FEATURE_FLAGS.MANAGED_TRINO_QUERY],],materializedViewOptions: [(s) => [s.dataWarehouseSavedQueries],@@ -213,11 +341,6 @@ export const modelsUsageLogic = kea<modelsUsageLogicType>([.map((view) => ({ id: view.id, name: view.name ?? 'Untitled view' })).sort((a, b) => a.name.localeCompare(b.name)),],- savedQueryNameById: [- (s) => [s.dataWarehouseSavedQueries],- (savedQueries: DataWarehouseSavedQuerySummary[]): Record<string, string> =>- Object.fromEntries(savedQueries.map((view) => [view.id, view.name ?? 'Untitled view'])),- ],jobsFailed: [(s) => [s.materializationJobsFailed, s.dataWarehouseSavedQueriesFailed],(materializationJobsFailed: boolean, dataWarehouseSavedQueriesFailed: boolean): boolean =>@@ -234,19 +357,37 @@ export const modelsUsageLogic = kea<modelsUsageLogicType>([],chartSeries: [(s) => [s.usage],- (usage: ModelUsageApi | null): Series[] => [- { key: 'bytes', label: 'Bytes read', data: usage?.trends.map((day) => day.read_bytes ?? NaN) ?? [] },- ],+ (usage: ModelUsageApi | null): Series[] =>+ (usage?.trends[0]?.engines ?? []).map(({ engine }) => ({+ key: engine,+ label: MATERIALIZATION_ENGINE_LABELS[engine] ?? engine,+ data:+ usage?.trends.map(+ (day) => day.engines.find((item) => item.engine === engine)?.read_bytes ?? NaN+ ) ?? [],+ })),],}),listeners(({ actions, values }) => ({- setDays: () => {- posthog.capture('models usage date range changed', { days: values.days })+ setDateRange: () => {+ posthog.capture('models usage date range changed', {+ date_from: values.dateFrom,+ date_to: values.dateTo,+ })actions.loadUsage()+ actions.loadMaterializationJobs()},setPage: () => actions.loadUsage(),setJobsPage: () => actions.loadMaterializationJobs(),- setSelectedSavedQueryId: () => actions.loadMaterializationJobs(),+ setSelectedEngine: () => {+ actions.loadUsage()+ actions.loadMaterializationJobs()+ },+ setSelectedJobStatus: () => actions.loadMaterializationJobs(),+ setSelectedSavedQueryId: () => {+ actions.loadUsage()+ actions.loadMaterializationJobs()+ },refreshJobs: () => {actions.loadDataWarehouseSavedQueries()actions.loadMaterializationJobs()@@ -257,7 +398,8 @@ export const modelsUsageLogic = kea<modelsUsageLogicType>([},loadUsageSuccess: ({ usage }) => {posthog.capture('models usage loaded', {- days: values.days,+ date_from: values.dateFrom,+ date_to: values.dateTo,has_runs: usage.overview.runs > 0,has_measurements: usage.overview.measured_runs > 0,})diff --git a/products/data_warehouse/backend/presentation/views/data_modeling_job.py b/products/data_warehouse/backend/presentation/views/data_modeling_job.pyindex 1876a68fee80..8bdf37edb9ed 100644--- a/products/data_warehouse/backend/presentation/views/data_modeling_job.py+++ b/products/data_warehouse/backend/presentation/views/data_modeling_job.py@@ -1,5 +1,6 @@from django.db.models import Q, QuerySet+import django_filtersfrom django_filters.rest_framework import DjangoFilterBackendfrom drf_spectacular.types import OpenApiTypesfrom drf_spectacular.utils import OpenApiParameter, extend_schema, extend_schema_view@@ -14,7 +15,24 @@ from products.data_modeling.backend.facade.models import DataModelingJob, DataMoclass DataModelingJobSerializer(serializers.ModelSerializer):+ saved_query_name = serializers.CharField(+ source="saved_query.name",+ read_only=True,+ allow_null=True,+ help_text="Model name, or null if permanently deleted.",+ )+ saved_query_deleted = serializers.BooleanField(+ source="saved_query.deleted",+ read_only=True,+ allow_null=True,+ help_text="Whether the model is soft-deleted, or null if permanently deleted.",+ )engine = serializers.CharField(read_only=True, help_text="The engine that ran this materialization job.")+ read_bytes = serializers.IntegerField(+ read_only=True,+ allow_null=True,+ help_text="Bytes read by this run, or null if it has not been measured yet.",+ )updated_at = serializers.DateTimeField(read_only=True,help_text="When the job row last changed. For finished jobs this is when the run reached its terminal status.",@@ -40,6 +58,8 @@ class DataModelingJobSerializer(serializers.ModelSerializer):fields = ["id","saved_query_id",+ "saved_query_name",+ "saved_query_deleted","status","engine","run_mode",@@ -66,6 +86,42 @@ class DataModelingJobPagination(pagination.LimitOffsetPagination):max_limit = 100+DATA_MODELING_JOB_LIST_FIELDS = (+ "id",+ "saved_query_id",+ "status",+ "engine",+ "run_mode",+ "full_refresh_reason",+ "rows_materialized",+ "error",+ "created_at",+ "last_run_at",+ "updated_at",+ "workflow_id",+ "workflow_run_id",+ "rows_expected",+ "read_bytes",+ "read_rows",+ "cpu_microseconds",+ "query_count",+ "usage_updated_at",+)+++class DataModelingJobFilterSet(django_filters.FilterSet):+ after = django_filters.IsoDateTimeFilter(+ field_name="last_run_at", lookup_expr="gte", help_text="Include jobs finished at or after this time."+ )+ before = django_filters.IsoDateTimeFilter(+ field_name="last_run_at", lookup_expr="lt", help_text="Include jobs finished before this time."+ )++ class Meta:+ model = DataModelingJob+ fields = ["saved_query_id", "status", "engine"]++@extend_schema_view(list=extend_schema(parameters=[@@ -89,7 +145,7 @@ class DataModelingJobViewSet(TeamAndOrgViewSetMixin, viewsets.ReadOnlyModelViewSpagination_class = DataModelingJobPaginationqueryset = DataModelingJob.objects.all()filter_backends = [DjangoFilterBackend]- filterset_fields = ["saved_query_id", "status"]+ filterset_class = DataModelingJobFilterSetsearch_fields = ["saved_query_id"]ordering_fields = ["created_at"]ordering = "-created_at"@@ -108,7 +164,15 @@ class DataModelingJobViewSet(TeamAndOrgViewSetMixin, viewsets.ReadOnlyModelViewS)queryset = queryset.filter(saved_query_id__in=accessible_saved_queries.values("id"))- return queryset.order_by("-created_at")+ return (+ queryset.select_related("saved_query")+ .only(+ *DATA_MODELING_JOB_LIST_FIELDS,+ "saved_query__name",+ "saved_query__deleted",+ )+ .order_by("-created_at")+ )@action(methods=["GET"], detail=False)def running(self, request, *args, **kwargs):diff --git a/products/data_warehouse/backend/tests/api/test_datamodelingjob.py b/products/data_warehouse/backend/tests/api/test_datamodelingjob.pyindex 5943e652a410..ba20aeabb1e0 100644--- a/products/data_warehouse/backend/tests/api/test_datamodelingjob.py+++ b/products/data_warehouse/backend/tests/api/test_datamodelingjob.py@@ -1,3 +1,5 @@+from datetime import UTC, datetime+from posthog.test.base import APIBaseTestfrom django.utils import timezone@@ -67,6 +69,8 @@ class TestDataModelingJob(APIBaseTest):first_job = results[0]self.assertEqual(first_job["status"], DataModelingJob.Status.FAILED)self.assertEqual(first_job["saved_query_id"], str(self.saved_query.id))+ self.assertEqual(first_job["saved_query_name"], "Test saved query")+ self.assertFalse(first_job["saved_query_deleted"])self.assertEqual(first_job["rows_materialized"], 0)self.assertEqual(first_job["error"], "Something went wrong")# The frontend derives failed-run duration and the log-search window from updated_at.@@ -118,6 +122,19 @@ class TestDataModelingJob(APIBaseTest):self.assertEqual(response.status_code, 200)self.assertEqual([job["id"] for job in response.json()["results"]], [str(self.job3.id)])+ def test_filters_by_finished_date_range(self):+ DataModelingJob.objects.filter(id=self.job1.id).update(last_run_at=datetime(2026, 10, 10, tzinfo=UTC))+ DataModelingJob.objects.filter(id=self.job2.id).update(last_run_at=datetime(2026, 10, 11, tzinfo=UTC))+ DataModelingJob.objects.filter(id=self.job3.id).update(last_run_at=datetime(2026, 10, 12, tzinfo=UTC))++ response = self.client.get(+ f"/api/environments/{self.team.pk}/data_modeling_jobs/",+ {"after": "2026-10-11T00:00:00Z", "before": "2026-10-12T00:00:00Z"},+ )++ self.assertEqual(response.status_code, 200)+ self.assertEqual([job["id"] for job in response.json()["results"]], [str(self.job2.id)])+def test_excludes_managed_warehouse_jobs(self):warehouse_job = DataModelingJob.objects.create(team=self.team,@@ -159,6 +176,22 @@ class TestDataModelingJob(APIBaseTest):self.assertEqual(jobs_by_id[str(warehouse_job.id)]["query_count"], 4)self.assertIsNotNone(jobs_by_id[str(warehouse_job.id)]["usage_updated_at"])+ managed_only = self.client.get(+ f"/api/environments/{self.team.pk}/data_modeling_jobs/?all_engines=true&engine=managed_warehouse"+ )+ self.assertEqual(managed_only.status_code, 200)+ self.assertEqual([job["id"] for job in managed_only.json()["results"]], [str(warehouse_job.id)])++ def test_job_keeps_name_for_an_endpoint_or_soft_deleted_model(self):+ self.saved_query.deleted = True+ self.saved_query.save(update_fields=["deleted"])++ response = self.client.get(f"/api/environments/{self.team.pk}/data_modeling_jobs/{self.job1.id}/")++ self.assertEqual(response.status_code, 200)+ self.assertEqual(response.json()["saved_query_name"], "Test saved query")+ self.assertTrue(response.json()["saved_query_deleted"])+def test_pagination_honors_offset_and_returns_count(self):paged_query = DataWarehouseSavedQuery.objects.create(team=self.team, name="Paged saved query")for _ in range(12):diff --git a/products/data_warehouse/frontend/generated/api.schemas.ts b/products/data_warehouse/frontend/generated/api.schemas.tsindex 8fa4e7b9b6e7..15168506f3cd 100644--- a/products/data_warehouse/frontend/generated/api.schemas.ts+++ b/products/data_warehouse/frontend/generated/api.schemas.ts@@ -41,6 +41,16 @@ export interface DataModelingJobApi {readonly id: string/** @nullable */readonly saved_query_id: string | null+ /**+ * Model name, or null if permanently deleted.+ * @nullable+ */+ readonly saved_query_name: string | null+ /**+ * Whether the model is soft-deleted, or null if permanently deleted.+ * @nullable+ */+ readonly saved_query_deleted: boolean | nullreadonly status: DataModelingJobStatusEnumApi/** The engine that ran this materialization job. */readonly engine: string@@ -71,7 +81,7 @@ export interface DataModelingJobApi {*/readonly rows_expected: number | null/**- * Bytes read by this run, including probes and retries.+ * Bytes read by this run, or null if it has not been measured yet.* @nullable*/readonly read_bytes: number | null@@ -5599,10 +5609,24 @@ export interface ViewLinkValidationErrorApi {}export type DataModelingJobsListParams = {+ /**+ * Include jobs finished at or after this time.+ */+ after?: string/*** Include jobs from every materialization engine. Defaults to ClickHouse jobs only.*/all_engines?: boolean+ /**+ * Include jobs finished before this time.+ */+ before?: string+ /**+ * * `clickhouse` - ClickHouse+ * * `duckgres` - Duckgres+ * * `managed_warehouse` - Managed warehouse+ */+ engine?: DataModelingJobsListEngine/*** Number of results to return per page.*/@@ -5622,6 +5646,14 @@ export type DataModelingJobsListParams = {status?: DataModelingJobsListStatus}+export type DataModelingJobsListEngine = (typeof DataModelingJobsListEngine)[keyof typeof DataModelingJobsListEngine]++export const DataModelingJobsListEngine = {+ Clickhouse: 'clickhouse',+ Duckgres: 'duckgres',+ ManagedWarehouse: 'managed_warehouse',+} as const+export type DataModelingJobsListStatus = (typeof DataModelingJobsListStatus)[keyof typeof DataModelingJobsListStatus]export const DataModelingJobsListStatus = {diff --git a/services/mcp/src/api/generated.ts b/services/mcp/src/api/generated.tsindex 248b4baea30f..69d7fa6e26c6 100644--- a/services/mcp/src/api/generated.ts+++ b/services/mcp/src/api/generated.ts@@ -29810,6 +29810,16 @@ export namespace Schemas {readonly id: string;/** @nullable */readonly saved_query_id: string | null;+ /**+ * Model name, or null if permanently deleted.+ * @nullable+ */+ readonly saved_query_name: string | null;+ /**+ * Whether the model is soft-deleted, or null if permanently deleted.+ * @nullable+ */+ readonly saved_query_deleted: boolean | null;readonly status: DataModelingJobStatusEnum;/** The engine that ran this materialization job. */readonly engine: string;@@ -29840,7 +29850,7 @@ export namespace Schemas {*/readonly rows_expected: number | null;/**- * Bytes read by this run, including probes and retries.+ * Bytes read by this run, or null if it has not been measured yet.* @nullable*/readonly read_bytes: number | null;@@ -70923,7 +70933,7 @@ export namespace Schemas {}export interface ModelUsageTotals {- /** Finished ClickHouse materialization runs, including failures and cancellations. */+ /** Finished materialization runs, including failures and cancellations. */runs: number;/** Runs with recorded usage. Other runs have unknown cost. */measured_runs: number;@@ -70949,8 +70959,37 @@ export namespace Schemas {query_count: number | null;}+ export interface ModelUsageEngine {+ /** Finished materialization runs, including failures and cancellations. */+ runs: number;+ /** Runs with recorded usage. Other runs have unknown cost. */+ measured_runs: number;+ /**+ * Measured bytes read, including probes and retries. Null if unmeasured.+ * @nullable+ */+ read_bytes: number | null;+ /**+ * Measured rows read. Null if unmeasured.+ * @nullable+ */+ read_rows: number | null;+ /**+ * Measured CPU microseconds. Null if unmeasured.+ * @nullable+ */+ cpu_microseconds: number | null;+ /**+ * Measured ClickHouse query count. Null if unmeasured.+ * @nullable+ */+ query_count: number | null;+ /** Materialization engine: clickhouse or managed_warehouse. */+ engine: string;+ }+export interface ModelUsageDay {- /** Finished ClickHouse materialization runs, including failures and cancellations. */+ /** Finished materialization runs, including failures and cancellations. */runs: number;/** Runs with recorded usage. Other runs have unknown cost. */measured_runs: number;@@ -70976,10 +71015,12 @@ export namespace Schemas {query_count: number | null;/** UTC day on which the runs finished. */day: string;+ /** Usage totals for each selected engine. */+ engines: ModelUsageEngine[];}export interface ModelUsageRow {- /** Finished ClickHouse materialization runs, including failures and cancellations. */+ /** Finished materialization runs, including failures and cancellations. */runs: number;/** Runs with recorded usage. Other runs have unknown cost. */measured_runs: number;@@ -71012,6 +71053,8 @@ export namespace Schemas {name: string;/** Whether this model has been deleted. */deleted: boolean;+ /** Usage totals for each selected engine. */+ engines: ModelUsageEngine[];}export interface ModelUsage {@@ -128485,10 +128528,24 @@ export namespace Schemas {};export type DataModelingJobsListParams = {+ /**+ * Include jobs finished at or after this time.+ */+ after?: string;/*** Include jobs from every materialization engine. Defaults to ClickHouse jobs only.*/all_engines?: boolean;+ /**+ * Include jobs finished before this time.+ */+ before?: string;+ /**+ * * `clickhouse` - ClickHouse+ * * `duckgres` - Duckgres+ * * `managed_warehouse` - Managed warehouse+ */+ engine?: DataModelingJobsListEngine;/*** Number of results to return per page.*/@@ -128508,6 +128565,15 @@ export namespace Schemas {status?: DataModelingJobsListStatus;};+ export type DataModelingJobsListEngine = typeof DataModelingJobsListEngine[keyof typeof DataModelingJobsListEngine];+++ export const DataModelingJobsListEngine = {+ Clickhouse: 'clickhouse',+ Duckgres: 'duckgres',+ ManagedWarehouse: 'managed_warehouse',+ } as const;+export type DataModelingJobsListStatus = typeof DataModelingJobsListStatus[keyof typeof DataModelingJobsListStatus];@@ -128563,12 +128629,29 @@ export namespace Schemas {};export type DataModelingUsageListParams = {+ /**+ * Inclusive UTC start date for a custom range, up to 366 complete days.+ */+ date_from?: string;+ /**+ * Inclusive UTC end date for a custom range. Today is always excluded.+ */+ date_to?: string;/*** Number of complete UTC days, up to 90.* @minimum 1* @maximum 90*/days?: number;+ /**+ * Materialization engine. All combines ClickHouse and managed warehouse usage.+ *+ * * `all` - All+ * * `clickhouse` - ClickHouse+ * * `managed_warehouse` - Managed warehouse+ * @minLength 1+ */+ engine?: DataModelingUsageListEngine;/*** Model breakdown page, starting at 1.* @minimum 1@@ -128580,8 +128663,22 @@ export namespace Schemas {* @maximum 100*/page_size?: number;+ /**+ * Limit usage to one accessible materialized view.+ * @nullable+ */+ saved_query_id?: string | null;};+ export type DataModelingUsageListEngine = typeof DataModelingUsageListEngine[keyof typeof DataModelingUsageListEngine];+++ export const DataModelingUsageListEngine = {+ All: 'all',+ Clickhouse: 'clickhouse',+ ManagedWarehouse: 'managed_warehouse',+ } as const;+export type DataQualityChecksListParams = {/*** Only the checks that make this assertion. See /check_types/.