Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

feat(api): add strict_fetch to catch lazy loads in list endpoints (#126317)

Paul D'Ambra0b4e4deb79a6
diff --git a/.agents/skills/improving-drf-endpoints/SKILL.md b/.agents/skills/improving-drf-endpoints/SKILL.md
index 8f680ac5d5e2..58c3cf8089cb 100644
--- a/.agents/skills/improving-drf-endpoints/SKILL.md
+++ b/.agents/skills/improving-drf-endpoints/SKILL.md
@@ -69,6 +69,7 @@ See [serializer-fields.md](references/serializer-fields.md) for patterns and exa
14. **Prefer `@validated_request`** over manual `serializer.is_valid()` + `@extend_schema` — it handles both in one decorator
15. **ViewSets outside `products/` need `@extend_schema(extensions={"x-product": "<product>"})`** — ViewSets in `products/<name>/backend/` are auto-attributed via module path; ViewSets in `posthog/api/` or `ee/` aren't and must declare attribution explicitly via the `x-product` extension. Accepts a plain string (`"product_analytics"`) or `ProductKey.X` enum (kebab values are normalized). Don't use `tags=["<product>"]` to influence codegen routing — `tags` is for Swagger UI display only. Without `x-product`, the MCP scaffold and frontend type generator can't route the endpoint to the right product
16. **`partial_update` `request=` override must be a superset of runtime write fields** — `extend_schema(request=CustomSerializer)` replaces drf-spectacular's inference from `serializer_class`; omitted fields disappear from OpenAPI, frontend types, and MCP tool schemas even when the runtime serializer still accepts them. After changing the override, run `hogli build:openapi` and verify generated MCP tool schemas still expose every OpenAPI body field
+17. **Wrap the list queryset with `strict_fetch()`** from `posthog.models.utils`, in `safely_get_queryset` when `self.action == "list"`. In tests and local dev, a lazy load of a forward relation or deferred field then raises `FieldFetchBlocked`, so a missing `select_related` fails the list test. In production, the missing field costs one extra query for the page, not one per row. Fix a failure with the `select_related` the error names, or stop deferring that field. The guard does not cover reverse FK or M2M managers, so keep `prefetch_related` and an `assertNumQueries` test for those. Reference: `AnnotationsViewSet` in `products/annotations/backend/api/annotation.py`
**Streaming endpoints:** For SSE or streaming responses, use `@extend_schema(request=InputSerializer, responses={(200, "text/event-stream"): OpenApiTypes.STR})` to document the request schema even though the response can't be fully typed.
diff --git a/posthog/models/utils.py b/posthog/models/utils.py
index d2f61f0d919a..56a0b4812c18 100644
--- a/posthog/models/utils.py
+++ b/posthog/models/utils.py
@@ -11,12 +11,13 @@ from decimal import Decimal
from typing import TYPE_CHECKING, Any, Literal, Optional, TypeVar, Union
from uuid import UUID
+from django.conf import settings
from django.contrib.auth.hashers import PBKDF2PasswordHasher
from django.core.exceptions import ValidationError
from django.db import IntegrityError, connections, models, transaction
from django.db.backends.ddl_references import Statement
from django.db.backends.utils import CursorWrapper
-from django.db.models import Q, Subquery, UniqueConstraint
+from django.db.models import Q, QuerySet, Subquery, UniqueConstraint
from django.db.models.constraints import BaseConstraint
from django.utils.text import slugify
@@ -27,6 +28,7 @@ if TYPE_CHECKING:
from posthog.hogql import ast
T = TypeVar("T")
+ModelT = TypeVar("ModelT", bound=models.Model)
BASE62 = string.digits + string.ascii_letters # All lowercase ASCII letters + all uppercase ASCII letters + digits
AMBIGUOUS_CHARS = frozenset("01OIl")
@@ -338,6 +340,13 @@ def get_deferred_field_set_for_model(
return {f"{field_prefix}{x.name}" for x in model._meta.fields if x.name not in fields_not_deferred}
+def strict_fetch(queryset: QuerySet[ModelT]) -> QuerySet[ModelT]:
+ # In tests and local dev, a lazy load of a related object or deferred field raises FieldFetchBlocked,
+ # so a missing select_related fails loudly. In production, Django loads that field for all rows of the
+ # queryset in one query, so a missing select_related costs one extra query and not one per row.
+ return queryset.fetch_mode(models.FETCH_RAISE if settings.DEBUG or settings.TEST else models.FETCH_PEERS)
+
+
class UniqueConstraintByExpression(BaseConstraint):
def __init__(self, *, name: str, expression: str, concurrently=True):
self.name = name
diff --git a/products/annotations/backend/api/annotation.py b/products/annotations/backend/api/annotation.py
index 1bee20c539bf..26dae0b58731 100644
--- a/products/annotations/backend/api/annotation.py
+++ b/products/annotations/backend/api/annotation.py
@@ -10,6 +10,7 @@ from posthog.api.routing import TeamAndOrgViewSetMixin
from posthog.api.scoped_related_fields import TeamScopedPrimaryKeyRelatedField
from posthog.api.shared import UserBasicSerializer
from posthog.models.user import User
+from posthog.models.utils import strict_fetch
from products.annotations.backend.models.annotation import Annotation
from products.dashboards.backend.models.dashboard import Dashboard
@@ -233,7 +234,7 @@ class AnnotationsViewSet(TeamAndOrgViewSetMixin, ForbidDestroyModel, viewsets.Mo
def safely_get_queryset(self, queryset) -> QuerySet:
if self.action == "list":
# `date_marker` alone is not a total order, so limit-offset pages can repeat or drop rows.
- queryset = queryset.order_by("-date_marker", "-id")
+ queryset = strict_fetch(queryset.order_by("-date_marker", "-id"))
if self.action != "partial_update":
# We never want deleted items to be included in the queryset… except when we want to restore an annotation
# That's because annotations are restored with a PATCH request setting `deleted` to `False`