:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
fix(flags): bill local evaluation 304 responses at the /flags rate (#128786)
Gustavo H. Strassburger5e1bf0de3d1c
diff --git a/docs/internal/feature-flags/billing.md b/docs/internal/feature-flags/billing.mdindex aca5749d05ad..6365e9b12373 100644--- a/docs/internal/feature-flags/billing.md+++ b/docs/internal/feature-flags/billing.md@@ -47,10 +47,11 @@ When a feature flag request is processed, the Rust service increments Redis counThe service stores counts in Redis hashes using time-bucketed fields:-| Request Type | Team Key | SDK Key |-| ---------------- | --------------------------------------------- | ------------------------------------------------------------ |-| `/decide` | `posthog:decide_requests:{team_id}` | `posthog:decide_requests:sdk:{team_id}:{sdk_name}` |-| Local evaluation | `posthog:local_evaluation_requests:{team_id}` | `posthog:local_evaluation_requests:sdk:{team_id}:{sdk_name}` |+| Request Type | Team Key | SDK Key |+| ---------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------- |+| `/decide` | `posthog:decide_requests:{team_id}` | `posthog:decide_requests:sdk:{team_id}:{sdk_name}` |+| Local evaluation (200) | `posthog:local_evaluation_requests:{team_id}` | `posthog:local_evaluation_requests:sdk:{team_id}:{sdk_name}` |+| Local evaluation (304) | `posthog:local_evaluation_not_modified_requests:{team_id}` | `posthog:local_evaluation_not_modified_requests:sdk:{team_id}:{sdk_name}` |### Time bucketing@@ -93,7 +94,13 @@ Rust's `/flags/definitions` endpoint enforces billing quotas. When a team exceed}```-The Rust endpoint checks `FeatureFlagsLimiter.is_limited(token)` before the ETag comparison and cache fetch. If the quota is exceeded, all requests (including conditional ones) return HTTP 402 — no 304 is issued while a team is over quota. Requests that pass the quota check but result in a 304 (ETag match) are not counted toward billing usage, matching Django's behavior.+The Rust endpoint checks `FeatureFlagsLimiter.is_limited(token)` before the ETag comparison and cache fetch. If the quota is exceeded, all requests (including conditional ones) return HTTP 402 — no 304 is issued while a team is over quota.++### Conditional requests (ETag)++A request that passes the quota check and results in a 304 (ETag match) is counted on the separate `local_evaluation_not_modified_requests` counter. The usage report bills these at 1 unit each, the same as a `/decide` request, instead of the 10 units a full local evaluation response costs. The usage record key is `feature_flag_local_evaluation_not_modified_requests`, apart from `feature_flag_requests`, because each key has its own usage counter and a shared key would count a 304 twice.++A 304 for definitions that hold only survey or product tour flags is not billed, the same as a 200. The service memoizes per team and ETag whether the definitions hold a billable flag, so it reads the payload once per pod per ETag rather than on every poll. The memo is sized by `DEFINITIONS_BILLABLE_CACHE_CAPACITY` and an entry lives for `DEFINITIONS_BILLABLE_CACHE_TTL_SECONDS` (default one hour), which bounds how long a wrong answer can last. Only a payload served from Redis decides, because an S3 copy can lag the ETag by a version. A payload that came from S3, failed to read, or took longer than 500 ms leaves that poll unbilled, and the miss is remembered for 30 seconds so a degraded payload tier is not hammered by every poll. These unbilled polls show as `flags_flag_definitions_not_modified_billing_total{outcome="unknown"}`, which is why the 304 billing counter can run below the ETag hit counter.### SDK tracking@@ -216,10 +223,11 @@ The aggregation task emits events to PostHog's internal analytics instance.### Event types-| Event Name | Description |-| ------------------------ | ---------------------------------------------- |-| `decide usage` | Counts from `/decide` endpoint requests |-| `local evaluation usage` | Counts from local evaluation endpoint requests |+| Event Name | Description |+| ------------------------------------- | ------------------------------------------------------- |+| `decide usage` | Counts from `/decide` endpoint requests |+| `local evaluation usage` | Counts from local evaluation requests answered with 200 |+| `local evaluation not modified usage` | Counts from local evaluation requests answered with 304 |### Event properties@@ -286,6 +294,7 @@ The team can also agree to bill the usage record count with the gap.- Events, enhanced person events, and exceptions: usage records use capture time, and the legacy queries use the event timestamp. The switch must not count an event on both dates or on neither date.- Feature flag requests: usage records use the flush time of the Rust flags service. The legacy query uses the later Celery export time.+- Feature flag local evaluation 304 requests: the same clock difference as feature flag requests.- Feature flag local evaluation requests: usage records from before [#118814](https://github.com/PostHog/posthog/pull/118814) omit the requests that the Django feature flag list endpoint served. Compare both producers over a complete window before you switch this counter.- CDP invocations and workflow emails, push, and SMS: usage records and app metrics flush on separate clocks. Each source can put one dispatch on a different date.- Workflow invocations: the legacy query counts fetch and push dispatches. Before [#118809](https://github.com/PostHog/posthog/pull/118809), a push dispatch wrote only a `workflow_push_sent` record, so `workflow_billable_invocations` held only fetches. Now a push dispatch writes both records. The clock difference above also applies.@@ -295,16 +304,17 @@ The team can also agree to bill the usage record count with the gap.### Billable calculation-Local evaluation requests are weighted 10x compared to decide requests:+Local evaluation requests that return full flag definitions are weighted 10x compared to decide requests. Local evaluation requests answered with 304 are weighted like decide requests:```pythonbillable_feature_flag_requests_count_in_period = (decide_requests_count_in_period+ (local_evaluation_requests_count_in_period * 10)+ + local_evaluation_not_modified_requests_count_in_period)```-This reflects the higher resource cost of local evaluation requests, which return full flag definitions rather than just evaluation results.+This reflects the higher resource cost of full local evaluation responses, which return full flag definitions rather than just evaluation results.### Token validation@@ -323,6 +333,7 @@ AND has([%(validity_token)s], replaceRegexpAll(JSONExtractRaw(properties, 'token| Distributed lock | Prevent concurrent processing of the same team's data || Redis pipelining | Minimize network round-trips for better performance || 10x local evaluation weight | Local evaluation returns full flag definitions, requiring more server resources |+| 1x weight for 304 responses | A 304 serves no definitions payload, so it costs the same as a `/decide` request || Selective billing | Survey and product tour flags are internal features, not customer-billable || SDK breakdown for analytics only | Billing charges per request regardless of SDK; breakdown is for internal analytics |@@ -334,7 +345,7 @@ The feature flags **Usage** tab shows the existing billing events grouped by SDK- Remote evaluation combines `/flags` and legacy `/decide` traffic.- Local evaluation combines `/flags/definitions` and its legacy alias.- Counts include billable requests only. They are not raw HTTP traffic totals.-- Local evaluation requests contribute 10 billing units each.+- A full local evaluation response contributes 10 billing units. A 304 contributes 1.- The SDK breakdown comes from the request user-agent and unknown SDKs appear as `other`.- Aggregation means recent usage can take approximately 30 minutes to appear. Hourly buckets use aggregation event time, so they approximate rather than preserve exact request time.diff --git a/docs/internal/feature-flags/rust-service-overview.md b/docs/internal/feature-flags/rust-service-overview.mdindex b70951996548..0e0e235a446a 100644--- a/docs/internal/feature-flags/rust-service-overview.md+++ b/docs/internal/feature-flags/rust-service-overview.md@@ -148,7 +148,7 @@ Billing quota enforcement matches Django's `/api/feature_flag/local_evaluation`- **Quota check**: Uses `FeatureFlagsLimiter.is_limited(token)` to verify the team hasn't exceeded their feature flag request quota. Returns HTTP 402 with a JSON body (`{"type": "quota_limited", "code": "payment_required", ...}`) when the quota is exceeded.- **Non-billable flag filtering**: Usage tracking skips requests where the response contains only non-billable flags — i.e., flags with keys starting with `survey-targeting-` or `product-tour-targeting-`. The shared `is_billable_flag_key()` predicate (in `flag_analytics.rs`) is used by both this endpoint and the `/flags` billing handler.-- **304 responses skip billing**: Usage is recorded after the ETag/304 path, so conditional responses that return `304 Not Modified` are not counted toward billing. This matches Django's behavior.+- **304 responses bill 1 unit**: A conditional response that returns `304 Not Modified` is recorded on its own counter and billed at the `/flags` rate, a tenth of a full response. See [Conditional requests (ETag)](billing.md#conditional-requests-etag) for the billable-flag memo and what goes unbilled.## Request and response typesdiff --git a/ee/billing/quota_limiting.py b/ee/billing/quota_limiting.pyindex 85bc2fb4dd3c..2fbff3126764 100644--- a/ee/billing/quota_limiting.py+++ b/ee/billing/quota_limiting.py@@ -43,6 +43,7 @@ from posthog.usage_counters import UsageCounter, UsageCounterMode, UsageCounterPfrom posthog.utils import get_current_dayfrom products.ai_observability.backend.offline_evaluation_billing import get_offline_evaluation_usage+from products.feature_flags.backend.flag_analytics import billable_feature_flag_requestsfrom ee.billing.quota_usage import QuotaUsageBatch, retry_quota_database_operation@@ -1266,14 +1267,20 @@ def _aggregate_quota_usage(for team in teams:decide_requests = all_data["teams_with_decide_requests_count"].get(team.id, 0)local_evaluation_requests = all_data["teams_with_local_evaluation_requests_count"].get(team.id, 0)+ local_evaluation_not_modified_requests = all_data[+ "teams_with_local_evaluation_not_modified_requests_count"+ ].get(team.id, 0)team_report = UsageCounters(events=all_data["teams_with_event_count_in_period"].get(team.id, 0),exceptions=all_data["teams_with_exceptions_captured_in_period"].get(team.id, 0),recordings=all_data["teams_with_recording_count_in_period"].get(team.id, 0),rows_synced=all_data["teams_with_rows_synced_in_period"].get(team.id, 0),- feature_flag_requests=decide_requests- + (local_evaluation_requests * 10), # Same weighting as in _get_team_report+ feature_flag_requests=billable_feature_flag_requests(+ decide=decide_requests,+ local_evaluation=local_evaluation_requests,+ local_evaluation_not_modified=local_evaluation_not_modified_requests,+ ),api_queries_read_bytes=all_data["teams_with_api_queries_read_bytes"].get(team.id, 0),survey_responses=all_data["teams_with_survey_responses_count_in_period"].get(team.id, 0),llm_events=all_data["teams_with_ai_event_count_in_period"].get(team.id, 0),@@ -1367,6 +1374,7 @@ def update_all_orgs_billing_quotas(UsageCounter.RECORDINGS,UsageCounter.FEATURE_FLAG_REQUESTS,UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS,+ UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS,UsageCounter.CDP_INVOCATIONS,UsageCounter.SURVEY_RESPONSES,UsageCounter.AI_EVENTS,@@ -1417,6 +1425,9 @@ def update_all_orgs_billing_quotas("teams_with_local_evaluation_requests_count": dict(counter_report.counts[UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS]),+ "teams_with_local_evaluation_not_modified_requests_count": dict(+ counter_report.counts[UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS]+ ),"teams_with_api_queries_read_bytes": convert_team_usage_rows_to_dict(api_queries_usage["read_bytes"]),"teams_with_cdp_trigger_events_metrics": dict(counter_report.counts[UsageCounter.CDP_INVOCATIONS]),"teams_with_rows_exported_in_period": dict(counter_report.counts[UsageCounter.ROWS_EXPORTED]),diff --git a/ee/billing/test/test_quota_limiting.py b/ee/billing/test/test_quota_limiting.pyindex c2ea5e00c749..2485eecb9cb9 100644--- a/ee/billing/test/test_quota_limiting.py+++ b/ee/billing/test/test_quota_limiting.py@@ -346,7 +346,12 @@ class TestQuotaLimiting(BaseTest):@patch("posthoganalytics.capture")def test_todays_usage_reads_every_legacy_row_shape(self, _capture) -> None:def flag_requests(_begin, _end, request_type):- return [(str(self.team.id), 7 if request_type == FlagRequestType.DECIDE else 2)]+ counts = {+ FlagRequestType.DECIDE: 7,+ FlagRequestType.LOCAL_EVALUATION: 2,+ FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED: 3,+ }+ return [(str(self.team.id), counts[request_type])]with (self.settings(USE_TZ=False),@@ -375,7 +380,7 @@ class TestQuotaLimiting(BaseTest):self.organization.refresh_from_db()assert self.organization.usage is not None- assert self.organization.usage["feature_flag_requests"]["todays_usage"] == 7 + 2 * 10+ assert self.organization.usage["feature_flag_requests"]["todays_usage"] == 7 + 2 * 10 + 3assert self.organization.usage["rows_synced"]["todays_usage"] == 5assert self.organization.usage["rows_exported"]["todays_usage"] == 3diff --git a/posthog/constants.py b/posthog/constants.pyindex f2b0c872d949..9fc83040dd8c 100644--- a/posthog/constants.py+++ b/posthog/constants.py@@ -246,6 +246,7 @@ class EventDefinitionType(StrEnum):class FlagRequestType(StrEnum):DECIDE = "decide"LOCAL_EVALUATION = "local-evaluation"+ LOCAL_EVALUATION_NOT_MODIFIED = "local-evaluation-not-modified"REMOTE_CONFIG = "remote-config"diff --git a/posthog/tasks/test/__snapshots__/test_usage_report.ambr b/posthog/tasks/test/__snapshots__/test_usage_report.ambrindex 6adb1f4a2ea7..9c897a9606de 100644--- a/posthog/tasks/test/__snapshots__/test_usage_report.ambr+++ b/posthog/tasks/test/__snapshots__/test_usage_report.ambr@@ -41,6 +41,20 @@# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.100'''+ SELECT team_id,+ SUM(count) as count+ FROM app_metrics2+ WHERE app_source='hog_function'+ AND metric_name IN ('fetch')+ AND timestamp >= '2022-01-10 00:00:00'+ AND timestamp < '2022-01-10 23:59:59'+ GROUP BY team_id,+ metric_name+ '''+# ---+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.101+ '''+SELECT team_id,SUM(count) as countFROM app_metrics2@@ -53,7 +67,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.101+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.102'''SELECT team_id,@@ -70,7 +84,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.102+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.103'''SELECT team_id,@@ -474,6 +488,20 @@# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.33'''+ SELECT distinct_id as team,+ sum(toInt64OrZero(replaceRegexpAll(JSONExtractRaw(properties, 'count'), '^"|"$', ''))) as sum+ FROM events+ WHERE team_id = 99999+ AND event='local evaluation not modified usage'+ AND timestamp >= '2022-01-10 00:00:00'+ AND timestamp < '2022-01-10 23:59:59'+ AND has(['correct'], replaceRegexpAll(JSONExtractRaw(properties, 'token'), '^"|"$', ''))+ GROUP BY team+ '''+# ---+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.34+ '''+SELECT team_id,SUM(count) as countFROM app_metrics2@@ -485,7 +513,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.34+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.35'''SELECT team_id,@@ -499,7 +527,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.35+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.36'''SELECT team_id,@@ -513,7 +541,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.36+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.37'''SELECT team_id,@@ -528,7 +556,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.37+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.38'''SELECT team_id,@@ -541,7 +569,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.38+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.39'''SELECT team_id,@@ -558,19 +586,6 @@metric_name'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.39- '''-- SELECT team_id,- SUM(count) as count- FROM app_metrics2- WHERE app_source='logs'- AND metric_name='retention_byte_days'- AND timestamp >= '2022-01-10 00:00:00'- AND timestamp < '2022-01-10 23:59:59'- GROUP BY team_id- '''-# ---# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.4'''@@ -587,6 +602,19 @@# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.40'''+ SELECT team_id,+ SUM(count) as count+ FROM app_metrics2+ WHERE app_source='logs'+ AND metric_name='retention_byte_days'+ AND timestamp >= '2022-01-10 00:00:00'+ AND timestamp < '2022-01-10 23:59:59'+ GROUP BY team_id+ '''+# ---+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.41+ '''+SELECT team_id,multiIf(event LIKE 'helicone%', 'helicone_events', event LIKE 'langfuse%', 'langfuse_events', event LIKE 'keywords_ai%', 'keywords_ai_events', event LIKE 'traceloop%', 'traceloop_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'web', 'web_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'js', 'web_lite_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-node', 'node_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-node-mcp', 'node_mcp_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-python-mcp', 'python_mcp_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-rs-mcp', 'rust_mcp_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-go-mcp', 'go_mcp_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-ruby-mcp', 'ruby_mcp_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-edge', 'edge_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-convex', 'convex_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-android', 'android_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-kmp', 'kmp_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-flutter', 'flutter_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-ios', 'ios_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-go', 'go_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-java', 'java_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-server', 'java_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-react-native', 'react_native_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-ruby', 'ruby_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-rails', 'ruby_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-python', 'python_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-php', 'php_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-laravel', 'laravel_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-dotnet', 'dotnet_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-aspnetcore', 'dotnet_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-elixir', 'elixir_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-unity', 'unity_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-godot', 'godot_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-unreal', 'unreal_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-roblox', 'roblox_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-rs', 'rust_events', 'other') AS metric,count(1) as count@@ -631,7 +659,7 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.41+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.42'''SELECT team_id,@@ -678,7 +706,7 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.42+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.43'''SELECT team_id,@@ -725,7 +753,7 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.43+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.44'''SELECT team_id,@@ -772,7 +800,7 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.44+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.45'''SELECT team_id,@@ -819,7 +847,7 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.45+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.46'''SELECT team_id,@@ -866,7 +894,7 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.46+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.47'''SELECT team_id,@@ -913,7 +941,7 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.47+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.48'''SELECT team_id,@@ -960,7 +988,7 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.48+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.49'''SELECT team_id,@@ -1007,7 +1035,20 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.49+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.5+ '''++ SELECT team_id,+ count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count+ FROM events+ WHERE timestamp >= '2022-01-10 10:00:00'+ AND timestamp < '2022-01-10 12:00:00'+ AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']+ AND NOT startsWith(event, '$ai_')+ GROUP BY team_id+ '''+# ---+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.50'''SELECT team_id,@@ -1054,20 +1095,7 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.5- '''-- SELECT team_id,- count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count- FROM events- WHERE timestamp >= '2022-01-10 10:00:00'- AND timestamp < '2022-01-10 12:00:00'- AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']- AND NOT startsWith(event, '$ai_')- GROUP BY team_id- '''-# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.50+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.51'''SELECT team_id,@@ -1114,7 +1142,7 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.51+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.52'''SELECT team_id,@@ -1161,7 +1189,7 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.52+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.53'''SELECT team_id,@@ -1172,7 +1200,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.53+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.54'''SELECT team_id,@@ -1195,7 +1223,7 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.54+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.55'''SELECT team_id,@@ -1218,7 +1246,7 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.55+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.56'''SELECT team_id,@@ -1241,7 +1269,7 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.56+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.57'''SELECT team_id,@@ -1264,7 +1292,7 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.57+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.58'''SELECT team_id,@@ -1287,7 +1315,7 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.58+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.59'''SELECT team_id,@@ -1310,7 +1338,20 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.59+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.6+ '''++ SELECT team_id,+ count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count+ FROM events+ WHERE timestamp >= '2022-01-10 12:00:00'+ AND timestamp < '2022-01-10 14:00:00'+ AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']+ AND NOT startsWith(event, '$ai_')+ GROUP BY team_id+ '''+# ---+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.60'''SELECT team_id,@@ -1333,20 +1374,7 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.6- '''-- SELECT team_id,- count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count- FROM events- WHERE timestamp >= '2022-01-10 12:00:00'- AND timestamp < '2022-01-10 14:00:00'- AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']- AND NOT startsWith(event, '$ai_')- GROUP BY team_id- '''-# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.60+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.61'''SELECT team_id,@@ -1369,7 +1397,7 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.61+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.62'''SELECT team_id,@@ -1392,7 +1420,7 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.62+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.63'''SELECT team_id,@@ -1415,7 +1443,7 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.63+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.64'''SELECT team_id,@@ -1438,7 +1466,7 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.64+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.65'''SELECT team_id,@@ -1461,7 +1489,7 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.65+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.66'''SELECT team_id,@@ -1480,7 +1508,7 @@event'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.66+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.67'''SELECT JSONExtractInt(log_comment, 'team_id') team_id,@@ -1496,7 +1524,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.67+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.68'''SELECT team_id,@@ -1509,7 +1537,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.68+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.69'''SELECT team_id,@@ -1525,7 +1553,20 @@metric_name'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.69+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.7+ '''++ SELECT team_id,+ count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count+ FROM events+ WHERE timestamp >= '2022-01-10 14:00:00'+ AND timestamp < '2022-01-10 16:00:00'+ AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']+ AND NOT startsWith(event, '$ai_')+ GROUP BY team_id+ '''+# ---+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.70'''SELECT team_id,@@ -1541,20 +1582,7 @@metric_name'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.7- '''-- SELECT team_id,- count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count- FROM events- WHERE timestamp >= '2022-01-10 14:00:00'- AND timestamp < '2022-01-10 16:00:00'- AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']- AND NOT startsWith(event, '$ai_')- GROUP BY team_id- '''-# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.70+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.71'''SELECT team_id,@@ -1572,7 +1600,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.71+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.72'''SELECT team_id,@@ -1590,7 +1618,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.72+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.73'''SELECT team_id,@@ -1608,7 +1636,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.73+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.74'''SELECT team_id,@@ -1626,7 +1654,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.74+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.75'''SELECT team_id,@@ -1644,7 +1672,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.75+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.76'''SELECT team_id,@@ -1662,7 +1690,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.76+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.77'''SELECT team_id,@@ -1680,7 +1708,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.77+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.78'''SELECT team_id,@@ -1698,7 +1726,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.78+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.79'''SELECT team_id,@@ -1716,38 +1744,38 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.79+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.8'''SELECT team_id,count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as countFROM events- WHERE timestamp >= '2022-01-10 18:00:00'- AND timestamp < '2022-01-10 20:00:00'+ WHERE timestamp >= '2022-01-10 16:00:00'+ AND timestamp < '2022-01-10 18:00:00'AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']AND NOT startsWith(event, '$ai_')- AND ($group_0 != ''- OR $group_1 != ''- OR $group_2 != ''- OR $group_3 != ''- OR $group_4 != '')GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.8+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.80'''SELECT team_id,count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as countFROM events- WHERE timestamp >= '2022-01-10 16:00:00'- AND timestamp < '2022-01-10 18:00:00'+ WHERE timestamp >= '2022-01-10 18:00:00'+ AND timestamp < '2022-01-10 20:00:00'AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']AND NOT startsWith(event, '$ai_')+ AND ($group_0 != ''+ OR $group_1 != ''+ OR $group_2 != ''+ OR $group_3 != ''+ OR $group_4 != '')GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.80+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.81'''SELECT team_id,@@ -1765,7 +1793,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.81+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.82'''SELECT team_id,@@ -1783,7 +1811,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.82+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.83'''SELECT team_id,@@ -1794,7 +1822,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.83+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.84'''SELECT team_id,@@ -1817,7 +1845,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.84+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.85'''SELECT team_id,@@ -1841,7 +1869,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.85+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.86'''SELECT team_id,@@ -1865,7 +1893,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.86+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.87'''WITH JSONExtractInt(log_comment, 'team_id') as team_id,JSONExtractString(log_comment, 'query_type') as query_type,@@ -1884,7 +1912,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.87+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.88'''WITH JSONExtractInt(log_comment, 'team_id') as team_id,JSONExtractString(log_comment, 'query_type') as query_type,@@ -1903,7 +1931,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.88+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.89'''WITH JSONExtractInt(log_comment, 'team_id') as team_id,JSONExtractString(log_comment, 'query_type') as query_type,@@ -1922,7 +1950,20 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.89+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.9+ '''++ SELECT team_id,+ count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count+ FROM events+ WHERE timestamp >= '2022-01-10 18:00:00'+ AND timestamp < '2022-01-10 20:00:00'+ AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']+ AND NOT startsWith(event, '$ai_')+ GROUP BY team_id+ '''+# ---+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.90'''WITH JSONExtractInt(log_comment, 'team_id') as team_id,JSONExtractString(log_comment, 'query_type') as query_type,@@ -1941,20 +1982,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.9- '''-- SELECT team_id,- count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count- FROM events- WHERE timestamp >= '2022-01-10 18:00:00'- AND timestamp < '2022-01-10 20:00:00'- AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']- AND NOT startsWith(event, '$ai_')- GROUP BY team_id- '''-# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.90+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.91'''WITH JSONExtractInt(log_comment, 'team_id') as team_id,JSONExtractString(log_comment, 'query_type') as query_type,@@ -1973,7 +2001,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.91+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.92'''WITH JSONExtractInt(log_comment, 'team_id') as team_id,JSONExtractString(log_comment, 'query_type') as query_type,@@ -1992,7 +2020,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.92+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.93'''WITH JSONExtractInt(log_comment, 'team_id') as team_id,JSONExtractString(log_comment, 'query_type') as query_type,@@ -2012,7 +2040,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.93+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.94'''WITH JSONExtractInt(log_comment, 'team_id') as team_id,JSONExtractString(log_comment, 'query_type') as query_type,@@ -2032,7 +2060,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.94+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.95'''WITH JSONExtractInt(log_comment, 'team_id') as team_id,JSONExtractString(log_comment, 'query_type') as query_type,@@ -2052,7 +2080,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.95+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.96'''WITH JSONExtractInt(log_comment, 'team_id') as team_id,JSONExtractString(log_comment, 'query_type') as query_type,@@ -2072,7 +2100,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.96+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.97'''WITH JSONExtractInt(log_comment, 'team_id') as team_id,JSONExtractString(log_comment, 'query_type') as query_type,@@ -2092,7 +2120,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.97+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.98'''WITH JSONExtractInt(log_comment, 'team_id') as team_id,JSONExtractString(log_comment, 'query_type') as query_type,@@ -2112,7 +2140,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.98+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.99'''SELECT team_id,@@ -2127,17 +2155,3 @@metric_name'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.99- '''-- SELECT team_id,- SUM(count) as count- FROM app_metrics2- WHERE app_source='hog_function'- AND metric_name IN ('fetch')- AND timestamp >= '2022-01-10 00:00:00'- AND timestamp < '2022-01-10 23:59:59'- GROUP BY team_id,- metric_name- '''-# ---diff --git a/posthog/tasks/test/__snapshots__/test_usage_report.new_events_schema.ambr b/posthog/tasks/test/__snapshots__/test_usage_report.new_events_schema.ambrindex e73977bcdd55..29da8d2a3e4b 100644--- a/posthog/tasks/test/__snapshots__/test_usage_report.new_events_schema.ambr+++ b/posthog/tasks/test/__snapshots__/test_usage_report.new_events_schema.ambr@@ -332,6 +332,35 @@# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.29]'''+ SELECT distinct_id as team,+ sum(toInt64OrZero(ifNull(if(if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.^count), '\\/', '/'), toJSONString(properties.^count)) != '{}', if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.^count), '\\/', '/'), toJSONString(properties.^count)), if(ascii(toString(properties.count)) IN (91, 123)+ AND ascii(if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.count), '\\/', '/'), toJSONString(properties.count))) IN (91, 123), nullIf(nullIf(if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.count), '\\/', '/'), toJSONString(properties.count)), '[]'), '{}'), toString(properties.count))), ''))) as sum+ FROM events_json+ WHERE team_id = 99999+ AND event='local evaluation not modified usage'+ AND timestamp >= '2022-01-10 00:00:00'+ AND timestamp < '2022-01-10 23:59:59'+ AND has(['correct'], ifNull(if(if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.^token), '\\/', '/'), toJSONString(properties.^token)) != '{}', if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.^token), '\\/', '/'), toJSONString(properties.^token)), if(ascii(toString(properties.token)) IN (91, 123)+ AND ascii(if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.token), '\\/', '/'), toJSONString(properties.token))) IN (91, 123), nullIf(nullIf(if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.token), '\\/', '/'), toJSONString(properties.token)), '[]'), '{}'), toString(properties.token))), ''))+ GROUP BY team+ '''+# ---+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.2]+ '''++ SELECT team_id,+ count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count+ FROM events_json+ WHERE timestamp >= '2022-01-10 04:00:00'+ AND timestamp < '2022-01-10 06:00:00'+ AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']+ AND NOT startsWith(event, '$ai_')+ GROUP BY team_id+ '''+# ---+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.30]+ '''+SELECT team_id,multiIf(event LIKE 'helicone%', 'helicone_events', event LIKE 'langfuse%', 'langfuse_events', event LIKE 'keywords_ai%', 'keywords_ai_events', event LIKE 'traceloop%', 'traceloop_events', ifNull(if(if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.^`$lib`), '\\/', '/'), toJSONString(properties.^`$lib`)) != '{}', if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.^`$lib`), '\\/', '/'), toJSONString(properties.^`$lib`)), if(ascii(toString(properties.`$lib`)) IN (91, 123)AND ascii(if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.`$lib`), '\\/', '/'), toJSONString(properties.`$lib`))) IN (91, 123), nullIf(nullIf(if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.`$lib`), '\\/', '/'), toJSONString(properties.`$lib`)), '[]'), '{}'), toString(properties.`$lib`))), '') = 'web', 'web_events', ifNull(if(if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.^`$lib`), '\\/', '/'), toJSONString(properties.^`$lib`)) != '{}', if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.^`$lib`), '\\/', '/'), toJSONString(properties.^`$lib`)), if(ascii(toString(properties.`$lib`)) IN (91, 123)@@ -408,20 +437,7 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.2]- '''-- SELECT team_id,- count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count- FROM events_json- WHERE timestamp >= '2022-01-10 04:00:00'- AND timestamp < '2022-01-10 06:00:00'- AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']- AND NOT startsWith(event, '$ai_')- GROUP BY team_id- '''-# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.30]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.31]'''SELECT team_id,@@ -500,7 +516,7 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.31]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.32]'''SELECT team_id,@@ -579,7 +595,7 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.32]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.33]'''SELECT team_id,@@ -658,7 +674,7 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.33]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.34]'''SELECT team_id,@@ -737,7 +753,7 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.34]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.35]'''SELECT team_id,@@ -816,7 +832,7 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.35]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.36]'''SELECT team_id,@@ -895,7 +911,7 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.36]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.37]'''SELECT team_id,@@ -974,7 +990,7 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.37]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.38]'''SELECT team_id,@@ -1053,7 +1069,7 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.38]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.39]'''SELECT team_id,@@ -1132,7 +1148,20 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.39]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.3]+ '''++ SELECT team_id,+ count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count+ FROM events_json+ WHERE timestamp >= '2022-01-10 06:00:00'+ AND timestamp < '2022-01-10 08:00:00'+ AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']+ AND NOT startsWith(event, '$ai_')+ GROUP BY team_id+ '''+# ---+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.40]'''SELECT team_id,@@ -1211,20 +1240,7 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.3]- '''-- SELECT team_id,- count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count- FROM events_json- WHERE timestamp >= '2022-01-10 06:00:00'- AND timestamp < '2022-01-10 08:00:00'- AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']- AND NOT startsWith(event, '$ai_')- GROUP BY team_id- '''-# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.40]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.41]'''SELECT team_id,@@ -1303,7 +1319,7 @@metric'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.41]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.42]'''SELECT team_id,@@ -1330,7 +1346,7 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.42]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.43]'''SELECT team_id,@@ -1357,7 +1373,7 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.43]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.44]'''SELECT team_id,@@ -1384,7 +1400,7 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.44]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.45]'''SELECT team_id,@@ -1411,7 +1427,7 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.45]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.46]'''SELECT team_id,@@ -1438,7 +1454,7 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.46]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.47]'''SELECT team_id,@@ -1465,7 +1481,7 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.47]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.48]'''SELECT team_id,@@ -1492,7 +1508,7 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.48]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.49]'''SELECT team_id,@@ -1519,7 +1535,20 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.49]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.4]+ '''++ SELECT team_id,+ count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count+ FROM events_json+ WHERE timestamp >= '2022-01-10 08:00:00'+ AND timestamp < '2022-01-10 10:00:00'+ AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']+ AND NOT startsWith(event, '$ai_')+ GROUP BY team_id+ '''+# ---+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.50]'''SELECT team_id,@@ -1546,20 +1575,7 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.4]- '''-- SELECT team_id,- count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count- FROM events_json- WHERE timestamp >= '2022-01-10 08:00:00'- AND timestamp < '2022-01-10 10:00:00'- AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']- AND NOT startsWith(event, '$ai_')- GROUP BY team_id- '''-# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.50]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.51]'''SELECT team_id,@@ -1586,7 +1602,7 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.51]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.52]'''SELECT team_id,@@ -1613,7 +1629,7 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.52]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.53]'''SELECT team_id,@@ -1640,7 +1656,7 @@ai_lib'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.53]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.54]'''SELECT team_id,@@ -1658,7 +1674,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.54]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.55]'''SELECT team_id,@@ -1676,7 +1692,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.55]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.56]'''SELECT team_id,@@ -1694,7 +1710,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.56]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.57]'''SELECT team_id,@@ -1712,7 +1728,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.57]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.58]'''SELECT team_id,@@ -1730,7 +1746,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.58]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.59]'''SELECT team_id,@@ -1748,38 +1764,38 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.59]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.5]'''SELECT team_id,count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as countFROM events_json- WHERE timestamp >= '2022-01-10 12:00:00'- AND timestamp < '2022-01-10 14:00:00'+ WHERE timestamp >= '2022-01-10 10:00:00'+ AND timestamp < '2022-01-10 12:00:00'AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']AND NOT startsWith(event, '$ai_')- AND (properties.`$group_0` != ''- OR properties.`$group_1` != ''- OR properties.`$group_2` != ''- OR properties.`$group_3` != ''- OR properties.`$group_4` != '')GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.5]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.60]'''SELECT team_id,count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as countFROM events_json- WHERE timestamp >= '2022-01-10 10:00:00'- AND timestamp < '2022-01-10 12:00:00'+ WHERE timestamp >= '2022-01-10 12:00:00'+ AND timestamp < '2022-01-10 14:00:00'AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']AND NOT startsWith(event, '$ai_')+ AND (properties.`$group_0` != ''+ OR properties.`$group_1` != ''+ OR properties.`$group_2` != ''+ OR properties.`$group_3` != ''+ OR properties.`$group_4` != '')GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.60]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.61]'''SELECT team_id,@@ -1797,7 +1813,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.61]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.62]'''SELECT team_id,@@ -1815,7 +1831,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.62]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.63]'''SELECT team_id,@@ -1833,7 +1849,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.63]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.64]'''SELECT team_id,@@ -1851,7 +1867,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.64]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.65]'''SELECT team_id,@@ -1869,7 +1885,7 @@GROUP BY team_id'''# ----# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.65]+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.66]'''SELECT team_id,diff --git a/posthog/tasks/test/test_usage_counters.py b/posthog/tasks/test/test_usage_counters.pyindex 2ac7d7ba0f9c..003d65c65e58 100644--- a/posthog/tasks/test/test_usage_counters.py+++ b/posthog/tasks/test/test_usage_counters.py@@ -11,6 +11,7 @@ from parameterized import parameterizedfrom posthog.tasks import usage_reportfrom posthog.usage_counters import (COUNTER_FLAG_NAMES,+ PLAN_DEFERRED_COUNTERS,SHADOW_FAILURES,UsageCounter,UsageCounterCaller,@@ -169,9 +170,15 @@ class TestUsageCounterReport(SimpleTestCase):"get_teams_with_cdp_billable_invocations_in_period": self.legacy[UsageCounter.CDP_INVOCATIONS],"get_teams_with_feature_flag_requests_count_in_period": Mock(side_effect=lambda begin, end, kind: self.legacy[- UsageCounter.FEATURE_FLAG_REQUESTS- if kind == usage_report.FlagRequestType.DECIDE- else UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS+ {+ usage_report.FlagRequestType.DECIDE: UsageCounter.FEATURE_FLAG_REQUESTS,+ usage_report.FlagRequestType.LOCAL_EVALUATION: (+ UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS+ ),+ usage_report.FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED: (+ UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS+ ),+ }[kind]](begin, end)),"get_teams_with_workflow_emails_sent_in_period": self.legacy[UsageCounter.WORKFLOW_EMAILS],@@ -217,13 +224,14 @@ class TestUsageCounterReport(SimpleTestCase):report = service.fetch_report(period, plan=plan)assert flag.call_count == len(COUNTER_FLAG_NAMES)+ planned_counters = [counter for counter in UsageCounter if counter not in PLAN_DEFERRED_COUNTERS]assert report.counts == {- **{key: [] for key in (*UsageCounter, *SANDBOX_COMPUTE_RESOURCE_KEYS)},+ **{key: [] for key in (*planned_counters, *SANDBOX_COMPUTE_RESOURCE_KEYS)},UsageCounter.CDP_INVOCATIONS.value: [(1, 12)],UsageCounter.WORKFLOW_EMAILS.value: [(1, 3)],}assert report.usage_sources == {- **{counter.value.removeprefix("teams_with_"): "legacy" for counter in UsageCounter},+ **{counter.value.removeprefix("teams_with_"): "legacy" for counter in planned_counters},"cdp_billable_invocations_in_period": "both",}assert report.counter_comparisons == (@@ -397,6 +405,7 @@ class TestUsageCounterReport(SimpleTestCase):legacy_by_team={}, realtime_by_team={})for counter in COUNTER_FLAG_NAMES+ if counter not in PLAN_DEFERRED_COUNTERS}@parameterized.expand([("success",), ("records",), ("legacy_comparison",)])@@ -466,12 +475,20 @@ class TestUsageCounterReport(SimpleTestCase):UsageCounter.CDP_INVOCATIONS: ("cdp_billable_invocations", 111),UsageCounter.FEATURE_FLAG_REQUESTS: ("feature_flag_requests", 108),UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS: ("feature_flag_local_evaluation_requests", 109),+ UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS: (+ "feature_flag_local_evaluation_not_modified_requests",+ 115,+ ),UsageCounter.WORKFLOW_EMAILS: ("workflow_emails_sent", 112),UsageCounter.WORKFLOW_PUSH: ("workflow_push_sent", 113),UsageCounter.WORKFLOW_SMS: ("workflow_sms_sent", 114),UsageCounter.WORKFLOW_INVOCATIONS: ("workflow_billable_invocations", 110),}- flag_counters = {UsageCounter.FEATURE_FLAG_REQUESTS, UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS}+ flag_counters = {+ UsageCounter.FEATURE_FLAG_REQUESTS,+ UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS,+ UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS,+ }for index, counter in enumerate(counter_keys):self.legacy[counter].return_value = [("1" if counter in flag_counters else 1, index + 1)]self.exceptions.side_effect = None@@ -491,6 +508,8 @@ class TestUsageCounterReport(SimpleTestCase):)),patch("posthoganalytics.get_feature_flag", return_value=mode if source == "flag" else "legacy"),+ # Deferred counters read their record keys once they join plans, so check those keys here too.+ patch("posthog.usage_counters.PLAN_DEFERRED_COUNTERS", frozenset()),):service = UsageCounterService()plan = service.resolve_plan(period, complete=True, counters=tuple(counter_keys))diff --git a/posthog/tasks/test/test_usage_report.py b/posthog/tasks/test/test_usage_report.pyindex d1ab0aa47617..210fc6e14ee2 100644--- a/posthog/tasks/test/test_usage_report.py+++ b/posthog/tasks/test/test_usage_report.py@@ -795,6 +795,7 @@ class TestUsageReport(APIBaseTest, ClickhouseTestMixin, ClickhouseDestroyTablesM"resolved_symbol_sets_count": 0,"decide_requests_count_in_period": 0,"local_evaluation_requests_count_in_period": 0,+ "local_evaluation_not_modified_requests_count_in_period": 0,"billable_feature_flag_requests_count_in_period": 0,"survey_count": 0,"survey_responses_count_in_period": 1,@@ -901,6 +902,7 @@ class TestUsageReport(APIBaseTest, ClickhouseTestMixin, ClickhouseDestroyTablesM"resolved_symbol_sets_count": 0,"decide_requests_count_in_period": 0,"local_evaluation_requests_count_in_period": 0,+ "local_evaluation_not_modified_requests_count_in_period": 0,"billable_feature_flag_requests_count_in_period": 0,"survey_count": 0,"survey_responses_count_in_period": 1,@@ -1001,6 +1003,7 @@ class TestUsageReport(APIBaseTest, ClickhouseTestMixin, ClickhouseDestroyTablesM"resolved_symbol_sets_count": 0,"decide_requests_count_in_period": 0,"local_evaluation_requests_count_in_period": 0,+ "local_evaluation_not_modified_requests_count_in_period": 0,"billable_feature_flag_requests_count_in_period": 0,"survey_count": 0,"survey_responses_count_in_period": 0,@@ -1124,6 +1127,7 @@ class TestUsageReport(APIBaseTest, ClickhouseTestMixin, ClickhouseDestroyTablesM"resolved_symbol_sets_count": 0,"decide_requests_count_in_period": 0,"local_evaluation_requests_count_in_period": 0,+ "local_evaluation_not_modified_requests_count_in_period": 0,"billable_feature_flag_requests_count_in_period": 0,"survey_count": 0,"survey_responses_count_in_period": 0,@@ -1230,6 +1234,7 @@ class TestUsageReport(APIBaseTest, ClickhouseTestMixin, ClickhouseDestroyTablesM"resolved_symbol_sets_count": 0,"decide_requests_count_in_period": 0,"local_evaluation_requests_count_in_period": 0,+ "local_evaluation_not_modified_requests_count_in_period": 0,"billable_feature_flag_requests_count_in_period": 0,"survey_count": 0,"survey_responses_count_in_period": 0,@@ -2147,6 +2152,14 @@ class TestFeatureFlagsUsageReport(ClickhouseDestroyTablesMixin, TestCase, Clickhteam=self.analytics_team,)+ _create_event(+ distinct_id="3",+ event="local evaluation not modified usage",+ properties={"count": 5, "token": "correct"},+ timestamp=now(),+ team=self.analytics_team,+ )+for i in range(5):_create_event(distinct_id="4",@@ -2198,11 +2211,13 @@ class TestFeatureFlagsUsageReport(ClickhouseDestroyTablesMixin, TestCase, Clickhassert org_1_report["organization_name"] == "Org 1"assert org_1_report["local_evaluation_requests_count_in_period"] == 11+ assert org_1_report["local_evaluation_not_modified_requests_count_in_period"] == 5assert org_1_report["decide_requests_count_in_period"] == 0- assert org_1_report["billable_feature_flag_requests_count_in_period"] == 110+ assert org_1_report["billable_feature_flag_requests_count_in_period"] == 115assert org_1_report["teams"]["3"]["local_evaluation_requests_count_in_period"] == 10+ assert org_1_report["teams"]["3"]["local_evaluation_not_modified_requests_count_in_period"] == 5assert org_1_report["teams"]["4"]["local_evaluation_requests_count_in_period"] == 1- assert org_1_report["teams"]["3"]["billable_feature_flag_requests_count_in_period"] == 100+ assert org_1_report["teams"]["3"]["billable_feature_flag_requests_count_in_period"] == 105assert org_1_report["teams"]["4"]["billable_feature_flag_requests_count_in_period"] == 10# because of wrong token, Org 2 has no decide counts.@@ -2526,6 +2541,7 @@ class TestHasNonZeroUsage(SimpleTestCase):("empty", None),("events", "event_count_in_period"),("mcp_events", "mcp_event_count_in_period"),+ ("local_evaluation_not_modified", "local_evaluation_not_modified_requests_count_in_period"),("business_knowledge_chunks", "business_knowledge_chunks_in_period"),("business_knowledge_queries", "business_knowledge_queries_in_period"),("logs_bytes", "logs_bytes_in_period"),diff --git a/posthog/tasks/usage_report.py b/posthog/tasks/usage_report.pyindex e980c235443a..f7c9d01aec2c 100644--- a/posthog/tasks/usage_report.py+++ b/posthog/tasks/usage_report.py@@ -72,6 +72,7 @@ from products.cdp.backend.models.plugin import PluginConfigfrom products.dashboards.backend.models.dashboard import Dashboardfrom products.data_modeling.backend.facade.models import DataWarehouseSavedQueryfrom products.error_tracking.backend.facade import api as error_tracking_api+from products.feature_flags.backend.flag_analytics import USAGE_EVENT_NAMES, billable_feature_flag_requestsfrom products.feature_flags.backend.models.feature_flag import FeatureFlagfrom products.replay_vision.backend.billing import (get_replay_vision_credits_by_team,@@ -253,6 +254,7 @@ class UsageReportCounters:ff_active_count: intdecide_requests_count_in_period: intlocal_evaluation_requests_count_in_period: int+ local_evaluation_not_modified_requests_count_in_period: intbillable_feature_flag_requests_count_in_period: int# Queries@@ -1549,7 +1551,7 @@ def get_teams_with_feature_flag_requests_count_in_period(team_to_query = 1 if get_instance_region() == "EU" else 2validity_token = settings.DECIDE_BILLING_ANALYTICS_TOKEN- target_event = "decide usage" if request_type == FlagRequestType.DECIDE else "local evaluation usage"+ target_event = USAGE_EVENT_NAMES[request_type]use_new = use_new_events_schema(None)count_expr, _ = get_property_string_expr("events", "count", "'count'", "properties", use_new_events_schema=use_new)@@ -1590,7 +1592,7 @@ def get_teams_with_feature_flag_requests_sdk_breakdown_in_period(team_to_query = 1 if get_instance_region() == "EU" else 2validity_token = settings.DECIDE_BILLING_ANALYTICS_TOKEN- target_event = "decide usage" if request_type == FlagRequestType.DECIDE else "local evaluation usage"+ target_event = USAGE_EVENT_NAMES[request_type]use_new = use_new_events_schema(None)sdk_breakdown_expr, _ = get_property_string_expr(@@ -3102,6 +3104,7 @@ def has_non_zero_usage(report: UsageReportCounters) -> bool:or report.mobile_recording_count_in_period > 0or report.decide_requests_count_in_period > 0or report.local_evaluation_requests_count_in_period > 0+ or report.local_evaluation_not_modified_requests_count_in_period > 0or report.survey_responses_count_in_period > 0or report.rows_synced_in_period > 0or report.free_historical_rows_synced_in_period > 0@@ -3512,6 +3515,9 @@ def _get_team_report(all_data: dict[str, Any], team: Team) -> UsageReportCounterlocal_evaluation_requests_count_in_period = all_data["teams_with_local_evaluation_requests_count_in_period"].get(team.id, 0)+ local_evaluation_not_modified_requests_count_in_period = all_data[+ "teams_with_local_evaluation_not_modified_requests_count_in_period"+ ].get(team.id, 0)logs_bytes_in_period = all_data["teams_with_logs_bytes_in_period"].get(team.id, 0)apm_tracing_bytes_in_period = all_data["teams_with_apm_tracing_bytes_in_period"].get(team.id, 0)return UsageReportCounters(@@ -3556,8 +3562,12 @@ def _get_team_report(all_data: dict[str, Any], team: Team) -> UsageReportCountergroup_types_total=all_data["teams_with_group_types_total"].get(team.id, 0),decide_requests_count_in_period=decide_requests_count_in_period,local_evaluation_requests_count_in_period=local_evaluation_requests_count_in_period,- billable_feature_flag_requests_count_in_period=decide_requests_count_in_period- + (local_evaluation_requests_count_in_period * 10),+ local_evaluation_not_modified_requests_count_in_period=local_evaluation_not_modified_requests_count_in_period,+ billable_feature_flag_requests_count_in_period=billable_feature_flag_requests(+ decide=decide_requests_count_in_period,+ local_evaluation=local_evaluation_requests_count_in_period,+ local_evaluation_not_modified=local_evaluation_not_modified_requests_count_in_period,+ ),dashboard_count=all_data["teams_with_dashboard_count"].get(team.id, 0),dashboard_template_count=all_data["teams_with_dashboard_template_count"].get(team.id, 0),dashboard_shared_count=all_data["teams_with_dashboard_shared_count"].get(team.id, 0),diff --git a/posthog/temporal/tests/usage_report/test_aggregator.py b/posthog/temporal/tests/usage_report/test_aggregator.pyindex 5a93e2a7b6f4..90b01a7749ad 100644--- a/posthog/temporal/tests/usage_report/test_aggregator.py+++ b/posthog/temporal/tests/usage_report/test_aggregator.py@@ -26,6 +26,7 @@ from posthog.tasks.usage_report import (from posthog.temporal.tests.usage_report.test_aggregate_activity import _instance_metadatafrom posthog.temporal.usage_report.aggregator import (add_pre_business_knowledge_usage_patch_defaults,+ add_pre_local_evaluation_not_modified_patch_defaults,add_pre_logs_distinct_patterns_patch_defaults,add_pre_mcp_event_count_patch_defaults,add_pre_replay_vision_spend_projection_patch_defaults,@@ -176,6 +177,10 @@ def test_load_all_data_defaults_compute_for_pre_patch_workflow_history() -> None(add_pre_logs_distinct_patterns_patch_defaults, "teams_with_logs_distinct_patterns_in_period"),(add_pre_transactional_emails_patch_defaults, "teams_with_workflow_transactional_emails_sent_in_period"),(add_pre_mcp_event_count_patch_defaults, "teams_with_mcp_event_count_in_period"),+ (+ add_pre_local_evaluation_not_modified_patch_defaults,+ "teams_with_local_evaluation_not_modified_requests_count_in_period",+ ),],)def test_load_all_data_defaults_single_query_for_pre_patch_workflow_history(diff --git a/posthog/temporal/tests/usage_report/test_parity.py b/posthog/temporal/tests/usage_report/test_parity.pyindex f83db418431b..aed181f6894a 100644--- a/posthog/temporal/tests/usage_report/test_parity.py+++ b/posthog/temporal/tests/usage_report/test_parity.py@@ -84,6 +84,7 @@ def _seed_all_data(team_a_id: int, team_b_id: int, team_c_id: int) -> dict[str,"teams_with_recording_bytes_in_period": {team_a_id: 1_234_567, team_c_id: 89_012},"teams_with_decide_requests_count_in_period": {team_a_id: 1_000, team_b_id: 200, team_c_id: 50},"teams_with_local_evaluation_requests_count_in_period": {team_b_id: 5},+ "teams_with_local_evaluation_not_modified_requests_count_in_period": {team_b_id: 40, team_c_id: 9},"teams_with_dashboard_count": {team_a_id: 4, team_b_id: 2, team_c_id: 1},"teams_with_ff_count": {team_a_id: 7, team_b_id: 3},"teams_with_survey_count": {team_a_id: 1, team_c_id: 2},@@ -361,9 +362,13 @@ def test_end_to_end_parity_celery_task_vs_temporal_activity(lambda begin, end, kind: [(str(team_id), total)for team_id, total in seeded[- UsageCounter.FEATURE_FLAG_REQUESTS- if kind == usage_report.FlagRequestType.DECIDE- else UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS+ {+ usage_report.FlagRequestType.DECIDE: UsageCounter.FEATURE_FLAG_REQUESTS,+ usage_report.FlagRequestType.LOCAL_EVALUATION: UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS,+ usage_report.FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED: (+ UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS+ ),+ }[kind]].items()],)diff --git a/posthog/temporal/tests/usage_report/test_workflow.py b/posthog/temporal/tests/usage_report/test_workflow.pyindex 3ead8755e816..3bb690b7f0ac 100644--- a/posthog/temporal/tests/usage_report/test_workflow.py+++ b/posthog/temporal/tests/usage_report/test_workflow.py@@ -48,6 +48,7 @@ from posthog.temporal.usage_report.types import (from posthog.temporal.usage_report.workflow import (AGGREGATION_QUEUE_PATCH_ID,BUSINESS_KNOWLEDGE_USAGE_QUERY_NAME,+ LOCAL_EVALUATION_NOT_MODIFIED_QUERY_NAME,LOGS_DISTINCT_PATTERNS_QUERY_NAME,MCP_EVENT_COUNT_QUERY_NAME,PARALLEL_COUNTER_QUERIES_PATCH_ID,@@ -58,6 +59,7 @@ from posthog.temporal.usage_report.workflow import (USAGE_COUNTER_SERVICE_PATCH_ID,RunUsageReportsWorkflow,_queries_for_business_knowledge_usage_patch,+ _queries_for_local_evaluation_not_modified_patch,_queries_for_logs_distinct_patterns_patch,_queries_for_mcp_event_count_patch,_queries_for_replay_vision_spend_projection_patch,@@ -68,6 +70,7 @@ from posthog.temporal.usage_report.workflow import (from posthog.usage_counters import (COUNTER_FLAG_NAMES,LEGACY_QUERY_NAMES,+ PLAN_DEFERRED_COUNTERS,UsageCounter,UsageCounterMode,UsageCounterPlan,@@ -90,6 +93,11 @@ def test_sandbox_compute_query_is_versioned_for_existing_histories() -> None:REPLAY_VISION_SPEND_PROJECTION_QUERY_NAME,),("business_knowledge_usage", _queries_for_business_knowledge_usage_patch, BUSINESS_KNOWLEDGE_USAGE_QUERY_NAME),+ (+ "local_evaluation_not_modified",+ _queries_for_local_evaluation_not_modified_patch,+ LOCAL_EVALUATION_NOT_MODIFIED_QUERY_NAME,+ ),])def test_single_query_is_versioned_for_existing_histories(@@ -471,6 +479,7 @@ async def test_workflow_runs_query_then_aggregate(report_behavior: str, caplog:assert plans[0].modes == {counter: expected_mode if counter in COUNTER_FLAG_NAMES else UsageCounterMode.LEGACYfor counter in UsageCounter+ if counter not in PLAN_DEFERRED_COUNTERS}assert aggregate_payloads[0].ctx.usage_counter_plan == (plans[0] if plans else None)diff --git a/posthog/temporal/usage_report/activities.py b/posthog/temporal/usage_report/activities.pyindex 21ab30256a62..5fc158f11489 100644--- a/posthog/temporal/usage_report/activities.py+++ b/posthog/temporal/usage_report/activities.py@@ -27,6 +27,7 @@ from posthog.temporal.common.heartbeat import Heartbeaterfrom posthog.temporal.common.metrics import ExecutionTimeRecorderfrom posthog.temporal.usage_report.aggregator import (add_pre_business_knowledge_usage_patch_defaults,+ add_pre_local_evaluation_not_modified_patch_defaults,add_pre_logs_distinct_patterns_patch_defaults,add_pre_mcp_event_count_patch_defaults,add_pre_replay_vision_spend_projection_patch_defaults,@@ -256,6 +257,7 @@ async def aggregate_and_chunk_org_reports(inputs: AggregateInputs) -> AggregateRadd_pre_mcp_event_count_patch_defaults(all_data, inputs.query_results)add_pre_replay_vision_spend_projection_patch_defaults(all_data, inputs.query_results)add_pre_business_knowledge_usage_patch_defaults(all_data, inputs.query_results)+ add_pre_local_evaluation_not_modified_patch_defaults(all_data, inputs.query_results)check_cancelled()counter_report = load_usage_counter_report(inputs)all_data.update({field: dict(rows) for field, rows in counter_report.counts.items()})diff --git a/posthog/temporal/usage_report/aggregator.py b/posthog/temporal/usage_report/aggregator.pyindex 10145604274d..b195c8973082 100644--- a/posthog/temporal/usage_report/aggregator.py+++ b/posthog/temporal/usage_report/aggregator.py@@ -53,6 +53,7 @@ from posthog.usage_counters import UsageCounterReportfrom products.ai_observability.backend.offline_evaluation_billing import get_offline_evaluation_usage_SANDBOX_COMPUTE_QUERY_NAME = "sandbox_compute_usage"+_LOCAL_EVALUATION_NOT_MODIFIED_QUERY_NAME = "teams_with_local_evaluation_not_modified_requests_count_in_period"_SANDBOX_COMPUTE_DESTINATION_KEYS = ("teams_with_sandbox_compute_credits_used_in_period","teams_with_sandbox_compute_cpu_millicore_seconds_in_period",@@ -147,6 +148,13 @@ def add_pre_business_knowledge_usage_patch_defaults(all_data[key] = {}+def add_pre_local_evaluation_not_modified_patch_defaults(+ all_data: dict[str, dict[int, int]], query_results: list[RunQueryToS3Result]+) -> None:+ if not any(result.query_name == _LOCAL_EVALUATION_NOT_MODIFIED_QUERY_NAME for result in query_results):+ all_data[_LOCAL_EVALUATION_NOT_MODIFIED_QUERY_NAME] = {}++def iter_chunk_lines(org_reports: Iterable[OrgReport],instance_metadata: InstanceMetadata,diff --git a/posthog/temporal/usage_report/queries.py b/posthog/temporal/usage_report/queries.pyindex 872c8c3d1991..407b8b4ff109 100644--- a/posthog/temporal/usage_report/queries.py+++ b/posthog/temporal/usage_report/queries.py@@ -404,6 +404,12 @@ QUERIES: list[QuerySpec] = [name="teams_with_local_evaluation_requests_count_in_period",fn=lambda b, e: UsageCounterService().get_legacy(UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS, b, e),),+ QuerySpec(+ name="teams_with_local_evaluation_not_modified_requests_count_in_period",+ fn=lambda b, e: UsageCounterService().get_legacy(+ UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS, b, e+ ),+ ),# ---- ClickHouse: query metrics -------------------------------------------QuerySpec(name="teams_with_query_app_bytes_read",diff --git a/posthog/temporal/usage_report/workflow.py b/posthog/temporal/usage_report/workflow.pyindex f44fdbe69dd7..37c88234f41c 100644--- a/posthog/temporal/usage_report/workflow.py+++ b/posthog/temporal/usage_report/workflow.py@@ -66,6 +66,8 @@ REPLAY_VISION_SPEND_PROJECTION_QUERY_PATCH_ID = "usage-report-replay-vision-spenREPLAY_VISION_SPEND_PROJECTION_QUERY_NAME = "replay_vision_spend_projection"BUSINESS_KNOWLEDGE_USAGE_QUERY_PATCH_ID = "usage-report-business-knowledge-usage-query-2026-11"BUSINESS_KNOWLEDGE_USAGE_QUERY_NAME = "business_knowledge_usage"+LOCAL_EVALUATION_NOT_MODIFIED_QUERY_PATCH_ID = "usage-report-local-evaluation-not-modified-query-2026-11"+LOCAL_EVALUATION_NOT_MODIFIED_QUERY_NAME = UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS.valuedef _queries_for_sandbox_compute_patch(patch_applied: bool) -> list[QuerySpec]:@@ -104,6 +106,12 @@ def _queries_for_business_knowledge_usage_patch(queries: list[QuerySpec], patch_return [spec for spec in queries if spec.name != BUSINESS_KNOWLEDGE_USAGE_QUERY_NAME]+def _queries_for_local_evaluation_not_modified_patch(queries: list[QuerySpec], patch_applied: bool) -> list[QuerySpec]:+ if patch_applied:+ return queries+ return [spec for spec in queries if spec.name != LOCAL_EVALUATION_NOT_MODIFIED_QUERY_NAME]++def build_context(inputs: RunUsageReportsInputs, run_id: str, now: datetime) -> WorkflowContext:"""Compute the period and S3 layout context. Pure function so it's safeto call from the workflow body (no real-time access).@@ -177,6 +185,9 @@ class RunUsageReportsWorkflow(PostHogWorkflow):queries = _queries_for_business_knowledge_usage_patch(queries, workflow.patched(BUSINESS_KNOWLEDGE_USAGE_QUERY_PATCH_ID))+ queries = _queries_for_local_evaluation_not_modified_patch(+ queries, workflow.patched(LOCAL_EVALUATION_NOT_MODIFIED_QUERY_PATCH_ID)+ )if workflow.patched(USAGE_COUNTER_SERVICE_PATCH_ID):ctx.usage_counter_plan = await workflow.execute_activity(plan_usage_counters,diff --git a/posthog/test/test_feature_flag_analytics.py b/posthog/test/test_feature_flag_analytics.pyindex c14202fd6d9e..21a7d238ab70 100644--- a/posthog/test/test_feature_flag_analytics.py+++ b/posthog/test/test_feature_flag_analytics.py@@ -102,7 +102,7 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):self.assertEqual(client.hgetall(f"posthog:decide_requests:other"), {})@patch("products.feature_flags.backend.flag_analytics.CACHE_BUCKET_SIZE", 10)- def test_increment_request_count_remote_config_uses_own_bucket(self):+ def test_increment_request_count_uses_one_bucket_per_request_type(self):team_id = 3with time_machine.travel("2022-05-07 12:23:07", tick=False):@@ -110,11 +110,14 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):increment_request_count(team_id)for _ in range(6):increment_request_count(team_id, 1, FlagRequestType.REMOTE_CONFIG)+ for _ in range(2):+ increment_request_count(team_id, 1, FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED)client = redis.get_client()# Remote config fetches are telemetry-only, so they must never leak into the- # decide bucket that billing consumes.+ # decide bucket that billing consumes. The literal keys are the contract with the+ # Rust service, which writes them, so a drift would leave those requests unbilled.self.assertEqual(client.hgetall(f"posthog:decide_requests:{team_id}"),{b"165192618": b"4"},@@ -123,6 +126,10 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):client.hgetall(f"posthog:remote_config_requests:{team_id}"),{b"165192618": b"6"},)+ self.assertEqual(+ client.hgetall(f"posthog:local_evaluation_not_modified_requests:{team_id}"),+ {b"165192618": b"2"},+ )@patch("products.feature_flags.backend.flag_analytics.CACHE_BUCKET_SIZE", 10)def test_capture_team_decide_usage(self):@@ -140,6 +147,7 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):# 10 requests in first bucketincrement_request_count(team_id)increment_request_count(team_id, 1, FlagRequestType.LOCAL_EVALUATION)+ increment_request_count(team_id, 1, FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED)increment_request_count(team_id, 1, FlagRequestType.REMOTE_CONFIG)for _ in range(7):# 7 requests for other team@@ -151,6 +159,7 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):# 5 requests in second bucketincrement_request_count(team_id)increment_request_count(team_id, 1, FlagRequestType.LOCAL_EVALUATION)+ increment_request_count(team_id, 1, FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED)increment_request_count(team_id, 1, FlagRequestType.REMOTE_CONFIG)for _ in range(3):# 3 requests for other team@@ -162,6 +171,7 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):# 5 requests in third bucketincrement_request_count(team_id)increment_request_count(team_id, 1, FlagRequestType.LOCAL_EVALUATION)+ increment_request_count(team_id, 1, FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED)increment_request_count(team_id, 1, FlagRequestType.REMOTE_CONFIG)increment_request_count(other_team_id)@@ -169,7 +179,7 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):# these other requests should not add duplicate countscapture_team_decide_usage(mock_capture, team_id, team_uuid)capture_team_decide_usage(mock_capture, team_id, team_uuid)- assert mock_capture.capture.call_count == 3+ assert mock_capture.capture.call_count == 4mock_capture.capture.assert_any_call(distinct_id=team_id,event="decide usage",@@ -194,6 +204,18 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):"token": "token",},)+ mock_capture.capture.assert_any_call(+ distinct_id=team_id,+ event="local evaluation not modified usage",+ properties={+ "count": 15,+ "team_id": team_id,+ "team_uuid": team_uuid,+ "max_time": 1651926190,+ "min_time": 1651926180,+ "token": "token",+ },+ )mock_capture.capture.assert_any_call(distinct_id=team_id,event="remote config usage",@@ -304,6 +326,7 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):)local_evaluation_team_id = Team.objects.create(organization=self.organization).idremote_config_team_id = Team.objects.create(organization=self.organization).id+ not_modified_team_id = Team.objects.create(organization=self.organization).idwith time_machine.travel("2022-05-07 12:23:07", tick=False) as frozen_datetime:for _ in range(10):@@ -314,6 +337,7 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):increment_request_count(other_team_id)increment_request_count(local_evaluation_team_id, request_type=FlagRequestType.LOCAL_EVALUATION)increment_request_count(remote_config_team_id, request_type=FlagRequestType.REMOTE_CONFIG)+ increment_request_count(not_modified_team_id, request_type=FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED)frozen_datetime.shift(datetime.timedelta(seconds=5))@@ -332,6 +356,7 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):increment_request_count(other_team_id)increment_request_count(local_evaluation_team_id, request_type=FlagRequestType.LOCAL_EVALUATION)increment_request_count(remote_config_team_id, request_type=FlagRequestType.REMOTE_CONFIG)+ increment_request_count(not_modified_team_id, request_type=FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED)with (self.settings(DECIDE_BILLING_ANALYTICS_TOKEN="token"),@@ -376,13 +401,15 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):(other_team_id, "decide usage"),(local_evaluation_team_id, "local evaluation usage"),(remote_config_team_id, "remote config usage"),+ (not_modified_team_id, "local evaluation not modified usage"),}- assert mock_capture.capture.call_count == 4+ assert mock_capture.capture.call_count == 5assert {call.args[1] for call in team_export.call_args_list} == {team_id,other_team_id,local_evaluation_team_id,remote_config_team_id,+ not_modified_team_id,}def test_failed_key_check_exports_every_team_in_the_batch(self) -> None:diff --git a/posthog/usage_counters.py b/posthog/usage_counters.pyindex b7efec49927e..276a68b3980e 100644--- a/posthog/usage_counters.py+++ b/posthog/usage_counters.py@@ -39,6 +39,9 @@ class UsageCounter(StrEnum):CDP_INVOCATIONS = "teams_with_cdp_billable_invocations_in_period"FEATURE_FLAG_REQUESTS = "teams_with_decide_requests_count_in_period"FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS = "teams_with_local_evaluation_requests_count_in_period"+ FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS = (+ "teams_with_local_evaluation_not_modified_requests_count_in_period"+ )WORKFLOW_EMAILS = "teams_with_workflow_emails_sent_in_period"WORKFLOW_PUSH = "teams_with_workflow_push_sent_in_period"WORKFLOW_SMS = "teams_with_workflow_sms_sent_in_period"@@ -65,6 +68,10 @@ LEGACY_QUERY_NAMES = {UsageCounter.LOGS_RETENTION_30D_BYTES: "logs_retention_bytes",UsageCounter.SANDBOX_COMPUTE_CREDITS: "sandbox_compute_usage",}+# Temporal workers decode a plan's modes into UsageCounter, so a worker without a counter rejects any plan that names it.+# During a rolling deploy or a rollback, older workers still run, so a new counter stays out of plans until every worker+# can decode it. Usage reports v2 reads these counters from their legacy query specs meanwhile.+PLAN_DEFERRED_COUNTERS = frozenset({UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS})class UsageCounterMode(StrEnum):@@ -102,6 +109,7 @@ RECORD_USAGE_KEYS = {UsageCounter.CDP_INVOCATIONS: "cdp_billable_invocations",UsageCounter.FEATURE_FLAG_REQUESTS: "feature_flag_requests",UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS: "feature_flag_local_evaluation_requests",+ UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS: "feature_flag_local_evaluation_not_modified_requests",UsageCounter.WORKFLOW_EMAILS: "workflow_emails_sent",UsageCounter.WORKFLOW_PUSH: "workflow_push_sent",UsageCounter.WORKFLOW_SMS: "workflow_sms_sent",@@ -118,6 +126,7 @@ RECORD_USAGE_UNITS = {"cdp_billable_invocations": "invocations","feature_flag_requests": "requests","feature_flag_local_evaluation_requests": "requests",+ "feature_flag_local_evaluation_not_modified_requests": "requests","workflow_emails_sent": "invocations","workflow_push_sent": "invocations","workflow_sms_sent": "invocations",@@ -135,6 +144,7 @@ COUNTER_FLAG_NAMES = {UsageCounter.CDP_INVOCATIONS: "cdp-invocations",UsageCounter.FEATURE_FLAG_REQUESTS: "feature-flag-requests",UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS: "feature-flag-local-evaluation-requests",+ UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS: "feature-flag-local-evaluation-not-modified-requests",UsageCounter.WORKFLOW_EMAILS: "workflow-emails",UsageCounter.WORKFLOW_PUSH: "workflow-push",UsageCounter.WORKFLOW_SMS: "workflow-sms",@@ -282,6 +292,11 @@ class UsageCounterService:begin, end, usage_report.FlagRequestType.LOCAL_EVALUATION)),+ UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS: lambda begin, end: (+ usage_report.get_teams_with_feature_flag_requests_count_in_period(+ begin, end, usage_report.FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED+ )+ ),UsageCounter.WORKFLOW_EMAILS: usage_report.get_teams_with_workflow_emails_sent_in_period,UsageCounter.WORKFLOW_PUSH: usage_report.get_teams_with_workflow_push_sent_in_period,UsageCounter.WORKFLOW_SMS: usage_report.get_teams_with_workflow_sms_sent_in_period,@@ -317,7 +332,11 @@ class UsageCounterService:return UsageCounterPlan(period=period,caller="usage_reports_v2",- modes={counter: mode for counter, mode in modes.items() if counters is None or counter in counters},+ modes={+ counter: mode+ for counter, mode in modes.items()+ if counter not in PLAN_DEFERRED_COUNTERS and (counters is None or counter in counters)+ },complete=complete,)diff --git a/products/feature_flags/backend/flag_analytics.py b/products/feature_flags/backend/flag_analytics.pyindex f5c44072f14f..32ea27835306 100644--- a/products/feature_flags/backend/flag_analytics.py+++ b/products/feature_flags/backend/flag_analytics.py@@ -81,22 +81,38 @@ SDK_LIBRARIES = [# locally. It's not included in CI because of tricky patching freeze time in thread issues.-# Remote config requests are tracked for telemetry only; billing consumes just the decide and+# Remote config requests are tracked for telemetry only; billing consumes the decide and# local evaluation events (see usage_report.py), so the remote config event never bills.+# Local evaluation 304s get their own counter because they bill at the decide rate._REQUEST_BUCKET_PREFIXES = {FlagRequestType.DECIDE: "decide_requests",FlagRequestType.LOCAL_EVALUATION: "local_evaluation_requests",+ FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED: "local_evaluation_not_modified_requests",FlagRequestType.REMOTE_CONFIG: "remote_config_requests",}-EXPORTED_REQUEST_TYPES = (FlagRequestType.DECIDE, FlagRequestType.LOCAL_EVALUATION, FlagRequestType.REMOTE_CONFIG)+EXPORTED_REQUEST_TYPES = (+ FlagRequestType.DECIDE,+ FlagRequestType.LOCAL_EVALUATION,+ FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED,+ FlagRequestType.REMOTE_CONFIG,+)USAGE_EVENT_NAMES = {FlagRequestType.DECIDE: "decide usage",FlagRequestType.LOCAL_EVALUATION: "local evaluation usage",+ FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED: "local evaluation not modified usage",FlagRequestType.REMOTE_CONFIG: "remote config usage",}+# A full local evaluation response costs ten decide requests. A 304 costs one, because it+# sends no definitions.+LOCAL_EVALUATION_BILLING_WEIGHT = 10+++def billable_feature_flag_requests(*, decide: int, local_evaluation: int, local_evaluation_not_modified: int) -> int:+ return decide + local_evaluation * LOCAL_EVALUATION_BILLING_WEIGHT + local_evaluation_not_modified+def _request_bucket_prefix(request_type: FlagRequestType) -> str:try:diff --git a/products/feature_flags/backend/presentation/request_usage.py b/products/feature_flags/backend/presentation/request_usage.pyindex 474e284ee9df..b693407bb541 100644--- a/products/feature_flags/backend/presentation/request_usage.py+++ b/products/feature_flags/backend/presentation/request_usage.py@@ -69,7 +69,7 @@ class FeatureFlagRequestUsageItemSerializer(DataclassSerializer):"sdk": {"help_text": "SDK family parsed from the request user agent."},"request_count": {"help_text": "Number of billable requests in this bucket."},"billing_units": {- "help_text": "Estimated billing units. Local evaluation requests count as 10 units each."+ "help_text": "Estimated billing units. Local evaluation requests count as 10 units each, except 304 Not Modified responses, which count as 1."},}diff --git a/products/feature_flags/backend/request_usage.py b/products/feature_flags/backend/request_usage.pyindex 40ad034b0bf3..a28fcdf4417c 100644--- a/products/feature_flags/backend/request_usage.py+++ b/products/feature_flags/backend/request_usage.py@@ -16,7 +16,7 @@ from posthog.models.event.new_events_schema import events_read_table, use_new_evfrom posthog.models.property.util import get_property_string_exprfrom posthog.utils import get_instance_region-from products.feature_flags.backend.flag_analytics import USAGE_EVENT_NAMES+from products.feature_flags.backend.flag_analytics import USAGE_EVENT_NAMES, billable_feature_flag_requestsclass FeatureFlagRequestType(StrEnum):@@ -40,10 +40,24 @@ REQUEST_USAGE_QUERY_SETTINGS = {"max_threads": 4,}-FeatureFlagRequestUsageRow = tuple[datetime, Literal["remote_evaluation", "local_evaluation"], int, str]--# Keep aligned with usage_report.py, which applies this multiplier to billed local evaluation requests.-LOCAL_EVALUATION_BILLING_WEIGHT = 10+RawRequestType = Literal["remote_evaluation", "local_evaluation", "local_evaluation_not_modified"]+FeatureFlagRequestUsageRow = tuple[datetime, RawRequestType, int, str]++# A 304 is a local evaluation request, so it shows with the others, but it bills like a remote one.+_RAW_REQUEST_TYPES: dict[RawRequestType, tuple[FeatureFlagRequestType, int]] = {+ "remote_evaluation": (+ FeatureFlagRequestType.REMOTE_EVALUATION,+ billable_feature_flag_requests(decide=1, local_evaluation=0, local_evaluation_not_modified=0),+ ),+ "local_evaluation": (+ FeatureFlagRequestType.LOCAL_EVALUATION,+ billable_feature_flag_requests(decide=0, local_evaluation=1, local_evaluation_not_modified=0),+ ),+ "local_evaluation_not_modified": (+ FeatureFlagRequestType.LOCAL_EVALUATION,+ billable_feature_flag_requests(decide=0, local_evaluation=0, local_evaluation_not_modified=1),+ ),+}def parse_sdk_breakdown(raw_sdk_breakdown: str) -> dict[str, int]:@@ -64,12 +78,15 @@ def aggregate_feature_flag_request_usage(rows: Iterable[FeatureFlagRequestUsageRow],) -> list[FeatureFlagRequestUsage]:counts: defaultdict[tuple[datetime, FeatureFlagRequestType, str], int] = defaultdict(int)+ billing_units: defaultdict[tuple[datetime, FeatureFlagRequestType, str], int] = defaultdict(int)for bucket, raw_request_type, total_count, raw_sdk_breakdown in rows:- request_type = FeatureFlagRequestType(raw_request_type)+ request_type, weight = _RAW_REQUEST_TYPES[raw_request_type]sdk_breakdown = parse_sdk_breakdown(raw_sdk_breakdown)- for sdk, count in sdk_breakdown.items():+ sdk_counts = dict(sdk_breakdown)+ sdk_counts["other"] = sdk_counts.get("other", 0) + max(total_count - sum(sdk_breakdown.values()), 0)+ for sdk, count in sdk_counts.items():counts[(bucket, request_type, sdk)] += count- counts[(bucket, request_type, "other")] += max(total_count - sum(sdk_breakdown.values()), 0)+ billing_units[(bucket, request_type, sdk)] += count * weightreturn [FeatureFlagRequestUsage(@@ -77,8 +94,7 @@ def aggregate_feature_flag_request_usage(request_type=request_type,sdk=sdk,request_count=request_count,- billing_units=request_count- * (LOCAL_EVALUATION_BILLING_WEIGHT if request_type == FeatureFlagRequestType.LOCAL_EVALUATION else 1),+ billing_units=billing_units[(bucket, request_type, sdk)],)for (bucket, request_type, sdk), request_count in sorted(counts.items())if request_count > 0@@ -99,6 +115,7 @@ def query_feature_flag_request_usage(bucket_function = "toStartOfHour" if time_interval == "hour" else "toStartOfDay"decide_event = USAGE_EVENT_NAMES[FlagRequestType.DECIDE]local_event = USAGE_EVENT_NAMES[FlagRequestType.LOCAL_EVALUATION]+ not_modified_event = USAGE_EVENT_NAMES[FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED]with tags_context(product=Product.FEATURE_FLAGS, feature=Feature.QUERY, team_id=team_id):# nosemgrep: clickhouse-fstring-param-audit - bucket function and table expressions are internal fragments@@ -106,13 +123,17 @@ def query_feature_flag_request_usage(f"""SELECT{bucket_function}(timestamp) AS bucket,- if(event = %(decide_event)s, 'remote_evaluation', 'local_evaluation') AS request_type,+ multiIf(+ event = %(decide_event)s, 'remote_evaluation',+ event = %(not_modified_event)s, 'local_evaluation_not_modified',+ 'local_evaluation'+ ) AS request_type,toInt64OrZero({count_expr}) AS total_count,{sdk_breakdown_expr} AS sdk_breakdownFROM {events_read_table(use_new)}WHERE team_id = %(internal_team_id)sAND distinct_id = toString(%(team_id)s)- AND event IN (%(decide_event)s, %(local_event)s)+ AND event IN (%(decide_event)s, %(local_event)s, %(not_modified_event)s)AND timestamp >= %(date_from)s AND timestamp < %(date_to)sAND has([%(validity_token)s], {token_expr})ORDER BY bucket, request_type@@ -125,6 +146,7 @@ def query_feature_flag_request_usage("validity_token": validity_token,"decide_event": decide_event,"local_event": local_event,+ "not_modified_event": not_modified_event,},workload=Workload.ONLINE,team_id=team_id,diff --git a/products/feature_flags/backend/test/test_request_usage.py b/products/feature_flags/backend/test/test_request_usage.pyindex b3252022883a..409ac4fa108f 100644--- a/products/feature_flags/backend/test/test_request_usage.py+++ b/products/feature_flags/backend/test/test_request_usage.py@@ -24,6 +24,7 @@ class TestFeatureFlagRequestUsageQuery(ClickhouseTestMixin, TestCase):[("missing breakdown", "", [("other", 10)]),("partial breakdown", '{"posthog-node": 4}', [("other", 6), ("posthog-node", 4)]),+ ("breakdown with other", '{"other": 4, "posthog-node": 6}', [("other", 4), ("posthog-node", 6)]),])def test_attributes_unclassified_billed_requests_to_other(@@ -90,6 +91,17 @@ class TestFeatureFlagRequestUsageQuery(ClickhouseTestMixin, TestCase):"token": "request-usage-test-token",},)+ _create_event(+ team=analytics_team,+ distinct_id="123",+ event="local evaluation not modified usage",+ timestamp=datetime(2026, 8, 20, 12, 40, tzinfo=UTC),+ properties={+ "count": 2,+ "sdk_breakdown": {"posthog-python": 2},+ "token": "request-usage-test-token",+ },+ )_create_event(team=analytics_team,distinct_id="123",@@ -123,10 +135,12 @@ class TestFeatureFlagRequestUsageQuery(ClickhouseTestMixin, TestCase):time_interval="day",)- assert [(item.request_type.value, item.sdk, item.request_count) for item in hourly_results] == [- ("local_evaluation", "posthog-python", 3),- ("remote_evaluation", "other", 5),- ("remote_evaluation", "posthog-node", 7),+ assert [+ (item.request_type.value, item.sdk, item.request_count, item.billing_units) for item in hourly_results+ ] == [+ ("local_evaluation", "posthog-python", 5, 32),+ ("remote_evaluation", "other", 5, 5),+ ("remote_evaluation", "posthog-node", 7, 7),]assert {item.bucket for item in hourly_results} == {datetime(2026, 8, 20, 12, tzinfo=UTC)}assert {item.bucket for item in daily_results} == {datetime(2026, 8, 20, tzinfo=UTC)}diff --git a/products/feature_flags/frontend/generated/api.schemas.ts b/products/feature_flags/frontend/generated/api.schemas.tsindex 9b312a28581f..f605a9301a92 100644--- a/products/feature_flags/frontend/generated/api.schemas.ts+++ b/products/feature_flags/frontend/generated/api.schemas.ts@@ -493,7 +493,7 @@ export interface FeatureFlagRequestUsageItemApi {sdk: string/** Number of billable requests in this bucket. */request_count: number- /** Estimated billing units. Local evaluation requests count as 10 units each. */+ /** Estimated billing units. Local evaluation requests count as 10 units each, except 304 Not Modified responses, which count as 1. */billing_units: number}diff --git a/products/feature_flags/frontend/requestUsage/FeatureFlagRequestUsage.tsx b/products/feature_flags/frontend/requestUsage/FeatureFlagRequestUsage.tsxindex 4111dd46783e..6510b117aeee 100644--- a/products/feature_flags/frontend/requestUsage/FeatureFlagRequestUsage.tsx+++ b/products/feature_flags/frontend/requestUsage/FeatureFlagRequestUsage.tsx@@ -90,11 +90,11 @@ export function FeatureFlagRequestUsage(): JSX.Element {title: 'Billing units',dataIndex: 'billingUnits',align: 'right',- render: (value, { remoteRequests, localRequests }) => (+ render: (value, { remoteRequests, localBillingUnits }) => (<span>{(value ?? 0).toLocaleString()}{' '}<span className="text-secondary whitespace-nowrap">- ({remoteRequests.toLocaleString()} + 10 × {localRequests.toLocaleString()})+ ({remoteRequests.toLocaleString()} remote + {localBillingUnits.toLocaleString()} local)</span></span>),@@ -168,9 +168,9 @@ export function FeatureFlagRequestUsage(): JSX.Element {<LemonBanner type="info">This view shows billable requests, not every HTTP request. Remote requests include <code>/flags</code>{' '}- and legacy <code>/decide</code> traffic. Local evaluation requests use 10 billing units each. Data can- take about 30 minutes to appear, and hourly buckets reflect billing aggregation time rather than exact- request time.{' '}+ and legacy <code>/decide</code> traffic. Local evaluation requests use 10 billing units each, except+ responses with no changes (304 Not Modified), which use 1. Data can take about 30 minutes to appear, and+ hourly buckets reflect billing aggregation time rather than exact request time.{' '}<Link to="https://posthog.com/docs/feature-flags/common-questions#billing--usage">Learn how usage is calculated</Link>diff --git a/products/feature_flags/frontend/requestUsage/featureFlagRequestUsageLogic.test.ts b/products/feature_flags/frontend/requestUsage/featureFlagRequestUsageLogic.test.tsindex 4a36665b86f4..4d54fc583f64 100644--- a/products/feature_flags/frontend/requestUsage/featureFlagRequestUsageLogic.test.ts+++ b/products/feature_flags/frontend/requestUsage/featureFlagRequestUsageLogic.test.ts@@ -39,20 +39,22 @@ describe('featureFlagRequestUsageLogic', () => {bucket: '2026-08-20T00:00:00Z',request_type: 'local_evaluation',sdk: 'posthog-ruby',- request_count: 3,- billing_units: 30,+ // Three full responses at 10 units and two 304s at 1 unit.+ request_count: 5,+ billing_units: 32,},],})await expectLogic(logic).toMatchValues({- totalBillingUnits: 50,+ totalBillingUnits: 52,sdkTotals: [{sdk: 'posthog-ruby',remoteRequests: 20,- localRequests: 3,- billingUnits: 50,+ localRequests: 5,+ localBillingUnits: 32,+ billingUnits: 52,billingUnitsShare: 100,},],@@ -67,7 +69,7 @@ describe('featureFlagRequestUsageLogic', () => {id: expect.any(Number),label: 'posthog-ruby (local)',dates: ['2026-08-20T00:00:00.000Z'],- data: [3],+ data: [5],},],})diff --git a/products/feature_flags/frontend/requestUsage/featureFlagRequestUsageLogic.ts b/products/feature_flags/frontend/requestUsage/featureFlagRequestUsageLogic.tsindex 03bc5466b6d0..caafa32d9bde 100644--- a/products/feature_flags/frontend/requestUsage/featureFlagRequestUsageLogic.ts+++ b/products/feature_flags/frontend/requestUsage/featureFlagRequestUsageLogic.ts@@ -28,6 +28,7 @@ export interface FeatureFlagRequestUsageSdkTotal {sdk: stringremoteRequests: numberlocalRequests: number+ localBillingUnits: numberbillingUnits: numberbillingUnitsShare: number}@@ -299,6 +300,7 @@ export const featureFlagRequestUsageLogic = kea<featureFlagRequestUsageLogicTypesdk: item.sdk,remoteRequests: 0,localRequests: 0,+ localBillingUnits: 0,billingUnits: 0,billingUnitsShare: 0,}@@ -306,6 +308,7 @@ export const featureFlagRequestUsageLogic = kea<featureFlagRequestUsageLogicTypecurrent.remoteRequests += item.request_count} else {current.localRequests += item.request_count+ current.localBillingUnits += item.billing_units}current.billingUnits += item.billing_unitstotals.set(item.sdk, current)diff --git a/rust/feature-flags/src/api/flag_definitions.rs b/rust/feature-flags/src/api/flag_definitions.rsindex 5ca325927428..0de719ec0506 100644--- a/rust/feature-flags/src/api/flag_definitions.rs+++ b/rust/feature-flags/src/api/flag_definitions.rs@@ -10,7 +10,7 @@ use crate::{metrics::consts::{FLAG_DEFINITIONS_AUTH_COUNTER, FLAG_DEFINITIONS_CACHE_HIT_COUNTER,FLAG_DEFINITIONS_CACHE_MISS_COUNTER, FLAG_DEFINITIONS_ETAG_COUNTER,- FLAG_DEFINITIONS_REBUILD_REQUESTED_COUNTER,+ FLAG_DEFINITIONS_NOT_MODIFIED_BILLING_COUNTER, FLAG_DEFINITIONS_REBUILD_REQUESTED_COUNTER,},router::State as AppState,team::team_models::Team,@@ -31,7 +31,7 @@ use serde_json::Value;use sqlx::PgPool;use std::collections::HashSet;use std::sync::Arc;-use std::time::{SystemTime, UNIX_EPOCH};+use std::time::{Duration, SystemTime, UNIX_EPOCH};use tracing::{info, warn};const ALLOWLIST_TTL_SECS: u64 = 60;@@ -125,6 +125,93 @@ pub async fn fetch_allowlist_from_db(pool: &PgPool) -> Result<Option<HashSet<Tea/// This is returned as raw JSON from cache to avoid deserialization overheadpub type FlagDefinitionsResponse = Value;+/// Bound on the payload read a 304 makes when the memo has no entry for the+/// current ETag. Matches the hypercache Redis timeout. A slow read delays the+/// 304 by at most this long, and then the poll goes unbilled and the next one+/// retries.+const BILLABLE_LOOKUP_TIMEOUT: Duration = Duration::from_millis(500);++/// How long a failed, slow, or non-Redis payload read is remembered. While+/// the payload tier is degraded this caps the retries at one per pod per+/// team per interval instead of one per poll, and every poll in between+/// stays unbilled.+const BILLABLE_MEMO_FAILURE_TTL: Duration = Duration::from_secs(30);++type BillableMemoKey = (i32, String);++/// A memoized answer outlives a mismatch between the ETag and the payload it+/// was computed from for at most `answer_ttl`: the writer can publish a new+/// version between the ETag read and the payload read of one request. A+/// failure to answer is kept only briefly so the next poll retries soon.+struct BillableMemoExpiry {+ answer_ttl: Duration,+ failure_ttl: Duration,+}++impl moka::Expiry<BillableMemoKey, Option<bool>> for BillableMemoExpiry {+ fn expire_after_create(+ &self,+ _key: &BillableMemoKey,+ value: &Option<bool>,+ _created_at: std::time::Instant,+ ) -> Option<Duration> {+ Some(match value {+ Some(_) => self.answer_ttl,+ None => self.failure_ttl,+ })+ }+}++/// Memo of whether the definitions behind a `(team, etag)` contain a billable+/// flag, so a 304 for definitions that hold only survey or product tour flags+/// is not billed, without reading the payload the 304 deliberately skips.+#[derive(Clone)]+pub struct DefinitionsBillableCache {+ entries: moka::future::Cache<BillableMemoKey, Option<bool>>,+}++impl DefinitionsBillableCache {+ pub fn new(max_capacity: u64, answer_ttl: Duration) -> Self {+ Self::with_failure_ttl(max_capacity, answer_ttl, BILLABLE_MEMO_FAILURE_TTL)+ }++ fn with_failure_ttl(max_capacity: u64, answer_ttl: Duration, failure_ttl: Duration) -> Self {+ Self {+ entries: moka::future::Cache::builder()+ .max_capacity(max_capacity)+ .expire_after(BillableMemoExpiry {+ answer_ttl,+ failure_ttl,+ })+ .build(),+ }+ }++ pub async fn insert(&self, team_id: i32, etag: &str, billable: bool) {+ self.entries+ .insert((team_id, etag.to_string()), Some(billable))+ .await;+ }++ /// Returns the memoized status, or runs `load` once for all concurrent+ /// callers with the same key and stores the result. `None` when the load+ /// returns `None`, fails, or exceeds the bound; that `None` is kept for+ /// `BILLABLE_MEMO_FAILURE_TTL` before a later call retries.+ pub async fn get_or_load<F>(&self, team_id: i32, etag: &str, load: F) -> Option<bool>+ where+ F: std::future::Future<Output = Option<bool>>,+ {+ self.entries+ .get_with((team_id, etag.to_string()), async {+ tokio::time::timeout(BILLABLE_LOOKUP_TIMEOUT, load)+ .await+ .ok()+ .flatten()+ })+ .await+ }+}+/// Query parameters for the flag definitions endpoint#[derive(Debug, Deserialize, Serialize)]pub struct FlagDefinitionsQueryParams {@@ -257,6 +344,15 @@ pub async fn flags_definitions(&[("result".to_string(), "hit".to_string())],1,);+ if !*state.config.skip_writes+ && is_billable_for_etag(&state, &team_key, team.id, current_val).await+ {+ state.billing_aggregator.record(+ team.id,+ FlagRequestType::FlagDefinitionsNotModified,+ Some(Library::from_headers(&headers)),+ );+ }return Ok(not_modified_response(current_val));}}@@ -281,12 +377,20 @@ pub async fn flags_definitions();// Retrieve cached response from HyperCache (always with cohorts)- let cached_response = get_from_cache(&state, &team_key, team.id).await?;+ let (cached_response, cache_source) = get_from_cache(&state, &team_key, team.id).await?;// Record usage for billing, filtering out non-billable flags (surveys, product tours).- // Placed after the ETag/304 path intentionally: 304 responses skip billing,- // matching Django's /local_evaluation behavior.- if !*state.config.skip_writes && has_billable_flags(&cached_response) {+ let billable = has_billable_flags(&cached_response);+ if let (Some(etag), Some(billable)) = (+ ¤t_etag,+ billable_for_etag_from(&cached_response, cache_source),+ ) {+ state+ .definitions_billable_cache+ .insert(team.id, etag, billable)+ .await;+ }+ if !*state.config.skip_writes && billable {let library = Library::from_headers(&headers);state.billing_aggregator@@ -432,6 +536,49 @@ fn cache_hit_metric_source_label(source: &CacheSource) -> &'static str {}}+/// Only Redis holds the payload the ETag was published with. An S3 fallback+/// may be another version, so it decides nothing.+fn billable_for_etag_from(response: &FlagDefinitionsResponse, source: CacheSource) -> Option<bool> {+ matches!(source, CacheSource::Redis).then(|| has_billable_flags(response))+}++/// Reads the payload at most once per pod per ETag per memo TTL. An+/// unreadable, slow, or non-Redis payload is not memoized as an answer and+/// counts as not billable, because over-billing is the worse failure.+async fn is_billable_for_etag(+ state: &AppState,+ team_key: &KeyType,+ team_id: i32,+ etag: &str,+) -> bool {+ let billable = state+ .definitions_billable_cache+ .get_or_load(team_id, etag, async {+ // Reads the hypercache directly: this lookup serves a 304, so it must not count as+ // a definitions cache hit or queue a rebuild the way a full response does. It reads+ // Redis only, because an S3 copy cannot decide (see `billable_for_etag_from`), so a+ // fallback read would only add latency and S3 load. A Redis miss is not an answer.+ let response = state+ .flags_with_cohorts_hypercache_reader+ .get_typed_from_redis::<FlagDefinitionsResponse>(team_key)+ .await+ .ok()?;+ Some(response.is_some_and(|response| has_billable_flags(&response)))+ })+ .await;+ let outcome = match billable {+ Some(true) => "billable",+ Some(false) => "not_billable",+ None => "unknown",+ };+ inc(+ FLAG_DEFINITIONS_NOT_MODIFIED_BILLING_COUNTER,+ &[("outcome".to_string(), outcome.to_string())],+ 1,+ );+ billable.unwrap_or(false)+}+/// Retrieves the cached response using the pre-initialized HyperCacheReader////// Always uses the cache with cohorts included to match Django's behavior and ensure@@ -444,7 +591,7 @@ async fn get_from_cache(state: &AppState,team_key: &KeyType,team_id: i32,-) -> Result<FlagDefinitionsResponse, FlagError> {+) -> Result<(FlagDefinitionsResponse, CacheSource), FlagError> {let result = state.flags_with_cohorts_hypercache_reader.get_with_source(team_key)@@ -472,7 +619,7 @@ async fn get_from_cache() {enqueue_flag_definitions_rebuild(state, team_id, REBUILD_TRIGGER_S3_HIT);}- Ok(data)+ Ok((data, source))}Err(e) => {let reason = match &e {@@ -720,6 +867,67 @@ mod tests {}}+ #[tokio::test]+ async fn test_billable_cache_memoizes_a_successful_load() {+ let cache = DefinitionsBillableCache::new(10, Duration::from_secs(3600));+ assert_eq!(+ cache.get_or_load(1, "etag", async { Some(true) }).await,+ Some(true)+ );+ // A later load is never run once the key is memoized.+ assert_eq!(+ cache.get_or_load(1, "etag", async { None }).await,+ Some(true)+ );+ // A new etag for the same team is a fresh key.+ assert_eq!(+ cache.get_or_load(1, "etag2", async { Some(false) }).await,+ Some(false)+ );+ }++ #[tokio::test]+ async fn test_billable_cache_memoizes_a_failed_load_until_the_failure_ttl() {+ let cache = DefinitionsBillableCache::with_failure_ttl(+ 10,+ Duration::from_secs(3600),+ Duration::from_millis(500),+ );+ assert_eq!(cache.get_or_load(1, "etag", async { None }).await, None);+ assert_eq!(+ cache.get_or_load(1, "etag", async { Some(true) }).await,+ None+ );+ // moka expires entries on its own clock, so a paused tokio clock would not age this one.+ tokio::time::sleep(Duration::from_secs(1)).await;+ assert_eq!(+ cache.get_or_load(1, "etag", async { Some(true) }).await,+ Some(true)+ );+ }++ #[tokio::test(start_paused = true)]+ async fn test_billable_cache_bounds_a_slow_load() {+ let cache = DefinitionsBillableCache::new(10, Duration::from_secs(3600));+ assert_eq!(+ cache.get_or_load(1, "etag", std::future::pending()).await,+ None+ );+ }++ #[rstest]+ #[case::redis(CacheSource::Redis, Some(true))]+ #[case::s3(CacheSource::S3, None)]+ #[case::s3_after_redis_error(CacheSource::S3AfterRedisError, None)]+ #[case::fallback(CacheSource::Fallback, None)]+ fn test_billable_for_etag_from_only_trusts_redis(+ #[case] source: CacheSource,+ #[case] expected: Option<bool>,+ ) {+ let response = json!({"flags": [{"key": "billable-flag"}]});+ assert_eq!(billable_for_etag_from(&response, source), expected);+ }+#[test]fn test_extract_etag_from_header_weak() {let val = axum::http::HeaderValue::from_static("W/\"a1b2c3d4e5f6g7h8\"");diff --git a/rust/feature-flags/src/billing/aggregator.rs b/rust/feature-flags/src/billing/aggregator.rsindex 71b2f004a5e6..85a273d82faa 100644--- a/rust/feature-flags/src/billing/aggregator.rs+++ b/rust/feature-flags/src/billing/aggregator.rs@@ -78,6 +78,7 @@ use std::time::Duration;use common_metrics::{gauge, histogram, inc};use common_redis::{Client as RedisClient, CustomRedisError, PipelineCommand};use rand::Rng;+use strum::IntoEnumIterator;use tokio::sync::Notify;use tokio::task::JoinHandle;@@ -328,6 +329,7 @@ struct Inner {/// instead of an `inc()` call that allocates label strings every time.record_count_decide: AtomicU64,record_count_flag_definitions: AtomicU64,+ record_count_flag_definitions_not_modified: AtomicU64,shutdown_signal: Notify,usage_reporter: Option<Arc<UsageReporter>>,}@@ -348,10 +350,21 @@ impl Inner {last_successful_flush_epoch_ms: AtomicU64::new(0),record_count_decide: AtomicU64::new(0),record_count_flag_definitions: AtomicU64::new(0),+ record_count_flag_definitions_not_modified: AtomicU64::new(0),shutdown_signal: Notify::new(),})}+ fn record_counter(&self, request_type: FlagRequestType) -> &AtomicU64 {+ match request_type {+ FlagRequestType::Decide => &self.record_count_decide,+ FlagRequestType::FlagDefinitions => &self.record_count_flag_definitions,+ FlagRequestType::FlagDefinitionsNotModified => {+ &self.record_count_flag_definitions_not_modified+ }+ }+ }+/// Sum any remaining entries in `pending` plus any drained-but-uncredited/// counts from an interrupted flush, and emit them as shutdown drops.fn record_shutdown_drops(&self) {@@ -434,11 +447,9 @@ impl BillingAggregator {// Bump the per-request-type atomic counter; the flusher emits the// `FLAGS_BILLING_RECORDS` metric in batches at flush time so the// hot path doesn't pay the per-call label clone in `inc()`.- match request_type {- FlagRequestType::Decide => &self.inner.record_count_decide,- FlagRequestType::FlagDefinitions => &self.inner.record_count_flag_definitions,- }- .fetch_add(1, Ordering::Relaxed);+ self.inner+ .record_counter(request_type)+ .fetch_add(1, Ordering::Relaxed);let key = AggregationKey {team_id,@@ -764,23 +775,17 @@ async fn flush_once(inner: &Arc<Inner>, policy: FlushPolicy) {// FLAGS_BILLING_RECORDS. Doing this here instead of in `record()`// keeps the hot path free of the per-call `apply_label_filter`// allocation that `inc()` triggers.- let decide_records = inner.record_count_decide.swap(0, Ordering::Relaxed);- if decide_records > 0 {- inc(- FLAGS_BILLING_RECORDS,- &record_labels_for(FlagRequestType::Decide),- decide_records,- );- }- let flag_def_records = inner- .record_count_flag_definitions- .swap(0, Ordering::Relaxed);- if flag_def_records > 0 {- inc(- FLAGS_BILLING_RECORDS,- &record_labels_for(FlagRequestType::FlagDefinitions),- flag_def_records,- );+ for request_type in FlagRequestType::iter() {+ let records = inner+ .record_counter(request_type)+ .swap(0, Ordering::Relaxed);+ if records > 0 {+ inc(+ FLAGS_BILLING_RECORDS,+ &record_labels_for(request_type),+ records,+ );+ }}// Swap pending and zero `pending_total` under the same lock so a@@ -1495,6 +1500,7 @@ mod tests {Some(Library::PosthogJs),);agg.record(7, FlagRequestType::Decide, None);+ agg.record(9, FlagRequestType::FlagDefinitionsNotModified, None);let bucket = current_bucket();flush_once(&agg.inner, FlushPolicy::BailOnError).await;@@ -1504,10 +1510,13 @@ mod tests {let expected_sdk_42_def =format!("posthog:local_evaluation_requests:sdk:42:posthog-js:{bucket}");let expected_team_7_decide = format!("posthog:decide_requests:7:{bucket}");- assert_eq!(calls.len(), 3);+ let expected_team_9_not_modified =+ format!("posthog:local_evaluation_not_modified_requests:9:{bucket}");+ assert_eq!(calls.len(), 4);assert!(calls.contains(&(expected_team_42_def, 1)));assert!(calls.contains(&(expected_sdk_42_def, 1)));assert!(calls.contains(&(expected_team_7_decide, 1)));+ assert!(calls.contains(&(expected_team_9_not_modified, 1)));// Belt-and-braces: the aggregator must never write `:shadow`-suffixed// keys. A regression here would mean we're writing into thediff --git a/rust/feature-flags/src/billing/usage_reporter.rs b/rust/feature-flags/src/billing/usage_reporter.rsindex cdacba264453..4f9db6c02ec3 100644--- a/rust/feature-flags/src/billing/usage_reporter.rs+++ b/rust/feature-flags/src/billing/usage_reporter.rs@@ -280,13 +280,18 @@ fn dedup(teams: impl Iterator<Item = i64>) -> Vec<i64> {teams}-/// The two billable request types are priced apart — the nightly report weighs one local-/// evaluation as ten decide requests — so they cannot share a key. The weighting itself stays-/// out of here: a producer reports what happened, and pricing is applied downstream.+/// Each billable request type has its own key because the nightly report prices them apart:+/// one local evaluation costs ten decide requests, and one 304 costs one. A 304 does not use+/// the decide key even at the same price, because each key feeds its own usage counter and a+/// shared key would count the 304 twice. The weighting itself stays out of here: a producer+/// reports what happened, and pricing is applied downstream.fn usage_key(request_type: FlagRequestType) -> &'static str {match request_type {FlagRequestType::Decide => "feature_flag_requests",FlagRequestType::FlagDefinitions => "feature_flag_local_evaluation_requests",+ FlagRequestType::FlagDefinitionsNotModified => {+ "feature_flag_local_evaluation_not_modified_requests"+ }}}@@ -339,6 +344,13 @@ mod tests {},4,),+ (+ AggregationKey {+ request_type: FlagRequestType::FlagDefinitionsNotModified,+ ..key(1, None)+ },+ 5,+ ),];let records = build_records(&entries, &TeamIdCollection::All, 1);@@ -349,7 +361,8 @@ mod tests {.collect::<Vec<_>>(),vec![("feature_flag_requests", 3),- ("feature_flag_local_evaluation_requests", 4)+ ("feature_flag_local_evaluation_requests", 4),+ ("feature_flag_local_evaluation_not_modified_requests", 5)]);}diff --git a/rust/feature-flags/src/config.rs b/rust/feature-flags/src/config.rsindex 5dd3d58c3ff8..7375ce10b7d0 100644--- a/rust/feature-flags/src/config.rs+++ b/rust/feature-flags/src/config.rs@@ -984,6 +984,15 @@ pub struct Config {#[envconfig(from = "TEAM_NEGATIVE_CACHE_TTL_SECONDS", default = "30")]pub team_negative_cache_ttl_seconds: u64,+ // In-memory memo of whether a team's flag definitions, keyed by ETag, hold a+ // billable flag, so a /flags/definitions 304 is billed without a payload read.+ // The TTL bounds how long a wrong answer can last, so it is tunable without a deploy.+ #[envconfig(from = "DEFINITIONS_BILLABLE_CACHE_CAPACITY", default = "100000")]+ pub definitions_billable_cache_capacity: u64,++ #[envconfig(from = "DEFINITIONS_BILLABLE_CACHE_TTL_SECONDS", default = "3600")]+ pub definitions_billable_cache_ttl_seconds: u64,+// Write an S3 hit back into Redis so the next reader for that key is served by Redis// instead of paying another S3 read. Applies to the team metadata and remote config// hypercaches, which have no in-process cache in front of them. 0 disables.@@ -1407,6 +1416,8 @@ impl Config {thread_pool_cores: 0,team_negative_cache_capacity: 10_000,team_negative_cache_ttl_seconds: 30,+ definitions_billable_cache_capacity: 100_000,+ definitions_billable_cache_ttl_seconds: 3_600,hypercache_read_repair_ttl_seconds: 600,skip_pg_team_fallback: FlexBool(false),accept_team_secret_tokens: FlexBool(true),diff --git a/rust/feature-flags/src/flags/flag_analytics.rs b/rust/feature-flags/src/flags/flag_analytics.rsindex fc8ccb31063d..128b8fc10ec3 100644--- a/rust/feature-flags/src/flags/flag_analytics.rs+++ b/rust/feature-flags/src/flags/flag_analytics.rs@@ -24,6 +24,9 @@ pub fn get_team_request_key(team_id: i32, request_type: FlagRequestType) -> Strimatch request_type {FlagRequestType::Decide => format!("posthog:decide_requests:{team_id}"),FlagRequestType::FlagDefinitions => format!("posthog:local_evaluation_requests:{team_id}"),+ FlagRequestType::FlagDefinitionsNotModified => {+ format!("posthog:local_evaluation_not_modified_requests:{team_id}")+ }}}@@ -37,6 +40,9 @@ pub fn get_team_request_library_key(FlagRequestType::FlagDefinitions => {format!("posthog:local_evaluation_requests:sdk:{team_id}:{library}")}+ FlagRequestType::FlagDefinitionsNotModified => {+ format!("posthog:local_evaluation_not_modified_requests:sdk:{team_id}:{library}")+ }}}@@ -54,6 +60,10 @@ mod tests {get_team_request_key(456, FlagRequestType::FlagDefinitions),"posthog:local_evaluation_requests:456");+ assert_eq!(+ get_team_request_key(456, FlagRequestType::FlagDefinitionsNotModified),+ "posthog:local_evaluation_not_modified_requests:456"+ );}#[tokio::test]@@ -83,6 +93,14 @@ mod tests {),"posthog:local_evaluation_requests:sdk:101:posthog-elixir");+ assert_eq!(+ get_team_request_library_key(+ 101,+ FlagRequestType::FlagDefinitionsNotModified,+ Library::PosthogElixir+ ),+ "posthog:local_evaluation_not_modified_requests:sdk:101:posthog-elixir"+ );// Test Other variantassert_eq!(get_team_request_library_key(102, FlagRequestType::Decide, Library::Other),diff --git a/rust/feature-flags/src/flags/flag_request.rs b/rust/feature-flags/src/flags/flag_request.rsindex ddc8aa74c79b..6b1eac036b38 100644--- a/rust/feature-flags/src/flags/flag_request.rs+++ b/rust/feature-flags/src/flags/flag_request.rs@@ -35,10 +35,12 @@ where}))}-#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]+#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, strum::EnumIter)]pub enum FlagRequestType {Decide,FlagDefinitions,+ /// A `/flags/definitions` poll answered with 304, billed at the `/flags` rate.+ FlagDefinitionsNotModified,}impl FlagRequestType {@@ -46,6 +48,7 @@ impl FlagRequestType {match self {FlagRequestType::Decide => "decide",FlagRequestType::FlagDefinitions => "flag_definitions",+ FlagRequestType::FlagDefinitionsNotModified => "flag_definitions_not_modified",}}}diff --git a/rust/feature-flags/src/metrics/consts.rs b/rust/feature-flags/src/metrics/consts.rsindex 46ac7d9d080b..f5c101c9da54 100644--- a/rust/feature-flags/src/metrics/consts.rs+++ b/rust/feature-flags/src/metrics/consts.rs@@ -383,6 +383,12 @@ pub const FLAG_DEFINITIONS_CACHE_MISS_COUNTER: &str = "flags_flag_definitions_ca// redis_error = the etag read failed or the stored value did not decode)pub const FLAG_DEFINITIONS_ETAG_COUNTER: &str = "flags_flag_definitions_etag_total";+// Billing decision for a /flags/definitions 304. Labels: outcome (billable, not_billable,+// unknown = the payload was unreadable, slow, or not served from Redis, so the poll went+// unbilled). Compare with the etag `hit` counter to see how many 304s went unbilled.+pub const FLAG_DEFINITIONS_NOT_MODIFIED_BILLING_COUNTER: &str =+ "flags_flag_definitions_not_modified_billing_total";+// Per-pod resolved cluster for the /flags/definitions reader: 1 on the dedicated flags Redis,// 0 on the shared one. Every emission carries a `reason` label, so// `{reason="no_dedicated_client"}` separates a pod that wanted the dedicated cluster and coulddiff --git a/rust/feature-flags/src/router.rs b/rust/feature-flags/src/router.rsindex 72ef0e5c8d5b..4be2d6e76d0c 100644--- a/rust/feature-flags/src/router.rs+++ b/rust/feature-flags/src/router.rs@@ -42,6 +42,7 @@ use crate::{body_read_metrics::{record_body_read, MAX_FLAGS_BODY_BYTES},concurrency_metrics::{record_concurrency_enter, record_concurrency_wait},endpoint, flag_definitions,+ flag_definitions::DefinitionsBillableCache,flag_definitions_rate_limiter::{FlagDefinitionsRateLimiter, RemoteConfigRateLimiter},flags_rate_limiter::{FlagsRateLimiter, IpRateLimiter},remote_config,@@ -115,6 +116,10 @@ pub struct State {/// Pre-initialized HyperCacheReader for feature flags with cohorts (flags_with_cohorts.json)/// Used by the /flags/definitions endpointpub flags_with_cohorts_hypercache_reader: Arc<HyperCacheReader>,+ /// Billable status of each team's current flag definitions, keyed by ETag, so a 304 on+ /// /flags/definitions for survey-only or product-tour-only definitions goes unbilled+ /// without a payload read on every poll+ pub definitions_billable_cache: DefinitionsBillableCache,/// Pre-initialized HyperCacheReader for team metadata (full_metadata.json)/// Uses token-based lookup instead of team_idpub team_hypercache_reader: Arc<HyperCacheReader>,@@ -434,6 +439,10 @@ whereconfig_hypercache_reader,rayon_dispatcher,team_negative_cache,+ definitions_billable_cache: DefinitionsBillableCache::new(+ config.definitions_billable_cache_capacity,+ std::time::Duration::from_secs(config.definitions_billable_cache_ttl_seconds),+ ),cohort_membership_provider,auth_token_cache,billing_aggregator,diff --git a/rust/feature-flags/src/utils/test_utils.rs b/rust/feature-flags/src/utils/test_utils.rsindex 7c486bf4a33c..0beb39cbf298 100644--- a/rust/feature-flags/src/utils/test_utils.rs+++ b/rust/feature-flags/src/utils/test_utils.rs@@ -2078,6 +2078,19 @@ impl TestContext {.await}+ /// Populate cache for a team with the given flags payload and an ETag.+ pub async fn populate_cache_for_team_with_flags_and_etag(+ &self,+ team_id: i32,+ flags_data: serde_json::Value,+ etag: &str,+ ) -> Result<(), Error> {+ self.populate_cache_for_team_with_flags(team_id, flags_data)+ .await?;+ let redis_client = setup_redis_client(Some(self.config.redis_url.clone())).await;+ self.set_etag_for_team(redis_client, team_id, etag).await+ }+/// Populate cache for a team and store an ETag alongside it, on the given Redis./// The ETag is stored at `{cache_key}:etag` using pickle serialization,/// matching Django's HyperCache behavior.@@ -2089,7 +2102,15 @@ impl TestContext {) -> Result<(), Error> {self.populate_flag_definitions_cache(redis_client.clone(), team_id).await?;+ self.set_etag_for_team(redis_client, team_id, etag).await+ }+ async fn set_etag_for_team(+ &self,+ redis_client: Arc<dyn RedisClientTrait + Send + Sync>,+ team_id: i32,+ etag: &str,+ ) -> Result<(), Error> {let etag_key =format!("posthog:1:cache/teams/{team_id}/feature_flags/flags_with_cohorts.json:etag");let pickled_etag =diff --git a/rust/feature-flags/tests/test_flag_definitions.rs b/rust/feature-flags/tests/test_flag_definitions.rsindex d610e965a9f7..18e2b635ddc0 100644--- a/rust/feature-flags/tests/test_flag_definitions.rs+++ b/rust/feature-flags/tests/test_flag_definitions.rs@@ -3199,6 +3199,276 @@ async fn test_flag_definitions_billing_counter(#[case] skip_writes: bool) {}}+/// A 304 lands on the not-modified counter and leaves the full local evaluation+/// counter untouched. It stays unbilled for definitions that hold only survey or+/// product tour flags, and when `skip_writes` is on.+#[rstest::rstest]+#[case::billable_flag("billable-flag", false, true)]+#[case::survey_only("survey-targeting-only", false, false)]+#[case::skip_writes("billable-flag", true, false)]+#[tokio::test]+async fn test_flag_definitions_304_records_not_modified_billing_counter(+ #[case] flag_key: &str,+ #[case] skip_writes: bool,+ #[case] expect_billed: bool,+) {+ use feature_flags::config::FlexBool;+ use feature_flags::flags::flag_analytics::{current_bucket, get_team_request_key};+ use feature_flags::flags::flag_request::FlagRequestType;+ use feature_flags::utils::test_utils::{setup_redis_client, TestContext};+ use serde_json::json;++ let mut config = feature_flags::config::Config::default_test_config();+ config.skip_writes = FlexBool(skip_writes);+ let context = TestContext::new(Some(&config)).await;+ let (team, secret_token, _) = context+ .create_team_with_secret_token(None, None, None)+ .await+ .unwrap();++ let etag_value = "billed304etag0001";+ context+ .populate_cache_for_team_with_flags_and_etag(+ team.id,+ json!({+ "flags": [{"key": flag_key, "active": true}],+ "group_type_mapping": {},+ "cohorts": {},+ }),+ etag_value,+ )+ .await+ .unwrap();++ let redis = setup_redis_client(Some(config.redis_url.clone())).await;+ let not_modified_key =+ get_team_request_key(team.id, FlagRequestType::FlagDefinitionsNotModified);+ let full_response_key = get_team_request_key(team.id, FlagRequestType::FlagDefinitions);+ redis.del(not_modified_key.clone()).await.unwrap();+ redis.del(full_response_key.clone()).await.unwrap();++ let server = common::ServerHandle::for_config(config).await;+ let http = reqwest::Client::new();++ let bucket_before = current_bucket();+ let response = http+ .get(format!(+ "http://{}/flags/definitions?token={}",+ server.addr, team.api_token+ ))+ .header("Authorization", format!("Bearer {secret_token}"))+ .header("If-None-Match", format!("W/\"{etag_value}\""))+ .send()+ .await+ .unwrap();+ assert_eq!(response.status(), 304);+ let bucket_after = current_bucket();++ if expect_billed {+ let counter = common::poll_for_billing_counter_across_buckets(+ &redis,+ ¬_modified_key,+ bucket_before,+ bucket_after,+ )+ .await;+ assert_eq!(+ counter, "1",+ "304 should be recorded once on the not-modified billing counter"+ );+ } else {+ tokio::time::sleep(std::time::Duration::from_millis(500)).await;+ for bucket in bucket_before..=bucket_after {+ let counter = redis+ .hget(not_modified_key.clone(), bucket.to_string())+ .await;+ assert!(+ counter.is_err(),+ "304 for non-billable flags must not be billed, got {counter:?}"+ );+ }+ }+ for bucket in bucket_before..=bucket_after {+ let full = redis+ .hget(full_response_key.clone(), bucket.to_string())+ .await;+ assert!(+ full.is_err(),+ "304 must not increment the full local evaluation counter, got {full:?}"+ );+ }+}++/// Only a payload served from Redis decides whether a 304 is billed. With the+/// ETag in Redis but the payload only in S3, neither the full response that+/// precedes the poll nor the poll's own read may memoize S3's answer, so the+/// 304 stays unbilled even though S3 says the definitions are billable.+#[tokio::test]+async fn test_flag_definitions_304_is_not_billed_from_an_s3_payload() {+ use feature_flags::flags::flag_analytics::{current_bucket, get_team_request_key};+ use feature_flags::flags::flag_request::FlagRequestType;+ use feature_flags::utils::test_utils::{setup_redis_client, static_s3_client, TestContext};+ use serde_json::json;++ let config = feature_flags::config::Config::default_test_config();+ let context = TestContext::new(Some(&config)).await;+ let (team, secret_token, _) = context+ .create_team_with_secret_token(None, None, None)+ .await+ .unwrap();++ let payload = json!({+ "flags": [{"key": "billable-flag", "active": true}],+ "group_type_mapping": {},+ "cohorts": {},+ });+ let etag_value = "s3only304etag0001";+ context+ .populate_cache_for_team_with_flags_and_etag(team.id, payload.clone(), etag_value)+ .await+ .unwrap();++ let redis = setup_redis_client(Some(config.redis_url.clone())).await;+ // Leave the ETag in Redis and the payload only in S3.+ redis+ .del(format!(+ "posthog:1:cache/teams/{}/feature_flags/flags_with_cohorts.json",+ team.id+ ))+ .await+ .unwrap();+ let not_modified_key =+ get_team_request_key(team.id, FlagRequestType::FlagDefinitionsNotModified);+ redis.del(not_modified_key.clone()).await.unwrap();++ let server = common::ServerHandle::for_config_with_s3(+ config,+ // S3 serves the same billable payload for every key, standing in for a copy+ // that may lag the ETag Redis holds by a version.+ Some(static_s3_client(&payload.to_string())),+ )+ .await;+ let http = reqwest::Client::new();+ let url = format!(+ "http://{}/flags/definitions?token={}",+ server.addr, team.api_token+ );++ let bucket_before = current_bucket();+ let response = http+ .get(&url)+ .header("Authorization", format!("Bearer {secret_token}"))+ .send()+ .await+ .unwrap();+ assert_eq!(response.status(), 200);++ let response = http+ .get(&url)+ .header("Authorization", format!("Bearer {secret_token}"))+ .header("If-None-Match", format!("W/\"{etag_value}\""))+ .send()+ .await+ .unwrap();+ assert_eq!(response.status(), 304);+ let bucket_after = current_bucket();++ tokio::time::sleep(std::time::Duration::from_millis(500)).await;+ for bucket in bucket_before..=bucket_after {+ let counter = redis+ .hget(not_modified_key.clone(), bucket.to_string())+ .await;+ assert!(+ counter.is_err(),+ "304 must not be billed from an S3 payload, got {counter:?}"+ );+ }+}++/// A full response served from Redis memoizes whether its ETag is billable, so+/// a later 304 for that ETag is billed from the memo even after the Redis+/// payload is gone and the 304's own read would find nothing.+#[tokio::test]+async fn test_flag_definitions_304_is_billed_from_the_full_response_memo() {+ use feature_flags::flags::flag_analytics::{current_bucket, get_team_request_key};+ use feature_flags::flags::flag_request::FlagRequestType;+ use feature_flags::utils::test_utils::{setup_redis_client, TestContext};+ use serde_json::json;++ let config = feature_flags::config::Config::default_test_config();+ let context = TestContext::new(Some(&config)).await;+ let (team, secret_token, _) = context+ .create_team_with_secret_token(None, None, None)+ .await+ .unwrap();++ let etag_value = "memo304etag00001";+ context+ .populate_cache_for_team_with_flags_and_etag(+ team.id,+ json!({+ "flags": [{"key": "billable-flag", "active": true}],+ "group_type_mapping": {},+ "cohorts": {},+ }),+ etag_value,+ )+ .await+ .unwrap();++ let redis = setup_redis_client(Some(config.redis_url.clone())).await;+ let not_modified_key =+ get_team_request_key(team.id, FlagRequestType::FlagDefinitionsNotModified);+ redis.del(not_modified_key.clone()).await.unwrap();++ let server = common::ServerHandle::for_config(config).await;+ let http = reqwest::Client::new();+ let url = format!(+ "http://{}/flags/definitions?token={}",+ server.addr, team.api_token+ );++ let response = http+ .get(&url)+ .header("Authorization", format!("Bearer {secret_token}"))+ .send()+ .await+ .unwrap();+ assert_eq!(response.status(), 200);++ // Keep the ETag in Redis but drop the payload, so only the memo can bill the 304.+ redis+ .del(format!(+ "posthog:1:cache/teams/{}/feature_flags/flags_with_cohorts.json",+ team.id+ ))+ .await+ .unwrap();++ let bucket_before = current_bucket();+ let response = http+ .get(&url)+ .header("Authorization", format!("Bearer {secret_token}"))+ .header("If-None-Match", format!("W/\"{etag_value}\""))+ .send()+ .await+ .unwrap();+ assert_eq!(response.status(), 304);+ let bucket_after = current_bucket();++ let counter = common::poll_for_billing_counter_across_buckets(+ &redis,+ ¬_modified_key,+ bucket_before,+ bucket_after,+ )+ .await;+ assert_eq!(+ counter, "1",+ "304 should be billed from the memo the full response wrote"+ );+}+/// Poll the self-heal rebuild-requests set until it contains `team_id`, or return/// false after ~2s. The enqueue runs in a background task, so a bounded retry is/// needed rather than a single read.diff --git a/rust/usage-ingestion/README.md b/rust/usage-ingestion/README.mdindex 902c511ebac4..d6310cdc89c4 100644--- a/rust/usage-ingestion/README.md+++ b/rust/usage-ingestion/README.md@@ -211,7 +211,7 @@ can count unmerged duplicates and is suitable only for an approximate read.| `ai-ingestion` | `ai_events` | events | `{day}:{sha256 of event, distinct_id, uuid}` | AI ingestion consumer || `error-tracking` | `exceptions` | events | `{day}:{sha256 of event, distinct_id, uuid}` | error tracking server || `cdp` | `cdp_billable_invocations` | invocations | `event:{eventUuid}` / `flow:{invocationId}:{actionStepCount}:{kind}` / `webhook:{invocationId}` | CDP consumers |-| `feature-flags` | `feature_flag_requests`, `feature_flag_local_evaluation_requests` | requests | fresh UUIDv7 per flush | feature flags service |+| `feature-flags` | `feature_flag_requests`, `feature_flag_local_evaluation_requests`, `feature_flag_local_evaluation_not_modified_requests` | requests | fresh UUIDv7 per flush | feature flags service || `ingestion` | `survey_responses` | events | `{day}:{sha256 of event, distinct_id, uuid}` | ingestion consumers, ingestion API || `warehouse-sources` | `warehouse_rows_synced` | rows | the ExternalDataJob ID | warehouse sources worker || `batch-exports` | `batch_export_rows` | rows | the BatchExportRun ID | batch exports worker |@@ -299,9 +299,11 @@ A bounded queue drained by one owned task sends records; shutdown closes andawaits it within the aggregator flush timeout. A full queue drops and countsthe drop rather than growing, while Redis retains the authoritative count.-The raw `feature_flag_requests` and-`feature_flag_local_evaluation_requests` keys stay separate because the report-prices them differently. Remote-config requests bill on neither side.+The raw `feature_flag_requests`,+`feature_flag_local_evaluation_requests`, and+`feature_flag_local_evaluation_not_modified_requests` keys stay separate. The+report prices full local evaluation responses differently, and each key has its+own usage counter, so a 304 on the decide key would be counted twice. Remote-config requests bill on neither side.### Trying it locallydiff --git a/services/mcp/src/api/generated.ts b/services/mcp/src/api/generated.tsindex eb44d44c910e..868ff846a1af 100644--- a/services/mcp/src/api/generated.ts+++ b/services/mcp/src/api/generated.ts@@ -48264,7 +48264,7 @@ export namespace Schemas {sdk: string;/** Number of billable requests in this bucket. */request_count: number;- /** Estimated billing units. Local evaluation requests count as 10 units each. */+ /** Estimated billing units. Local evaluation requests count as 10 units each, except 304 Not Modified responses, which count as 1. */billing_units: number;}