Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

fix(flags): bill local evaluation 304 responses at the /flags rate (#128786)

Gustavo H. Strassburger5e1bf0de3d1c
diff --git a/docs/internal/feature-flags/billing.md b/docs/internal/feature-flags/billing.md
index aca5749d05ad..6365e9b12373 100644
--- a/docs/internal/feature-flags/billing.md
+++ b/docs/internal/feature-flags/billing.md
@@ -47,10 +47,11 @@ When a feature flag request is processed, the Rust service increments Redis coun
The service stores counts in Redis hashes using time-bucketed fields:
-| Request Type | Team Key | SDK Key |
-| ---------------- | --------------------------------------------- | ------------------------------------------------------------ |
-| `/decide` | `posthog:decide_requests:{team_id}` | `posthog:decide_requests:sdk:{team_id}:{sdk_name}` |
-| Local evaluation | `posthog:local_evaluation_requests:{team_id}` | `posthog:local_evaluation_requests:sdk:{team_id}:{sdk_name}` |
+| Request Type | Team Key | SDK Key |
+| ---------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------- |
+| `/decide` | `posthog:decide_requests:{team_id}` | `posthog:decide_requests:sdk:{team_id}:{sdk_name}` |
+| Local evaluation (200) | `posthog:local_evaluation_requests:{team_id}` | `posthog:local_evaluation_requests:sdk:{team_id}:{sdk_name}` |
+| Local evaluation (304) | `posthog:local_evaluation_not_modified_requests:{team_id}` | `posthog:local_evaluation_not_modified_requests:sdk:{team_id}:{sdk_name}` |
### Time bucketing
@@ -93,7 +94,13 @@ Rust's `/flags/definitions` endpoint enforces billing quotas. When a team exceed
}
```
-The Rust endpoint checks `FeatureFlagsLimiter.is_limited(token)` before the ETag comparison and cache fetch. If the quota is exceeded, all requests (including conditional ones) return HTTP 402 — no 304 is issued while a team is over quota. Requests that pass the quota check but result in a 304 (ETag match) are not counted toward billing usage, matching Django's behavior.
+The Rust endpoint checks `FeatureFlagsLimiter.is_limited(token)` before the ETag comparison and cache fetch. If the quota is exceeded, all requests (including conditional ones) return HTTP 402 — no 304 is issued while a team is over quota.
+
+### Conditional requests (ETag)
+
+A request that passes the quota check and results in a 304 (ETag match) is counted on the separate `local_evaluation_not_modified_requests` counter. The usage report bills these at 1 unit each, the same as a `/decide` request, instead of the 10 units a full local evaluation response costs. The usage record key is `feature_flag_local_evaluation_not_modified_requests`, apart from `feature_flag_requests`, because each key has its own usage counter and a shared key would count a 304 twice.
+
+A 304 for definitions that hold only survey or product tour flags is not billed, the same as a 200. The service memoizes per team and ETag whether the definitions hold a billable flag, so it reads the payload once per pod per ETag rather than on every poll. The memo is sized by `DEFINITIONS_BILLABLE_CACHE_CAPACITY` and an entry lives for `DEFINITIONS_BILLABLE_CACHE_TTL_SECONDS` (default one hour), which bounds how long a wrong answer can last. Only a payload served from Redis decides, because an S3 copy can lag the ETag by a version. A payload that came from S3, failed to read, or took longer than 500 ms leaves that poll unbilled, and the miss is remembered for 30 seconds so a degraded payload tier is not hammered by every poll. These unbilled polls show as `flags_flag_definitions_not_modified_billing_total{outcome="unknown"}`, which is why the 304 billing counter can run below the ETag hit counter.
### SDK tracking
@@ -216,10 +223,11 @@ The aggregation task emits events to PostHog's internal analytics instance.
### Event types
-| Event Name | Description |
-| ------------------------ | ---------------------------------------------- |
-| `decide usage` | Counts from `/decide` endpoint requests |
-| `local evaluation usage` | Counts from local evaluation endpoint requests |
+| Event Name | Description |
+| ------------------------------------- | ------------------------------------------------------- |
+| `decide usage` | Counts from `/decide` endpoint requests |
+| `local evaluation usage` | Counts from local evaluation requests answered with 200 |
+| `local evaluation not modified usage` | Counts from local evaluation requests answered with 304 |
### Event properties
@@ -286,6 +294,7 @@ The team can also agree to bill the usage record count with the gap.
- Events, enhanced person events, and exceptions: usage records use capture time, and the legacy queries use the event timestamp. The switch must not count an event on both dates or on neither date.
- Feature flag requests: usage records use the flush time of the Rust flags service. The legacy query uses the later Celery export time.
+- Feature flag local evaluation 304 requests: the same clock difference as feature flag requests.
- Feature flag local evaluation requests: usage records from before [#118814](https://github.com/PostHog/posthog/pull/118814) omit the requests that the Django feature flag list endpoint served. Compare both producers over a complete window before you switch this counter.
- CDP invocations and workflow emails, push, and SMS: usage records and app metrics flush on separate clocks. Each source can put one dispatch on a different date.
- Workflow invocations: the legacy query counts fetch and push dispatches. Before [#118809](https://github.com/PostHog/posthog/pull/118809), a push dispatch wrote only a `workflow_push_sent` record, so `workflow_billable_invocations` held only fetches. Now a push dispatch writes both records. The clock difference above also applies.
@@ -295,16 +304,17 @@ The team can also agree to bill the usage record count with the gap.
### Billable calculation
-Local evaluation requests are weighted 10x compared to decide requests:
+Local evaluation requests that return full flag definitions are weighted 10x compared to decide requests. Local evaluation requests answered with 304 are weighted like decide requests:
```python
billable_feature_flag_requests_count_in_period = (
decide_requests_count_in_period
+ (local_evaluation_requests_count_in_period * 10)
+ + local_evaluation_not_modified_requests_count_in_period
)
```
-This reflects the higher resource cost of local evaluation requests, which return full flag definitions rather than just evaluation results.
+This reflects the higher resource cost of full local evaluation responses, which return full flag definitions rather than just evaluation results.
### Token validation
@@ -323,6 +333,7 @@ AND has([%(validity_token)s], replaceRegexpAll(JSONExtractRaw(properties, 'token
| Distributed lock | Prevent concurrent processing of the same team's data |
| Redis pipelining | Minimize network round-trips for better performance |
| 10x local evaluation weight | Local evaluation returns full flag definitions, requiring more server resources |
+| 1x weight for 304 responses | A 304 serves no definitions payload, so it costs the same as a `/decide` request |
| Selective billing | Survey and product tour flags are internal features, not customer-billable |
| SDK breakdown for analytics only | Billing charges per request regardless of SDK; breakdown is for internal analytics |
@@ -334,7 +345,7 @@ The feature flags **Usage** tab shows the existing billing events grouped by SDK
- Remote evaluation combines `/flags` and legacy `/decide` traffic.
- Local evaluation combines `/flags/definitions` and its legacy alias.
- Counts include billable requests only. They are not raw HTTP traffic totals.
-- Local evaluation requests contribute 10 billing units each.
+- A full local evaluation response contributes 10 billing units. A 304 contributes 1.
- The SDK breakdown comes from the request user-agent and unknown SDKs appear as `other`.
- Aggregation means recent usage can take approximately 30 minutes to appear. Hourly buckets use aggregation event time, so they approximate rather than preserve exact request time.
diff --git a/docs/internal/feature-flags/rust-service-overview.md b/docs/internal/feature-flags/rust-service-overview.md
index b70951996548..0e0e235a446a 100644
--- a/docs/internal/feature-flags/rust-service-overview.md
+++ b/docs/internal/feature-flags/rust-service-overview.md
@@ -148,7 +148,7 @@ Billing quota enforcement matches Django's `/api/feature_flag/local_evaluation`
- **Quota check**: Uses `FeatureFlagsLimiter.is_limited(token)` to verify the team hasn't exceeded their feature flag request quota. Returns HTTP 402 with a JSON body (`{"type": "quota_limited", "code": "payment_required", ...}`) when the quota is exceeded.
- **Non-billable flag filtering**: Usage tracking skips requests where the response contains only non-billable flags — i.e., flags with keys starting with `survey-targeting-` or `product-tour-targeting-`. The shared `is_billable_flag_key()` predicate (in `flag_analytics.rs`) is used by both this endpoint and the `/flags` billing handler.
-- **304 responses skip billing**: Usage is recorded after the ETag/304 path, so conditional responses that return `304 Not Modified` are not counted toward billing. This matches Django's behavior.
+- **304 responses bill 1 unit**: A conditional response that returns `304 Not Modified` is recorded on its own counter and billed at the `/flags` rate, a tenth of a full response. See [Conditional requests (ETag)](billing.md#conditional-requests-etag) for the billable-flag memo and what goes unbilled.
## Request and response types
diff --git a/ee/billing/quota_limiting.py b/ee/billing/quota_limiting.py
index 85bc2fb4dd3c..2fbff3126764 100644
--- a/ee/billing/quota_limiting.py
+++ b/ee/billing/quota_limiting.py
@@ -43,6 +43,7 @@ from posthog.usage_counters import UsageCounter, UsageCounterMode, UsageCounterP
from posthog.utils import get_current_day
from products.ai_observability.backend.offline_evaluation_billing import get_offline_evaluation_usage
+from products.feature_flags.backend.flag_analytics import billable_feature_flag_requests
from ee.billing.quota_usage import QuotaUsageBatch, retry_quota_database_operation
@@ -1266,14 +1267,20 @@ def _aggregate_quota_usage(
for team in teams:
decide_requests = all_data["teams_with_decide_requests_count"].get(team.id, 0)
local_evaluation_requests = all_data["teams_with_local_evaluation_requests_count"].get(team.id, 0)
+ local_evaluation_not_modified_requests = all_data[
+ "teams_with_local_evaluation_not_modified_requests_count"
+ ].get(team.id, 0)
team_report = UsageCounters(
events=all_data["teams_with_event_count_in_period"].get(team.id, 0),
exceptions=all_data["teams_with_exceptions_captured_in_period"].get(team.id, 0),
recordings=all_data["teams_with_recording_count_in_period"].get(team.id, 0),
rows_synced=all_data["teams_with_rows_synced_in_period"].get(team.id, 0),
- feature_flag_requests=decide_requests
- + (local_evaluation_requests * 10), # Same weighting as in _get_team_report
+ feature_flag_requests=billable_feature_flag_requests(
+ decide=decide_requests,
+ local_evaluation=local_evaluation_requests,
+ local_evaluation_not_modified=local_evaluation_not_modified_requests,
+ ),
api_queries_read_bytes=all_data["teams_with_api_queries_read_bytes"].get(team.id, 0),
survey_responses=all_data["teams_with_survey_responses_count_in_period"].get(team.id, 0),
llm_events=all_data["teams_with_ai_event_count_in_period"].get(team.id, 0),
@@ -1367,6 +1374,7 @@ def update_all_orgs_billing_quotas(
UsageCounter.RECORDINGS,
UsageCounter.FEATURE_FLAG_REQUESTS,
UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS,
+ UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS,
UsageCounter.CDP_INVOCATIONS,
UsageCounter.SURVEY_RESPONSES,
UsageCounter.AI_EVENTS,
@@ -1417,6 +1425,9 @@ def update_all_orgs_billing_quotas(
"teams_with_local_evaluation_requests_count": dict(
counter_report.counts[UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS]
),
+ "teams_with_local_evaluation_not_modified_requests_count": dict(
+ counter_report.counts[UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS]
+ ),
"teams_with_api_queries_read_bytes": convert_team_usage_rows_to_dict(api_queries_usage["read_bytes"]),
"teams_with_cdp_trigger_events_metrics": dict(counter_report.counts[UsageCounter.CDP_INVOCATIONS]),
"teams_with_rows_exported_in_period": dict(counter_report.counts[UsageCounter.ROWS_EXPORTED]),
diff --git a/ee/billing/test/test_quota_limiting.py b/ee/billing/test/test_quota_limiting.py
index c2ea5e00c749..2485eecb9cb9 100644
--- a/ee/billing/test/test_quota_limiting.py
+++ b/ee/billing/test/test_quota_limiting.py
@@ -346,7 +346,12 @@ class TestQuotaLimiting(BaseTest):
@patch("posthoganalytics.capture")
def test_todays_usage_reads_every_legacy_row_shape(self, _capture) -> None:
def flag_requests(_begin, _end, request_type):
- return [(str(self.team.id), 7 if request_type == FlagRequestType.DECIDE else 2)]
+ counts = {
+ FlagRequestType.DECIDE: 7,
+ FlagRequestType.LOCAL_EVALUATION: 2,
+ FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED: 3,
+ }
+ return [(str(self.team.id), counts[request_type])]
with (
self.settings(USE_TZ=False),
@@ -375,7 +380,7 @@ class TestQuotaLimiting(BaseTest):
self.organization.refresh_from_db()
assert self.organization.usage is not None
- assert self.organization.usage["feature_flag_requests"]["todays_usage"] == 7 + 2 * 10
+ assert self.organization.usage["feature_flag_requests"]["todays_usage"] == 7 + 2 * 10 + 3
assert self.organization.usage["rows_synced"]["todays_usage"] == 5
assert self.organization.usage["rows_exported"]["todays_usage"] == 3
diff --git a/posthog/constants.py b/posthog/constants.py
index f2b0c872d949..9fc83040dd8c 100644
--- a/posthog/constants.py
+++ b/posthog/constants.py
@@ -246,6 +246,7 @@ class EventDefinitionType(StrEnum):
class FlagRequestType(StrEnum):
DECIDE = "decide"
LOCAL_EVALUATION = "local-evaluation"
+ LOCAL_EVALUATION_NOT_MODIFIED = "local-evaluation-not-modified"
REMOTE_CONFIG = "remote-config"
diff --git a/posthog/tasks/test/__snapshots__/test_usage_report.ambr b/posthog/tasks/test/__snapshots__/test_usage_report.ambr
index 6adb1f4a2ea7..9c897a9606de 100644
--- a/posthog/tasks/test/__snapshots__/test_usage_report.ambr
+++ b/posthog/tasks/test/__snapshots__/test_usage_report.ambr
@@ -41,6 +41,20 @@
# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.100
'''
+ SELECT team_id,
+ SUM(count) as count
+ FROM app_metrics2
+ WHERE app_source='hog_function'
+ AND metric_name IN ('fetch')
+ AND timestamp >= '2022-01-10 00:00:00'
+ AND timestamp < '2022-01-10 23:59:59'
+ GROUP BY team_id,
+ metric_name
+ '''
+# ---
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.101
+ '''
+
SELECT team_id,
SUM(count) as count
FROM app_metrics2
@@ -53,7 +67,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.101
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.102
'''
SELECT team_id,
@@ -70,7 +84,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.102
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.103
'''
SELECT team_id,
@@ -474,6 +488,20 @@
# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.33
'''
+ SELECT distinct_id as team,
+ sum(toInt64OrZero(replaceRegexpAll(JSONExtractRaw(properties, 'count'), '^"|"$', ''))) as sum
+ FROM events
+ WHERE team_id = 99999
+ AND event='local evaluation not modified usage'
+ AND timestamp >= '2022-01-10 00:00:00'
+ AND timestamp < '2022-01-10 23:59:59'
+ AND has(['correct'], replaceRegexpAll(JSONExtractRaw(properties, 'token'), '^"|"$', ''))
+ GROUP BY team
+ '''
+# ---
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.34
+ '''
+
SELECT team_id,
SUM(count) as count
FROM app_metrics2
@@ -485,7 +513,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.34
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.35
'''
SELECT team_id,
@@ -499,7 +527,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.35
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.36
'''
SELECT team_id,
@@ -513,7 +541,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.36
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.37
'''
SELECT team_id,
@@ -528,7 +556,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.37
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.38
'''
SELECT team_id,
@@ -541,7 +569,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.38
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.39
'''
SELECT team_id,
@@ -558,19 +586,6 @@
metric_name
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.39
- '''
-
- SELECT team_id,
- SUM(count) as count
- FROM app_metrics2
- WHERE app_source='logs'
- AND metric_name='retention_byte_days'
- AND timestamp >= '2022-01-10 00:00:00'
- AND timestamp < '2022-01-10 23:59:59'
- GROUP BY team_id
- '''
-# ---
# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.4
'''
@@ -587,6 +602,19 @@
# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.40
'''
+ SELECT team_id,
+ SUM(count) as count
+ FROM app_metrics2
+ WHERE app_source='logs'
+ AND metric_name='retention_byte_days'
+ AND timestamp >= '2022-01-10 00:00:00'
+ AND timestamp < '2022-01-10 23:59:59'
+ GROUP BY team_id
+ '''
+# ---
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.41
+ '''
+
SELECT team_id,
multiIf(event LIKE 'helicone%', 'helicone_events', event LIKE 'langfuse%', 'langfuse_events', event LIKE 'keywords_ai%', 'keywords_ai_events', event LIKE 'traceloop%', 'traceloop_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'web', 'web_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'js', 'web_lite_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-node', 'node_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-node-mcp', 'node_mcp_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-python-mcp', 'python_mcp_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-rs-mcp', 'rust_mcp_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-go-mcp', 'go_mcp_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-ruby-mcp', 'ruby_mcp_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-edge', 'edge_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-convex', 'convex_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-android', 'android_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-kmp', 'kmp_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-flutter', 'flutter_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-ios', 'ios_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-go', 'go_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-java', 'java_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-server', 'java_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-react-native', 'react_native_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-ruby', 'ruby_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-rails', 'ruby_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-python', 'python_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-php', 'php_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-laravel', 'laravel_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-dotnet', 'dotnet_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-aspnetcore', 'dotnet_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-elixir', 'elixir_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-unity', 'unity_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-godot', 'godot_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-unreal', 'unreal_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-roblox', 'roblox_events', replaceRegexpAll(JSONExtractRaw(properties, '$lib'), '^"|"$', '') = 'posthog-rs', 'rust_events', 'other') AS metric,
count(1) as count
@@ -631,7 +659,7 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.41
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.42
'''
SELECT team_id,
@@ -678,7 +706,7 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.42
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.43
'''
SELECT team_id,
@@ -725,7 +753,7 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.43
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.44
'''
SELECT team_id,
@@ -772,7 +800,7 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.44
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.45
'''
SELECT team_id,
@@ -819,7 +847,7 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.45
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.46
'''
SELECT team_id,
@@ -866,7 +894,7 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.46
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.47
'''
SELECT team_id,
@@ -913,7 +941,7 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.47
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.48
'''
SELECT team_id,
@@ -960,7 +988,7 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.48
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.49
'''
SELECT team_id,
@@ -1007,7 +1035,20 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.49
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.5
+ '''
+
+ SELECT team_id,
+ count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count
+ FROM events
+ WHERE timestamp >= '2022-01-10 10:00:00'
+ AND timestamp < '2022-01-10 12:00:00'
+ AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']
+ AND NOT startsWith(event, '$ai_')
+ GROUP BY team_id
+ '''
+# ---
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.50
'''
SELECT team_id,
@@ -1054,20 +1095,7 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.5
- '''
-
- SELECT team_id,
- count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count
- FROM events
- WHERE timestamp >= '2022-01-10 10:00:00'
- AND timestamp < '2022-01-10 12:00:00'
- AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']
- AND NOT startsWith(event, '$ai_')
- GROUP BY team_id
- '''
-# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.50
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.51
'''
SELECT team_id,
@@ -1114,7 +1142,7 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.51
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.52
'''
SELECT team_id,
@@ -1161,7 +1189,7 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.52
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.53
'''
SELECT team_id,
@@ -1172,7 +1200,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.53
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.54
'''
SELECT team_id,
@@ -1195,7 +1223,7 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.54
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.55
'''
SELECT team_id,
@@ -1218,7 +1246,7 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.55
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.56
'''
SELECT team_id,
@@ -1241,7 +1269,7 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.56
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.57
'''
SELECT team_id,
@@ -1264,7 +1292,7 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.57
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.58
'''
SELECT team_id,
@@ -1287,7 +1315,7 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.58
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.59
'''
SELECT team_id,
@@ -1310,7 +1338,20 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.59
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.6
+ '''
+
+ SELECT team_id,
+ count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count
+ FROM events
+ WHERE timestamp >= '2022-01-10 12:00:00'
+ AND timestamp < '2022-01-10 14:00:00'
+ AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']
+ AND NOT startsWith(event, '$ai_')
+ GROUP BY team_id
+ '''
+# ---
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.60
'''
SELECT team_id,
@@ -1333,20 +1374,7 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.6
- '''
-
- SELECT team_id,
- count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count
- FROM events
- WHERE timestamp >= '2022-01-10 12:00:00'
- AND timestamp < '2022-01-10 14:00:00'
- AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']
- AND NOT startsWith(event, '$ai_')
- GROUP BY team_id
- '''
-# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.60
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.61
'''
SELECT team_id,
@@ -1369,7 +1397,7 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.61
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.62
'''
SELECT team_id,
@@ -1392,7 +1420,7 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.62
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.63
'''
SELECT team_id,
@@ -1415,7 +1443,7 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.63
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.64
'''
SELECT team_id,
@@ -1438,7 +1466,7 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.64
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.65
'''
SELECT team_id,
@@ -1461,7 +1489,7 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.65
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.66
'''
SELECT team_id,
@@ -1480,7 +1508,7 @@
event
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.66
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.67
'''
SELECT JSONExtractInt(log_comment, 'team_id') team_id,
@@ -1496,7 +1524,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.67
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.68
'''
SELECT team_id,
@@ -1509,7 +1537,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.68
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.69
'''
SELECT team_id,
@@ -1525,7 +1553,20 @@
metric_name
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.69
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.7
+ '''
+
+ SELECT team_id,
+ count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count
+ FROM events
+ WHERE timestamp >= '2022-01-10 14:00:00'
+ AND timestamp < '2022-01-10 16:00:00'
+ AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']
+ AND NOT startsWith(event, '$ai_')
+ GROUP BY team_id
+ '''
+# ---
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.70
'''
SELECT team_id,
@@ -1541,20 +1582,7 @@
metric_name
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.7
- '''
-
- SELECT team_id,
- count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count
- FROM events
- WHERE timestamp >= '2022-01-10 14:00:00'
- AND timestamp < '2022-01-10 16:00:00'
- AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']
- AND NOT startsWith(event, '$ai_')
- GROUP BY team_id
- '''
-# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.70
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.71
'''
SELECT team_id,
@@ -1572,7 +1600,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.71
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.72
'''
SELECT team_id,
@@ -1590,7 +1618,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.72
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.73
'''
SELECT team_id,
@@ -1608,7 +1636,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.73
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.74
'''
SELECT team_id,
@@ -1626,7 +1654,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.74
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.75
'''
SELECT team_id,
@@ -1644,7 +1672,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.75
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.76
'''
SELECT team_id,
@@ -1662,7 +1690,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.76
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.77
'''
SELECT team_id,
@@ -1680,7 +1708,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.77
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.78
'''
SELECT team_id,
@@ -1698,7 +1726,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.78
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.79
'''
SELECT team_id,
@@ -1716,38 +1744,38 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.79
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.8
'''
SELECT team_id,
count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count
FROM events
- WHERE timestamp >= '2022-01-10 18:00:00'
- AND timestamp < '2022-01-10 20:00:00'
+ WHERE timestamp >= '2022-01-10 16:00:00'
+ AND timestamp < '2022-01-10 18:00:00'
AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']
AND NOT startsWith(event, '$ai_')
- AND ($group_0 != ''
- OR $group_1 != ''
- OR $group_2 != ''
- OR $group_3 != ''
- OR $group_4 != '')
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.8
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.80
'''
SELECT team_id,
count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count
FROM events
- WHERE timestamp >= '2022-01-10 16:00:00'
- AND timestamp < '2022-01-10 18:00:00'
+ WHERE timestamp >= '2022-01-10 18:00:00'
+ AND timestamp < '2022-01-10 20:00:00'
AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']
AND NOT startsWith(event, '$ai_')
+ AND ($group_0 != ''
+ OR $group_1 != ''
+ OR $group_2 != ''
+ OR $group_3 != ''
+ OR $group_4 != '')
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.80
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.81
'''
SELECT team_id,
@@ -1765,7 +1793,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.81
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.82
'''
SELECT team_id,
@@ -1783,7 +1811,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.82
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.83
'''
SELECT team_id,
@@ -1794,7 +1822,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.83
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.84
'''
SELECT team_id,
@@ -1817,7 +1845,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.84
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.85
'''
SELECT team_id,
@@ -1841,7 +1869,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.85
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.86
'''
SELECT team_id,
@@ -1865,7 +1893,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.86
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.87
'''
WITH JSONExtractInt(log_comment, 'team_id') as team_id,
JSONExtractString(log_comment, 'query_type') as query_type,
@@ -1884,7 +1912,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.87
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.88
'''
WITH JSONExtractInt(log_comment, 'team_id') as team_id,
JSONExtractString(log_comment, 'query_type') as query_type,
@@ -1903,7 +1931,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.88
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.89
'''
WITH JSONExtractInt(log_comment, 'team_id') as team_id,
JSONExtractString(log_comment, 'query_type') as query_type,
@@ -1922,7 +1950,20 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.89
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.9
+ '''
+
+ SELECT team_id,
+ count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count
+ FROM events
+ WHERE timestamp >= '2022-01-10 18:00:00'
+ AND timestamp < '2022-01-10 20:00:00'
+ AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']
+ AND NOT startsWith(event, '$ai_')
+ GROUP BY team_id
+ '''
+# ---
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.90
'''
WITH JSONExtractInt(log_comment, 'team_id') as team_id,
JSONExtractString(log_comment, 'query_type') as query_type,
@@ -1941,20 +1982,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.9
- '''
-
- SELECT team_id,
- count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count
- FROM events
- WHERE timestamp >= '2022-01-10 18:00:00'
- AND timestamp < '2022-01-10 20:00:00'
- AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']
- AND NOT startsWith(event, '$ai_')
- GROUP BY team_id
- '''
-# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.90
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.91
'''
WITH JSONExtractInt(log_comment, 'team_id') as team_id,
JSONExtractString(log_comment, 'query_type') as query_type,
@@ -1973,7 +2001,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.91
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.92
'''
WITH JSONExtractInt(log_comment, 'team_id') as team_id,
JSONExtractString(log_comment, 'query_type') as query_type,
@@ -1992,7 +2020,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.92
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.93
'''
WITH JSONExtractInt(log_comment, 'team_id') as team_id,
JSONExtractString(log_comment, 'query_type') as query_type,
@@ -2012,7 +2040,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.93
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.94
'''
WITH JSONExtractInt(log_comment, 'team_id') as team_id,
JSONExtractString(log_comment, 'query_type') as query_type,
@@ -2032,7 +2060,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.94
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.95
'''
WITH JSONExtractInt(log_comment, 'team_id') as team_id,
JSONExtractString(log_comment, 'query_type') as query_type,
@@ -2052,7 +2080,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.95
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.96
'''
WITH JSONExtractInt(log_comment, 'team_id') as team_id,
JSONExtractString(log_comment, 'query_type') as query_type,
@@ -2072,7 +2100,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.96
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.97
'''
WITH JSONExtractInt(log_comment, 'team_id') as team_id,
JSONExtractString(log_comment, 'query_type') as query_type,
@@ -2092,7 +2120,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.97
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.98
'''
WITH JSONExtractInt(log_comment, 'team_id') as team_id,
JSONExtractString(log_comment, 'query_type') as query_type,
@@ -2112,7 +2140,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.98
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.99
'''
SELECT team_id,
@@ -2127,17 +2155,3 @@
metric_name
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests.99
- '''
-
- SELECT team_id,
- SUM(count) as count
- FROM app_metrics2
- WHERE app_source='hog_function'
- AND metric_name IN ('fetch')
- AND timestamp >= '2022-01-10 00:00:00'
- AND timestamp < '2022-01-10 23:59:59'
- GROUP BY team_id,
- metric_name
- '''
-# ---
diff --git a/posthog/tasks/test/__snapshots__/test_usage_report.new_events_schema.ambr b/posthog/tasks/test/__snapshots__/test_usage_report.new_events_schema.ambr
index e73977bcdd55..29da8d2a3e4b 100644
--- a/posthog/tasks/test/__snapshots__/test_usage_report.new_events_schema.ambr
+++ b/posthog/tasks/test/__snapshots__/test_usage_report.new_events_schema.ambr
@@ -332,6 +332,35 @@
# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.29]
'''
+ SELECT distinct_id as team,
+ sum(toInt64OrZero(ifNull(if(if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.^count), '\\/', '/'), toJSONString(properties.^count)) != '{}', if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.^count), '\\/', '/'), toJSONString(properties.^count)), if(ascii(toString(properties.count)) IN (91, 123)
+ AND ascii(if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.count), '\\/', '/'), toJSONString(properties.count))) IN (91, 123), nullIf(nullIf(if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.count), '\\/', '/'), toJSONString(properties.count)), '[]'), '{}'), toString(properties.count))), ''))) as sum
+ FROM events_json
+ WHERE team_id = 99999
+ AND event='local evaluation not modified usage'
+ AND timestamp >= '2022-01-10 00:00:00'
+ AND timestamp < '2022-01-10 23:59:59'
+ AND has(['correct'], ifNull(if(if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.^token), '\\/', '/'), toJSONString(properties.^token)) != '{}', if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.^token), '\\/', '/'), toJSONString(properties.^token)), if(ascii(toString(properties.token)) IN (91, 123)
+ AND ascii(if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.token), '\\/', '/'), toJSONString(properties.token))) IN (91, 123), nullIf(nullIf(if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.token), '\\/', '/'), toJSONString(properties.token)), '[]'), '{}'), toString(properties.token))), ''))
+ GROUP BY team
+ '''
+# ---
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.2]
+ '''
+
+ SELECT team_id,
+ count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count
+ FROM events_json
+ WHERE timestamp >= '2022-01-10 04:00:00'
+ AND timestamp < '2022-01-10 06:00:00'
+ AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']
+ AND NOT startsWith(event, '$ai_')
+ GROUP BY team_id
+ '''
+# ---
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.30]
+ '''
+
SELECT team_id,
multiIf(event LIKE 'helicone%', 'helicone_events', event LIKE 'langfuse%', 'langfuse_events', event LIKE 'keywords_ai%', 'keywords_ai_events', event LIKE 'traceloop%', 'traceloop_events', ifNull(if(if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.^`$lib`), '\\/', '/'), toJSONString(properties.^`$lib`)) != '{}', if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.^`$lib`), '\\/', '/'), toJSONString(properties.^`$lib`)), if(ascii(toString(properties.`$lib`)) IN (91, 123)
AND ascii(if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.`$lib`), '\\/', '/'), toJSONString(properties.`$lib`))) IN (91, 123), nullIf(nullIf(if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.`$lib`), '\\/', '/'), toJSONString(properties.`$lib`)), '[]'), '{}'), toString(properties.`$lib`))), '') = 'web', 'web_events', ifNull(if(if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.^`$lib`), '\\/', '/'), toJSONString(properties.^`$lib`)) != '{}', if(getSetting('output_format_json_escape_forward_slashes'), replaceAll(toJSONString(properties.^`$lib`), '\\/', '/'), toJSONString(properties.^`$lib`)), if(ascii(toString(properties.`$lib`)) IN (91, 123)
@@ -408,20 +437,7 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.2]
- '''
-
- SELECT team_id,
- count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count
- FROM events_json
- WHERE timestamp >= '2022-01-10 04:00:00'
- AND timestamp < '2022-01-10 06:00:00'
- AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']
- AND NOT startsWith(event, '$ai_')
- GROUP BY team_id
- '''
-# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.30]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.31]
'''
SELECT team_id,
@@ -500,7 +516,7 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.31]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.32]
'''
SELECT team_id,
@@ -579,7 +595,7 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.32]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.33]
'''
SELECT team_id,
@@ -658,7 +674,7 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.33]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.34]
'''
SELECT team_id,
@@ -737,7 +753,7 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.34]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.35]
'''
SELECT team_id,
@@ -816,7 +832,7 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.35]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.36]
'''
SELECT team_id,
@@ -895,7 +911,7 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.36]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.37]
'''
SELECT team_id,
@@ -974,7 +990,7 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.37]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.38]
'''
SELECT team_id,
@@ -1053,7 +1069,7 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.38]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.39]
'''
SELECT team_id,
@@ -1132,7 +1148,20 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.39]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.3]
+ '''
+
+ SELECT team_id,
+ count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count
+ FROM events_json
+ WHERE timestamp >= '2022-01-10 06:00:00'
+ AND timestamp < '2022-01-10 08:00:00'
+ AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']
+ AND NOT startsWith(event, '$ai_')
+ GROUP BY team_id
+ '''
+# ---
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.40]
'''
SELECT team_id,
@@ -1211,20 +1240,7 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.3]
- '''
-
- SELECT team_id,
- count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count
- FROM events_json
- WHERE timestamp >= '2022-01-10 06:00:00'
- AND timestamp < '2022-01-10 08:00:00'
- AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']
- AND NOT startsWith(event, '$ai_')
- GROUP BY team_id
- '''
-# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.40]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.41]
'''
SELECT team_id,
@@ -1303,7 +1319,7 @@
metric
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.41]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.42]
'''
SELECT team_id,
@@ -1330,7 +1346,7 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.42]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.43]
'''
SELECT team_id,
@@ -1357,7 +1373,7 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.43]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.44]
'''
SELECT team_id,
@@ -1384,7 +1400,7 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.44]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.45]
'''
SELECT team_id,
@@ -1411,7 +1427,7 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.45]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.46]
'''
SELECT team_id,
@@ -1438,7 +1454,7 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.46]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.47]
'''
SELECT team_id,
@@ -1465,7 +1481,7 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.47]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.48]
'''
SELECT team_id,
@@ -1492,7 +1508,7 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.48]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.49]
'''
SELECT team_id,
@@ -1519,7 +1535,20 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.49]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.4]
+ '''
+
+ SELECT team_id,
+ count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count
+ FROM events_json
+ WHERE timestamp >= '2022-01-10 08:00:00'
+ AND timestamp < '2022-01-10 10:00:00'
+ AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']
+ AND NOT startsWith(event, '$ai_')
+ GROUP BY team_id
+ '''
+# ---
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.50]
'''
SELECT team_id,
@@ -1546,20 +1575,7 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.4]
- '''
-
- SELECT team_id,
- count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count
- FROM events_json
- WHERE timestamp >= '2022-01-10 08:00:00'
- AND timestamp < '2022-01-10 10:00:00'
- AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']
- AND NOT startsWith(event, '$ai_')
- GROUP BY team_id
- '''
-# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.50]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.51]
'''
SELECT team_id,
@@ -1586,7 +1602,7 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.51]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.52]
'''
SELECT team_id,
@@ -1613,7 +1629,7 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.52]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.53]
'''
SELECT team_id,
@@ -1640,7 +1656,7 @@
ai_lib
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.53]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.54]
'''
SELECT team_id,
@@ -1658,7 +1674,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.54]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.55]
'''
SELECT team_id,
@@ -1676,7 +1692,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.55]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.56]
'''
SELECT team_id,
@@ -1694,7 +1710,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.56]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.57]
'''
SELECT team_id,
@@ -1712,7 +1728,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.57]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.58]
'''
SELECT team_id,
@@ -1730,7 +1746,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.58]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.59]
'''
SELECT team_id,
@@ -1748,38 +1764,38 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.59]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.5]
'''
SELECT team_id,
count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count
FROM events_json
- WHERE timestamp >= '2022-01-10 12:00:00'
- AND timestamp < '2022-01-10 14:00:00'
+ WHERE timestamp >= '2022-01-10 10:00:00'
+ AND timestamp < '2022-01-10 12:00:00'
AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']
AND NOT startsWith(event, '$ai_')
- AND (properties.`$group_0` != ''
- OR properties.`$group_1` != ''
- OR properties.`$group_2` != ''
- OR properties.`$group_3` != ''
- OR properties.`$group_4` != '')
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.5]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.60]
'''
SELECT team_id,
count(distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)) as count
FROM events_json
- WHERE timestamp >= '2022-01-10 10:00:00'
- AND timestamp < '2022-01-10 12:00:00'
+ WHERE timestamp >= '2022-01-10 12:00:00'
+ AND timestamp < '2022-01-10 14:00:00'
AND event NOT IN ['$feature_flag_called', '$experiment_exposure', 'survey sent', 'survey shown', 'survey dismissed', '$exception', '$llm_prompt_fetched', '$conversations_loaded', '$conversations_widget_loaded', '$conversations_message_sent', '$conversations_user_identified', '$conversations_restore_link_requested', '$conversations_widget_state_changed', '$conversations_back_to_tickets', '$anomaly_score', '$anomaly_score_shadow']
AND NOT startsWith(event, '$ai_')
+ AND (properties.`$group_0` != ''
+ OR properties.`$group_1` != ''
+ OR properties.`$group_2` != ''
+ OR properties.`$group_3` != ''
+ OR properties.`$group_4` != '')
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.60]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.61]
'''
SELECT team_id,
@@ -1797,7 +1813,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.61]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.62]
'''
SELECT team_id,
@@ -1815,7 +1831,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.62]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.63]
'''
SELECT team_id,
@@ -1833,7 +1849,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.63]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.64]
'''
SELECT team_id,
@@ -1851,7 +1867,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.64]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.65]
'''
SELECT team_id,
@@ -1869,7 +1885,7 @@
GROUP BY team_id
'''
# ---
-# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.65]
+# name: TestFeatureFlagsUsageReport.test_usage_report_decide_requests[new_events_schema.66]
'''
SELECT team_id,
diff --git a/posthog/tasks/test/test_usage_counters.py b/posthog/tasks/test/test_usage_counters.py
index 2ac7d7ba0f9c..003d65c65e58 100644
--- a/posthog/tasks/test/test_usage_counters.py
+++ b/posthog/tasks/test/test_usage_counters.py
@@ -11,6 +11,7 @@ from parameterized import parameterized
from posthog.tasks import usage_report
from posthog.usage_counters import (
COUNTER_FLAG_NAMES,
+ PLAN_DEFERRED_COUNTERS,
SHADOW_FAILURES,
UsageCounter,
UsageCounterCaller,
@@ -169,9 +170,15 @@ class TestUsageCounterReport(SimpleTestCase):
"get_teams_with_cdp_billable_invocations_in_period": self.legacy[UsageCounter.CDP_INVOCATIONS],
"get_teams_with_feature_flag_requests_count_in_period": Mock(
side_effect=lambda begin, end, kind: self.legacy[
- UsageCounter.FEATURE_FLAG_REQUESTS
- if kind == usage_report.FlagRequestType.DECIDE
- else UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS
+ {
+ usage_report.FlagRequestType.DECIDE: UsageCounter.FEATURE_FLAG_REQUESTS,
+ usage_report.FlagRequestType.LOCAL_EVALUATION: (
+ UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS
+ ),
+ usage_report.FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED: (
+ UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS
+ ),
+ }[kind]
](begin, end)
),
"get_teams_with_workflow_emails_sent_in_period": self.legacy[UsageCounter.WORKFLOW_EMAILS],
@@ -217,13 +224,14 @@ class TestUsageCounterReport(SimpleTestCase):
report = service.fetch_report(period, plan=plan)
assert flag.call_count == len(COUNTER_FLAG_NAMES)
+ planned_counters = [counter for counter in UsageCounter if counter not in PLAN_DEFERRED_COUNTERS]
assert report.counts == {
- **{key: [] for key in (*UsageCounter, *SANDBOX_COMPUTE_RESOURCE_KEYS)},
+ **{key: [] for key in (*planned_counters, *SANDBOX_COMPUTE_RESOURCE_KEYS)},
UsageCounter.CDP_INVOCATIONS.value: [(1, 12)],
UsageCounter.WORKFLOW_EMAILS.value: [(1, 3)],
}
assert report.usage_sources == {
- **{counter.value.removeprefix("teams_with_"): "legacy" for counter in UsageCounter},
+ **{counter.value.removeprefix("teams_with_"): "legacy" for counter in planned_counters},
"cdp_billable_invocations_in_period": "both",
}
assert report.counter_comparisons == (
@@ -397,6 +405,7 @@ class TestUsageCounterReport(SimpleTestCase):
legacy_by_team={}, realtime_by_team={}
)
for counter in COUNTER_FLAG_NAMES
+ if counter not in PLAN_DEFERRED_COUNTERS
}
@parameterized.expand([("success",), ("records",), ("legacy_comparison",)])
@@ -466,12 +475,20 @@ class TestUsageCounterReport(SimpleTestCase):
UsageCounter.CDP_INVOCATIONS: ("cdp_billable_invocations", 111),
UsageCounter.FEATURE_FLAG_REQUESTS: ("feature_flag_requests", 108),
UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS: ("feature_flag_local_evaluation_requests", 109),
+ UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS: (
+ "feature_flag_local_evaluation_not_modified_requests",
+ 115,
+ ),
UsageCounter.WORKFLOW_EMAILS: ("workflow_emails_sent", 112),
UsageCounter.WORKFLOW_PUSH: ("workflow_push_sent", 113),
UsageCounter.WORKFLOW_SMS: ("workflow_sms_sent", 114),
UsageCounter.WORKFLOW_INVOCATIONS: ("workflow_billable_invocations", 110),
}
- flag_counters = {UsageCounter.FEATURE_FLAG_REQUESTS, UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS}
+ flag_counters = {
+ UsageCounter.FEATURE_FLAG_REQUESTS,
+ UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS,
+ UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS,
+ }
for index, counter in enumerate(counter_keys):
self.legacy[counter].return_value = [("1" if counter in flag_counters else 1, index + 1)]
self.exceptions.side_effect = None
@@ -491,6 +508,8 @@ class TestUsageCounterReport(SimpleTestCase):
)
),
patch("posthoganalytics.get_feature_flag", return_value=mode if source == "flag" else "legacy"),
+ # Deferred counters read their record keys once they join plans, so check those keys here too.
+ patch("posthog.usage_counters.PLAN_DEFERRED_COUNTERS", frozenset()),
):
service = UsageCounterService()
plan = service.resolve_plan(period, complete=True, counters=tuple(counter_keys))
diff --git a/posthog/tasks/test/test_usage_report.py b/posthog/tasks/test/test_usage_report.py
index d1ab0aa47617..210fc6e14ee2 100644
--- a/posthog/tasks/test/test_usage_report.py
+++ b/posthog/tasks/test/test_usage_report.py
@@ -795,6 +795,7 @@ class TestUsageReport(APIBaseTest, ClickhouseTestMixin, ClickhouseDestroyTablesM
"resolved_symbol_sets_count": 0,
"decide_requests_count_in_period": 0,
"local_evaluation_requests_count_in_period": 0,
+ "local_evaluation_not_modified_requests_count_in_period": 0,
"billable_feature_flag_requests_count_in_period": 0,
"survey_count": 0,
"survey_responses_count_in_period": 1,
@@ -901,6 +902,7 @@ class TestUsageReport(APIBaseTest, ClickhouseTestMixin, ClickhouseDestroyTablesM
"resolved_symbol_sets_count": 0,
"decide_requests_count_in_period": 0,
"local_evaluation_requests_count_in_period": 0,
+ "local_evaluation_not_modified_requests_count_in_period": 0,
"billable_feature_flag_requests_count_in_period": 0,
"survey_count": 0,
"survey_responses_count_in_period": 1,
@@ -1001,6 +1003,7 @@ class TestUsageReport(APIBaseTest, ClickhouseTestMixin, ClickhouseDestroyTablesM
"resolved_symbol_sets_count": 0,
"decide_requests_count_in_period": 0,
"local_evaluation_requests_count_in_period": 0,
+ "local_evaluation_not_modified_requests_count_in_period": 0,
"billable_feature_flag_requests_count_in_period": 0,
"survey_count": 0,
"survey_responses_count_in_period": 0,
@@ -1124,6 +1127,7 @@ class TestUsageReport(APIBaseTest, ClickhouseTestMixin, ClickhouseDestroyTablesM
"resolved_symbol_sets_count": 0,
"decide_requests_count_in_period": 0,
"local_evaluation_requests_count_in_period": 0,
+ "local_evaluation_not_modified_requests_count_in_period": 0,
"billable_feature_flag_requests_count_in_period": 0,
"survey_count": 0,
"survey_responses_count_in_period": 0,
@@ -1230,6 +1234,7 @@ class TestUsageReport(APIBaseTest, ClickhouseTestMixin, ClickhouseDestroyTablesM
"resolved_symbol_sets_count": 0,
"decide_requests_count_in_period": 0,
"local_evaluation_requests_count_in_period": 0,
+ "local_evaluation_not_modified_requests_count_in_period": 0,
"billable_feature_flag_requests_count_in_period": 0,
"survey_count": 0,
"survey_responses_count_in_period": 0,
@@ -2147,6 +2152,14 @@ class TestFeatureFlagsUsageReport(ClickhouseDestroyTablesMixin, TestCase, Clickh
team=self.analytics_team,
)
+ _create_event(
+ distinct_id="3",
+ event="local evaluation not modified usage",
+ properties={"count": 5, "token": "correct"},
+ timestamp=now(),
+ team=self.analytics_team,
+ )
+
for i in range(5):
_create_event(
distinct_id="4",
@@ -2198,11 +2211,13 @@ class TestFeatureFlagsUsageReport(ClickhouseDestroyTablesMixin, TestCase, Clickh
assert org_1_report["organization_name"] == "Org 1"
assert org_1_report["local_evaluation_requests_count_in_period"] == 11
+ assert org_1_report["local_evaluation_not_modified_requests_count_in_period"] == 5
assert org_1_report["decide_requests_count_in_period"] == 0
- assert org_1_report["billable_feature_flag_requests_count_in_period"] == 110
+ assert org_1_report["billable_feature_flag_requests_count_in_period"] == 115
assert org_1_report["teams"]["3"]["local_evaluation_requests_count_in_period"] == 10
+ assert org_1_report["teams"]["3"]["local_evaluation_not_modified_requests_count_in_period"] == 5
assert org_1_report["teams"]["4"]["local_evaluation_requests_count_in_period"] == 1
- assert org_1_report["teams"]["3"]["billable_feature_flag_requests_count_in_period"] == 100
+ assert org_1_report["teams"]["3"]["billable_feature_flag_requests_count_in_period"] == 105
assert org_1_report["teams"]["4"]["billable_feature_flag_requests_count_in_period"] == 10
# because of wrong token, Org 2 has no decide counts.
@@ -2526,6 +2541,7 @@ class TestHasNonZeroUsage(SimpleTestCase):
("empty", None),
("events", "event_count_in_period"),
("mcp_events", "mcp_event_count_in_period"),
+ ("local_evaluation_not_modified", "local_evaluation_not_modified_requests_count_in_period"),
("business_knowledge_chunks", "business_knowledge_chunks_in_period"),
("business_knowledge_queries", "business_knowledge_queries_in_period"),
("logs_bytes", "logs_bytes_in_period"),
diff --git a/posthog/tasks/usage_report.py b/posthog/tasks/usage_report.py
index e980c235443a..f7c9d01aec2c 100644
--- a/posthog/tasks/usage_report.py
+++ b/posthog/tasks/usage_report.py
@@ -72,6 +72,7 @@ from products.cdp.backend.models.plugin import PluginConfig
from products.dashboards.backend.models.dashboard import Dashboard
from products.data_modeling.backend.facade.models import DataWarehouseSavedQuery
from products.error_tracking.backend.facade import api as error_tracking_api
+from products.feature_flags.backend.flag_analytics import USAGE_EVENT_NAMES, billable_feature_flag_requests
from products.feature_flags.backend.models.feature_flag import FeatureFlag
from products.replay_vision.backend.billing import (
get_replay_vision_credits_by_team,
@@ -253,6 +254,7 @@ class UsageReportCounters:
ff_active_count: int
decide_requests_count_in_period: int
local_evaluation_requests_count_in_period: int
+ local_evaluation_not_modified_requests_count_in_period: int
billable_feature_flag_requests_count_in_period: int
# Queries
@@ -1549,7 +1551,7 @@ def get_teams_with_feature_flag_requests_count_in_period(
team_to_query = 1 if get_instance_region() == "EU" else 2
validity_token = settings.DECIDE_BILLING_ANALYTICS_TOKEN
- target_event = "decide usage" if request_type == FlagRequestType.DECIDE else "local evaluation usage"
+ target_event = USAGE_EVENT_NAMES[request_type]
use_new = use_new_events_schema(None)
count_expr, _ = get_property_string_expr("events", "count", "'count'", "properties", use_new_events_schema=use_new)
@@ -1590,7 +1592,7 @@ def get_teams_with_feature_flag_requests_sdk_breakdown_in_period(
team_to_query = 1 if get_instance_region() == "EU" else 2
validity_token = settings.DECIDE_BILLING_ANALYTICS_TOKEN
- target_event = "decide usage" if request_type == FlagRequestType.DECIDE else "local evaluation usage"
+ target_event = USAGE_EVENT_NAMES[request_type]
use_new = use_new_events_schema(None)
sdk_breakdown_expr, _ = get_property_string_expr(
@@ -3102,6 +3104,7 @@ def has_non_zero_usage(report: UsageReportCounters) -> bool:
or report.mobile_recording_count_in_period > 0
or report.decide_requests_count_in_period > 0
or report.local_evaluation_requests_count_in_period > 0
+ or report.local_evaluation_not_modified_requests_count_in_period > 0
or report.survey_responses_count_in_period > 0
or report.rows_synced_in_period > 0
or report.free_historical_rows_synced_in_period > 0
@@ -3512,6 +3515,9 @@ def _get_team_report(all_data: dict[str, Any], team: Team) -> UsageReportCounter
local_evaluation_requests_count_in_period = all_data["teams_with_local_evaluation_requests_count_in_period"].get(
team.id, 0
)
+ local_evaluation_not_modified_requests_count_in_period = all_data[
+ "teams_with_local_evaluation_not_modified_requests_count_in_period"
+ ].get(team.id, 0)
logs_bytes_in_period = all_data["teams_with_logs_bytes_in_period"].get(team.id, 0)
apm_tracing_bytes_in_period = all_data["teams_with_apm_tracing_bytes_in_period"].get(team.id, 0)
return UsageReportCounters(
@@ -3556,8 +3562,12 @@ def _get_team_report(all_data: dict[str, Any], team: Team) -> UsageReportCounter
group_types_total=all_data["teams_with_group_types_total"].get(team.id, 0),
decide_requests_count_in_period=decide_requests_count_in_period,
local_evaluation_requests_count_in_period=local_evaluation_requests_count_in_period,
- billable_feature_flag_requests_count_in_period=decide_requests_count_in_period
- + (local_evaluation_requests_count_in_period * 10),
+ local_evaluation_not_modified_requests_count_in_period=local_evaluation_not_modified_requests_count_in_period,
+ billable_feature_flag_requests_count_in_period=billable_feature_flag_requests(
+ decide=decide_requests_count_in_period,
+ local_evaluation=local_evaluation_requests_count_in_period,
+ local_evaluation_not_modified=local_evaluation_not_modified_requests_count_in_period,
+ ),
dashboard_count=all_data["teams_with_dashboard_count"].get(team.id, 0),
dashboard_template_count=all_data["teams_with_dashboard_template_count"].get(team.id, 0),
dashboard_shared_count=all_data["teams_with_dashboard_shared_count"].get(team.id, 0),
diff --git a/posthog/temporal/tests/usage_report/test_aggregator.py b/posthog/temporal/tests/usage_report/test_aggregator.py
index 5a93e2a7b6f4..90b01a7749ad 100644
--- a/posthog/temporal/tests/usage_report/test_aggregator.py
+++ b/posthog/temporal/tests/usage_report/test_aggregator.py
@@ -26,6 +26,7 @@ from posthog.tasks.usage_report import (
from posthog.temporal.tests.usage_report.test_aggregate_activity import _instance_metadata
from posthog.temporal.usage_report.aggregator import (
add_pre_business_knowledge_usage_patch_defaults,
+ add_pre_local_evaluation_not_modified_patch_defaults,
add_pre_logs_distinct_patterns_patch_defaults,
add_pre_mcp_event_count_patch_defaults,
add_pre_replay_vision_spend_projection_patch_defaults,
@@ -176,6 +177,10 @@ def test_load_all_data_defaults_compute_for_pre_patch_workflow_history() -> None
(add_pre_logs_distinct_patterns_patch_defaults, "teams_with_logs_distinct_patterns_in_period"),
(add_pre_transactional_emails_patch_defaults, "teams_with_workflow_transactional_emails_sent_in_period"),
(add_pre_mcp_event_count_patch_defaults, "teams_with_mcp_event_count_in_period"),
+ (
+ add_pre_local_evaluation_not_modified_patch_defaults,
+ "teams_with_local_evaluation_not_modified_requests_count_in_period",
+ ),
],
)
def test_load_all_data_defaults_single_query_for_pre_patch_workflow_history(
diff --git a/posthog/temporal/tests/usage_report/test_parity.py b/posthog/temporal/tests/usage_report/test_parity.py
index f83db418431b..aed181f6894a 100644
--- a/posthog/temporal/tests/usage_report/test_parity.py
+++ b/posthog/temporal/tests/usage_report/test_parity.py
@@ -84,6 +84,7 @@ def _seed_all_data(team_a_id: int, team_b_id: int, team_c_id: int) -> dict[str,
"teams_with_recording_bytes_in_period": {team_a_id: 1_234_567, team_c_id: 89_012},
"teams_with_decide_requests_count_in_period": {team_a_id: 1_000, team_b_id: 200, team_c_id: 50},
"teams_with_local_evaluation_requests_count_in_period": {team_b_id: 5},
+ "teams_with_local_evaluation_not_modified_requests_count_in_period": {team_b_id: 40, team_c_id: 9},
"teams_with_dashboard_count": {team_a_id: 4, team_b_id: 2, team_c_id: 1},
"teams_with_ff_count": {team_a_id: 7, team_b_id: 3},
"teams_with_survey_count": {team_a_id: 1, team_c_id: 2},
@@ -361,9 +362,13 @@ def test_end_to_end_parity_celery_task_vs_temporal_activity(
lambda begin, end, kind: [
(str(team_id), total)
for team_id, total in seeded[
- UsageCounter.FEATURE_FLAG_REQUESTS
- if kind == usage_report.FlagRequestType.DECIDE
- else UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS
+ {
+ usage_report.FlagRequestType.DECIDE: UsageCounter.FEATURE_FLAG_REQUESTS,
+ usage_report.FlagRequestType.LOCAL_EVALUATION: UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS,
+ usage_report.FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED: (
+ UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS
+ ),
+ }[kind]
].items()
],
)
diff --git a/posthog/temporal/tests/usage_report/test_workflow.py b/posthog/temporal/tests/usage_report/test_workflow.py
index 3ead8755e816..3bb690b7f0ac 100644
--- a/posthog/temporal/tests/usage_report/test_workflow.py
+++ b/posthog/temporal/tests/usage_report/test_workflow.py
@@ -48,6 +48,7 @@ from posthog.temporal.usage_report.types import (
from posthog.temporal.usage_report.workflow import (
AGGREGATION_QUEUE_PATCH_ID,
BUSINESS_KNOWLEDGE_USAGE_QUERY_NAME,
+ LOCAL_EVALUATION_NOT_MODIFIED_QUERY_NAME,
LOGS_DISTINCT_PATTERNS_QUERY_NAME,
MCP_EVENT_COUNT_QUERY_NAME,
PARALLEL_COUNTER_QUERIES_PATCH_ID,
@@ -58,6 +59,7 @@ from posthog.temporal.usage_report.workflow import (
USAGE_COUNTER_SERVICE_PATCH_ID,
RunUsageReportsWorkflow,
_queries_for_business_knowledge_usage_patch,
+ _queries_for_local_evaluation_not_modified_patch,
_queries_for_logs_distinct_patterns_patch,
_queries_for_mcp_event_count_patch,
_queries_for_replay_vision_spend_projection_patch,
@@ -68,6 +70,7 @@ from posthog.temporal.usage_report.workflow import (
from posthog.usage_counters import (
COUNTER_FLAG_NAMES,
LEGACY_QUERY_NAMES,
+ PLAN_DEFERRED_COUNTERS,
UsageCounter,
UsageCounterMode,
UsageCounterPlan,
@@ -90,6 +93,11 @@ def test_sandbox_compute_query_is_versioned_for_existing_histories() -> None:
REPLAY_VISION_SPEND_PROJECTION_QUERY_NAME,
),
("business_knowledge_usage", _queries_for_business_knowledge_usage_patch, BUSINESS_KNOWLEDGE_USAGE_QUERY_NAME),
+ (
+ "local_evaluation_not_modified",
+ _queries_for_local_evaluation_not_modified_patch,
+ LOCAL_EVALUATION_NOT_MODIFIED_QUERY_NAME,
+ ),
]
)
def test_single_query_is_versioned_for_existing_histories(
@@ -471,6 +479,7 @@ async def test_workflow_runs_query_then_aggregate(report_behavior: str, caplog:
assert plans[0].modes == {
counter: expected_mode if counter in COUNTER_FLAG_NAMES else UsageCounterMode.LEGACY
for counter in UsageCounter
+ if counter not in PLAN_DEFERRED_COUNTERS
}
assert aggregate_payloads[0].ctx.usage_counter_plan == (plans[0] if plans else None)
diff --git a/posthog/temporal/usage_report/activities.py b/posthog/temporal/usage_report/activities.py
index 21ab30256a62..5fc158f11489 100644
--- a/posthog/temporal/usage_report/activities.py
+++ b/posthog/temporal/usage_report/activities.py
@@ -27,6 +27,7 @@ from posthog.temporal.common.heartbeat import Heartbeater
from posthog.temporal.common.metrics import ExecutionTimeRecorder
from posthog.temporal.usage_report.aggregator import (
add_pre_business_knowledge_usage_patch_defaults,
+ add_pre_local_evaluation_not_modified_patch_defaults,
add_pre_logs_distinct_patterns_patch_defaults,
add_pre_mcp_event_count_patch_defaults,
add_pre_replay_vision_spend_projection_patch_defaults,
@@ -256,6 +257,7 @@ async def aggregate_and_chunk_org_reports(inputs: AggregateInputs) -> AggregateR
add_pre_mcp_event_count_patch_defaults(all_data, inputs.query_results)
add_pre_replay_vision_spend_projection_patch_defaults(all_data, inputs.query_results)
add_pre_business_knowledge_usage_patch_defaults(all_data, inputs.query_results)
+ add_pre_local_evaluation_not_modified_patch_defaults(all_data, inputs.query_results)
check_cancelled()
counter_report = load_usage_counter_report(inputs)
all_data.update({field: dict(rows) for field, rows in counter_report.counts.items()})
diff --git a/posthog/temporal/usage_report/aggregator.py b/posthog/temporal/usage_report/aggregator.py
index 10145604274d..b195c8973082 100644
--- a/posthog/temporal/usage_report/aggregator.py
+++ b/posthog/temporal/usage_report/aggregator.py
@@ -53,6 +53,7 @@ from posthog.usage_counters import UsageCounterReport
from products.ai_observability.backend.offline_evaluation_billing import get_offline_evaluation_usage
_SANDBOX_COMPUTE_QUERY_NAME = "sandbox_compute_usage"
+_LOCAL_EVALUATION_NOT_MODIFIED_QUERY_NAME = "teams_with_local_evaluation_not_modified_requests_count_in_period"
_SANDBOX_COMPUTE_DESTINATION_KEYS = (
"teams_with_sandbox_compute_credits_used_in_period",
"teams_with_sandbox_compute_cpu_millicore_seconds_in_period",
@@ -147,6 +148,13 @@ def add_pre_business_knowledge_usage_patch_defaults(
all_data[key] = {}
+def add_pre_local_evaluation_not_modified_patch_defaults(
+ all_data: dict[str, dict[int, int]], query_results: list[RunQueryToS3Result]
+) -> None:
+ if not any(result.query_name == _LOCAL_EVALUATION_NOT_MODIFIED_QUERY_NAME for result in query_results):
+ all_data[_LOCAL_EVALUATION_NOT_MODIFIED_QUERY_NAME] = {}
+
+
def iter_chunk_lines(
org_reports: Iterable[OrgReport],
instance_metadata: InstanceMetadata,
diff --git a/posthog/temporal/usage_report/queries.py b/posthog/temporal/usage_report/queries.py
index 872c8c3d1991..407b8b4ff109 100644
--- a/posthog/temporal/usage_report/queries.py
+++ b/posthog/temporal/usage_report/queries.py
@@ -404,6 +404,12 @@ QUERIES: list[QuerySpec] = [
name="teams_with_local_evaluation_requests_count_in_period",
fn=lambda b, e: UsageCounterService().get_legacy(UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS, b, e),
),
+ QuerySpec(
+ name="teams_with_local_evaluation_not_modified_requests_count_in_period",
+ fn=lambda b, e: UsageCounterService().get_legacy(
+ UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS, b, e
+ ),
+ ),
# ---- ClickHouse: query metrics -------------------------------------------
QuerySpec(
name="teams_with_query_app_bytes_read",
diff --git a/posthog/temporal/usage_report/workflow.py b/posthog/temporal/usage_report/workflow.py
index f44fdbe69dd7..37c88234f41c 100644
--- a/posthog/temporal/usage_report/workflow.py
+++ b/posthog/temporal/usage_report/workflow.py
@@ -66,6 +66,8 @@ REPLAY_VISION_SPEND_PROJECTION_QUERY_PATCH_ID = "usage-report-replay-vision-spen
REPLAY_VISION_SPEND_PROJECTION_QUERY_NAME = "replay_vision_spend_projection"
BUSINESS_KNOWLEDGE_USAGE_QUERY_PATCH_ID = "usage-report-business-knowledge-usage-query-2026-11"
BUSINESS_KNOWLEDGE_USAGE_QUERY_NAME = "business_knowledge_usage"
+LOCAL_EVALUATION_NOT_MODIFIED_QUERY_PATCH_ID = "usage-report-local-evaluation-not-modified-query-2026-11"
+LOCAL_EVALUATION_NOT_MODIFIED_QUERY_NAME = UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS.value
def _queries_for_sandbox_compute_patch(patch_applied: bool) -> list[QuerySpec]:
@@ -104,6 +106,12 @@ def _queries_for_business_knowledge_usage_patch(queries: list[QuerySpec], patch_
return [spec for spec in queries if spec.name != BUSINESS_KNOWLEDGE_USAGE_QUERY_NAME]
+def _queries_for_local_evaluation_not_modified_patch(queries: list[QuerySpec], patch_applied: bool) -> list[QuerySpec]:
+ if patch_applied:
+ return queries
+ return [spec for spec in queries if spec.name != LOCAL_EVALUATION_NOT_MODIFIED_QUERY_NAME]
+
+
def build_context(inputs: RunUsageReportsInputs, run_id: str, now: datetime) -> WorkflowContext:
"""Compute the period and S3 layout context. Pure function so it's safe
to call from the workflow body (no real-time access).
@@ -177,6 +185,9 @@ class RunUsageReportsWorkflow(PostHogWorkflow):
queries = _queries_for_business_knowledge_usage_patch(
queries, workflow.patched(BUSINESS_KNOWLEDGE_USAGE_QUERY_PATCH_ID)
)
+ queries = _queries_for_local_evaluation_not_modified_patch(
+ queries, workflow.patched(LOCAL_EVALUATION_NOT_MODIFIED_QUERY_PATCH_ID)
+ )
if workflow.patched(USAGE_COUNTER_SERVICE_PATCH_ID):
ctx.usage_counter_plan = await workflow.execute_activity(
plan_usage_counters,
diff --git a/posthog/test/test_feature_flag_analytics.py b/posthog/test/test_feature_flag_analytics.py
index c14202fd6d9e..21a7d238ab70 100644
--- a/posthog/test/test_feature_flag_analytics.py
+++ b/posthog/test/test_feature_flag_analytics.py
@@ -102,7 +102,7 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):
self.assertEqual(client.hgetall(f"posthog:decide_requests:other"), {})
@patch("products.feature_flags.backend.flag_analytics.CACHE_BUCKET_SIZE", 10)
- def test_increment_request_count_remote_config_uses_own_bucket(self):
+ def test_increment_request_count_uses_one_bucket_per_request_type(self):
team_id = 3
with time_machine.travel("2022-05-07 12:23:07", tick=False):
@@ -110,11 +110,14 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):
increment_request_count(team_id)
for _ in range(6):
increment_request_count(team_id, 1, FlagRequestType.REMOTE_CONFIG)
+ for _ in range(2):
+ increment_request_count(team_id, 1, FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED)
client = redis.get_client()
# Remote config fetches are telemetry-only, so they must never leak into the
- # decide bucket that billing consumes.
+ # decide bucket that billing consumes. The literal keys are the contract with the
+ # Rust service, which writes them, so a drift would leave those requests unbilled.
self.assertEqual(
client.hgetall(f"posthog:decide_requests:{team_id}"),
{b"165192618": b"4"},
@@ -123,6 +126,10 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):
client.hgetall(f"posthog:remote_config_requests:{team_id}"),
{b"165192618": b"6"},
)
+ self.assertEqual(
+ client.hgetall(f"posthog:local_evaluation_not_modified_requests:{team_id}"),
+ {b"165192618": b"2"},
+ )
@patch("products.feature_flags.backend.flag_analytics.CACHE_BUCKET_SIZE", 10)
def test_capture_team_decide_usage(self):
@@ -140,6 +147,7 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):
# 10 requests in first bucket
increment_request_count(team_id)
increment_request_count(team_id, 1, FlagRequestType.LOCAL_EVALUATION)
+ increment_request_count(team_id, 1, FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED)
increment_request_count(team_id, 1, FlagRequestType.REMOTE_CONFIG)
for _ in range(7):
# 7 requests for other team
@@ -151,6 +159,7 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):
# 5 requests in second bucket
increment_request_count(team_id)
increment_request_count(team_id, 1, FlagRequestType.LOCAL_EVALUATION)
+ increment_request_count(team_id, 1, FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED)
increment_request_count(team_id, 1, FlagRequestType.REMOTE_CONFIG)
for _ in range(3):
# 3 requests for other team
@@ -162,6 +171,7 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):
# 5 requests in third bucket
increment_request_count(team_id)
increment_request_count(team_id, 1, FlagRequestType.LOCAL_EVALUATION)
+ increment_request_count(team_id, 1, FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED)
increment_request_count(team_id, 1, FlagRequestType.REMOTE_CONFIG)
increment_request_count(other_team_id)
@@ -169,7 +179,7 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):
# these other requests should not add duplicate counts
capture_team_decide_usage(mock_capture, team_id, team_uuid)
capture_team_decide_usage(mock_capture, team_id, team_uuid)
- assert mock_capture.capture.call_count == 3
+ assert mock_capture.capture.call_count == 4
mock_capture.capture.assert_any_call(
distinct_id=team_id,
event="decide usage",
@@ -194,6 +204,18 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):
"token": "token",
},
)
+ mock_capture.capture.assert_any_call(
+ distinct_id=team_id,
+ event="local evaluation not modified usage",
+ properties={
+ "count": 15,
+ "team_id": team_id,
+ "team_uuid": team_uuid,
+ "max_time": 1651926190,
+ "min_time": 1651926180,
+ "token": "token",
+ },
+ )
mock_capture.capture.assert_any_call(
distinct_id=team_id,
event="remote config usage",
@@ -304,6 +326,7 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):
)
local_evaluation_team_id = Team.objects.create(organization=self.organization).id
remote_config_team_id = Team.objects.create(organization=self.organization).id
+ not_modified_team_id = Team.objects.create(organization=self.organization).id
with time_machine.travel("2022-05-07 12:23:07", tick=False) as frozen_datetime:
for _ in range(10):
@@ -314,6 +337,7 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):
increment_request_count(other_team_id)
increment_request_count(local_evaluation_team_id, request_type=FlagRequestType.LOCAL_EVALUATION)
increment_request_count(remote_config_team_id, request_type=FlagRequestType.REMOTE_CONFIG)
+ increment_request_count(not_modified_team_id, request_type=FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED)
frozen_datetime.shift(datetime.timedelta(seconds=5))
@@ -332,6 +356,7 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):
increment_request_count(other_team_id)
increment_request_count(local_evaluation_team_id, request_type=FlagRequestType.LOCAL_EVALUATION)
increment_request_count(remote_config_team_id, request_type=FlagRequestType.REMOTE_CONFIG)
+ increment_request_count(not_modified_team_id, request_type=FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED)
with (
self.settings(DECIDE_BILLING_ANALYTICS_TOKEN="token"),
@@ -376,13 +401,15 @@ class TestFeatureFlagAnalytics(BaseTest, QueryMatchingTest):
(other_team_id, "decide usage"),
(local_evaluation_team_id, "local evaluation usage"),
(remote_config_team_id, "remote config usage"),
+ (not_modified_team_id, "local evaluation not modified usage"),
}
- assert mock_capture.capture.call_count == 4
+ assert mock_capture.capture.call_count == 5
assert {call.args[1] for call in team_export.call_args_list} == {
team_id,
other_team_id,
local_evaluation_team_id,
remote_config_team_id,
+ not_modified_team_id,
}
def test_failed_key_check_exports_every_team_in_the_batch(self) -> None:
diff --git a/posthog/usage_counters.py b/posthog/usage_counters.py
index b7efec49927e..276a68b3980e 100644
--- a/posthog/usage_counters.py
+++ b/posthog/usage_counters.py
@@ -39,6 +39,9 @@ class UsageCounter(StrEnum):
CDP_INVOCATIONS = "teams_with_cdp_billable_invocations_in_period"
FEATURE_FLAG_REQUESTS = "teams_with_decide_requests_count_in_period"
FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS = "teams_with_local_evaluation_requests_count_in_period"
+ FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS = (
+ "teams_with_local_evaluation_not_modified_requests_count_in_period"
+ )
WORKFLOW_EMAILS = "teams_with_workflow_emails_sent_in_period"
WORKFLOW_PUSH = "teams_with_workflow_push_sent_in_period"
WORKFLOW_SMS = "teams_with_workflow_sms_sent_in_period"
@@ -65,6 +68,10 @@ LEGACY_QUERY_NAMES = {
UsageCounter.LOGS_RETENTION_30D_BYTES: "logs_retention_bytes",
UsageCounter.SANDBOX_COMPUTE_CREDITS: "sandbox_compute_usage",
}
+# Temporal workers decode a plan's modes into UsageCounter, so a worker without a counter rejects any plan that names it.
+# During a rolling deploy or a rollback, older workers still run, so a new counter stays out of plans until every worker
+# can decode it. Usage reports v2 reads these counters from their legacy query specs meanwhile.
+PLAN_DEFERRED_COUNTERS = frozenset({UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS})
class UsageCounterMode(StrEnum):
@@ -102,6 +109,7 @@ RECORD_USAGE_KEYS = {
UsageCounter.CDP_INVOCATIONS: "cdp_billable_invocations",
UsageCounter.FEATURE_FLAG_REQUESTS: "feature_flag_requests",
UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS: "feature_flag_local_evaluation_requests",
+ UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS: "feature_flag_local_evaluation_not_modified_requests",
UsageCounter.WORKFLOW_EMAILS: "workflow_emails_sent",
UsageCounter.WORKFLOW_PUSH: "workflow_push_sent",
UsageCounter.WORKFLOW_SMS: "workflow_sms_sent",
@@ -118,6 +126,7 @@ RECORD_USAGE_UNITS = {
"cdp_billable_invocations": "invocations",
"feature_flag_requests": "requests",
"feature_flag_local_evaluation_requests": "requests",
+ "feature_flag_local_evaluation_not_modified_requests": "requests",
"workflow_emails_sent": "invocations",
"workflow_push_sent": "invocations",
"workflow_sms_sent": "invocations",
@@ -135,6 +144,7 @@ COUNTER_FLAG_NAMES = {
UsageCounter.CDP_INVOCATIONS: "cdp-invocations",
UsageCounter.FEATURE_FLAG_REQUESTS: "feature-flag-requests",
UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_REQUESTS: "feature-flag-local-evaluation-requests",
+ UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS: "feature-flag-local-evaluation-not-modified-requests",
UsageCounter.WORKFLOW_EMAILS: "workflow-emails",
UsageCounter.WORKFLOW_PUSH: "workflow-push",
UsageCounter.WORKFLOW_SMS: "workflow-sms",
@@ -282,6 +292,11 @@ class UsageCounterService:
begin, end, usage_report.FlagRequestType.LOCAL_EVALUATION
)
),
+ UsageCounter.FEATURE_FLAG_LOCAL_EVALUATION_NOT_MODIFIED_REQUESTS: lambda begin, end: (
+ usage_report.get_teams_with_feature_flag_requests_count_in_period(
+ begin, end, usage_report.FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED
+ )
+ ),
UsageCounter.WORKFLOW_EMAILS: usage_report.get_teams_with_workflow_emails_sent_in_period,
UsageCounter.WORKFLOW_PUSH: usage_report.get_teams_with_workflow_push_sent_in_period,
UsageCounter.WORKFLOW_SMS: usage_report.get_teams_with_workflow_sms_sent_in_period,
@@ -317,7 +332,11 @@ class UsageCounterService:
return UsageCounterPlan(
period=period,
caller="usage_reports_v2",
- modes={counter: mode for counter, mode in modes.items() if counters is None or counter in counters},
+ modes={
+ counter: mode
+ for counter, mode in modes.items()
+ if counter not in PLAN_DEFERRED_COUNTERS and (counters is None or counter in counters)
+ },
complete=complete,
)
diff --git a/products/feature_flags/backend/flag_analytics.py b/products/feature_flags/backend/flag_analytics.py
index f5c44072f14f..32ea27835306 100644
--- a/products/feature_flags/backend/flag_analytics.py
+++ b/products/feature_flags/backend/flag_analytics.py
@@ -81,22 +81,38 @@ SDK_LIBRARIES = [
# locally. It's not included in CI because of tricky patching freeze time in thread issues.
-# Remote config requests are tracked for telemetry only; billing consumes just the decide and
+# Remote config requests are tracked for telemetry only; billing consumes the decide and
# local evaluation events (see usage_report.py), so the remote config event never bills.
+# Local evaluation 304s get their own counter because they bill at the decide rate.
_REQUEST_BUCKET_PREFIXES = {
FlagRequestType.DECIDE: "decide_requests",
FlagRequestType.LOCAL_EVALUATION: "local_evaluation_requests",
+ FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED: "local_evaluation_not_modified_requests",
FlagRequestType.REMOTE_CONFIG: "remote_config_requests",
}
-EXPORTED_REQUEST_TYPES = (FlagRequestType.DECIDE, FlagRequestType.LOCAL_EVALUATION, FlagRequestType.REMOTE_CONFIG)
+EXPORTED_REQUEST_TYPES = (
+ FlagRequestType.DECIDE,
+ FlagRequestType.LOCAL_EVALUATION,
+ FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED,
+ FlagRequestType.REMOTE_CONFIG,
+)
USAGE_EVENT_NAMES = {
FlagRequestType.DECIDE: "decide usage",
FlagRequestType.LOCAL_EVALUATION: "local evaluation usage",
+ FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED: "local evaluation not modified usage",
FlagRequestType.REMOTE_CONFIG: "remote config usage",
}
+# A full local evaluation response costs ten decide requests. A 304 costs one, because it
+# sends no definitions.
+LOCAL_EVALUATION_BILLING_WEIGHT = 10
+
+
+def billable_feature_flag_requests(*, decide: int, local_evaluation: int, local_evaluation_not_modified: int) -> int:
+ return decide + local_evaluation * LOCAL_EVALUATION_BILLING_WEIGHT + local_evaluation_not_modified
+
def _request_bucket_prefix(request_type: FlagRequestType) -> str:
try:
diff --git a/products/feature_flags/backend/presentation/request_usage.py b/products/feature_flags/backend/presentation/request_usage.py
index 474e284ee9df..b693407bb541 100644
--- a/products/feature_flags/backend/presentation/request_usage.py
+++ b/products/feature_flags/backend/presentation/request_usage.py
@@ -69,7 +69,7 @@ class FeatureFlagRequestUsageItemSerializer(DataclassSerializer):
"sdk": {"help_text": "SDK family parsed from the request user agent."},
"request_count": {"help_text": "Number of billable requests in this bucket."},
"billing_units": {
- "help_text": "Estimated billing units. Local evaluation requests count as 10 units each."
+ "help_text": "Estimated billing units. Local evaluation requests count as 10 units each, except 304 Not Modified responses, which count as 1."
},
}
diff --git a/products/feature_flags/backend/request_usage.py b/products/feature_flags/backend/request_usage.py
index 40ad034b0bf3..a28fcdf4417c 100644
--- a/products/feature_flags/backend/request_usage.py
+++ b/products/feature_flags/backend/request_usage.py
@@ -16,7 +16,7 @@ from posthog.models.event.new_events_schema import events_read_table, use_new_ev
from posthog.models.property.util import get_property_string_expr
from posthog.utils import get_instance_region
-from products.feature_flags.backend.flag_analytics import USAGE_EVENT_NAMES
+from products.feature_flags.backend.flag_analytics import USAGE_EVENT_NAMES, billable_feature_flag_requests
class FeatureFlagRequestType(StrEnum):
@@ -40,10 +40,24 @@ REQUEST_USAGE_QUERY_SETTINGS = {
"max_threads": 4,
}
-FeatureFlagRequestUsageRow = tuple[datetime, Literal["remote_evaluation", "local_evaluation"], int, str]
-
-# Keep aligned with usage_report.py, which applies this multiplier to billed local evaluation requests.
-LOCAL_EVALUATION_BILLING_WEIGHT = 10
+RawRequestType = Literal["remote_evaluation", "local_evaluation", "local_evaluation_not_modified"]
+FeatureFlagRequestUsageRow = tuple[datetime, RawRequestType, int, str]
+
+# A 304 is a local evaluation request, so it shows with the others, but it bills like a remote one.
+_RAW_REQUEST_TYPES: dict[RawRequestType, tuple[FeatureFlagRequestType, int]] = {
+ "remote_evaluation": (
+ FeatureFlagRequestType.REMOTE_EVALUATION,
+ billable_feature_flag_requests(decide=1, local_evaluation=0, local_evaluation_not_modified=0),
+ ),
+ "local_evaluation": (
+ FeatureFlagRequestType.LOCAL_EVALUATION,
+ billable_feature_flag_requests(decide=0, local_evaluation=1, local_evaluation_not_modified=0),
+ ),
+ "local_evaluation_not_modified": (
+ FeatureFlagRequestType.LOCAL_EVALUATION,
+ billable_feature_flag_requests(decide=0, local_evaluation=0, local_evaluation_not_modified=1),
+ ),
+}
def parse_sdk_breakdown(raw_sdk_breakdown: str) -> dict[str, int]:
@@ -64,12 +78,15 @@ def aggregate_feature_flag_request_usage(
rows: Iterable[FeatureFlagRequestUsageRow],
) -> list[FeatureFlagRequestUsage]:
counts: defaultdict[tuple[datetime, FeatureFlagRequestType, str], int] = defaultdict(int)
+ billing_units: defaultdict[tuple[datetime, FeatureFlagRequestType, str], int] = defaultdict(int)
for bucket, raw_request_type, total_count, raw_sdk_breakdown in rows:
- request_type = FeatureFlagRequestType(raw_request_type)
+ request_type, weight = _RAW_REQUEST_TYPES[raw_request_type]
sdk_breakdown = parse_sdk_breakdown(raw_sdk_breakdown)
- for sdk, count in sdk_breakdown.items():
+ sdk_counts = dict(sdk_breakdown)
+ sdk_counts["other"] = sdk_counts.get("other", 0) + max(total_count - sum(sdk_breakdown.values()), 0)
+ for sdk, count in sdk_counts.items():
counts[(bucket, request_type, sdk)] += count
- counts[(bucket, request_type, "other")] += max(total_count - sum(sdk_breakdown.values()), 0)
+ billing_units[(bucket, request_type, sdk)] += count * weight
return [
FeatureFlagRequestUsage(
@@ -77,8 +94,7 @@ def aggregate_feature_flag_request_usage(
request_type=request_type,
sdk=sdk,
request_count=request_count,
- billing_units=request_count
- * (LOCAL_EVALUATION_BILLING_WEIGHT if request_type == FeatureFlagRequestType.LOCAL_EVALUATION else 1),
+ billing_units=billing_units[(bucket, request_type, sdk)],
)
for (bucket, request_type, sdk), request_count in sorted(counts.items())
if request_count > 0
@@ -99,6 +115,7 @@ def query_feature_flag_request_usage(
bucket_function = "toStartOfHour" if time_interval == "hour" else "toStartOfDay"
decide_event = USAGE_EVENT_NAMES[FlagRequestType.DECIDE]
local_event = USAGE_EVENT_NAMES[FlagRequestType.LOCAL_EVALUATION]
+ not_modified_event = USAGE_EVENT_NAMES[FlagRequestType.LOCAL_EVALUATION_NOT_MODIFIED]
with tags_context(product=Product.FEATURE_FLAGS, feature=Feature.QUERY, team_id=team_id):
# nosemgrep: clickhouse-fstring-param-audit - bucket function and table expressions are internal fragments
@@ -106,13 +123,17 @@ def query_feature_flag_request_usage(
f"""
SELECT
{bucket_function}(timestamp) AS bucket,
- if(event = %(decide_event)s, 'remote_evaluation', 'local_evaluation') AS request_type,
+ multiIf(
+ event = %(decide_event)s, 'remote_evaluation',
+ event = %(not_modified_event)s, 'local_evaluation_not_modified',
+ 'local_evaluation'
+ ) AS request_type,
toInt64OrZero({count_expr}) AS total_count,
{sdk_breakdown_expr} AS sdk_breakdown
FROM {events_read_table(use_new)}
WHERE team_id = %(internal_team_id)s
AND distinct_id = toString(%(team_id)s)
- AND event IN (%(decide_event)s, %(local_event)s)
+ AND event IN (%(decide_event)s, %(local_event)s, %(not_modified_event)s)
AND timestamp >= %(date_from)s AND timestamp < %(date_to)s
AND has([%(validity_token)s], {token_expr})
ORDER BY bucket, request_type
@@ -125,6 +146,7 @@ def query_feature_flag_request_usage(
"validity_token": validity_token,
"decide_event": decide_event,
"local_event": local_event,
+ "not_modified_event": not_modified_event,
},
workload=Workload.ONLINE,
team_id=team_id,
diff --git a/products/feature_flags/backend/test/test_request_usage.py b/products/feature_flags/backend/test/test_request_usage.py
index b3252022883a..409ac4fa108f 100644
--- a/products/feature_flags/backend/test/test_request_usage.py
+++ b/products/feature_flags/backend/test/test_request_usage.py
@@ -24,6 +24,7 @@ class TestFeatureFlagRequestUsageQuery(ClickhouseTestMixin, TestCase):
[
("missing breakdown", "", [("other", 10)]),
("partial breakdown", '{"posthog-node": 4}', [("other", 6), ("posthog-node", 4)]),
+ ("breakdown with other", '{"other": 4, "posthog-node": 6}', [("other", 4), ("posthog-node", 6)]),
]
)
def test_attributes_unclassified_billed_requests_to_other(
@@ -90,6 +91,17 @@ class TestFeatureFlagRequestUsageQuery(ClickhouseTestMixin, TestCase):
"token": "request-usage-test-token",
},
)
+ _create_event(
+ team=analytics_team,
+ distinct_id="123",
+ event="local evaluation not modified usage",
+ timestamp=datetime(2026, 8, 20, 12, 40, tzinfo=UTC),
+ properties={
+ "count": 2,
+ "sdk_breakdown": {"posthog-python": 2},
+ "token": "request-usage-test-token",
+ },
+ )
_create_event(
team=analytics_team,
distinct_id="123",
@@ -123,10 +135,12 @@ class TestFeatureFlagRequestUsageQuery(ClickhouseTestMixin, TestCase):
time_interval="day",
)
- assert [(item.request_type.value, item.sdk, item.request_count) for item in hourly_results] == [
- ("local_evaluation", "posthog-python", 3),
- ("remote_evaluation", "other", 5),
- ("remote_evaluation", "posthog-node", 7),
+ assert [
+ (item.request_type.value, item.sdk, item.request_count, item.billing_units) for item in hourly_results
+ ] == [
+ ("local_evaluation", "posthog-python", 5, 32),
+ ("remote_evaluation", "other", 5, 5),
+ ("remote_evaluation", "posthog-node", 7, 7),
]
assert {item.bucket for item in hourly_results} == {datetime(2026, 8, 20, 12, tzinfo=UTC)}
assert {item.bucket for item in daily_results} == {datetime(2026, 8, 20, tzinfo=UTC)}
diff --git a/products/feature_flags/frontend/generated/api.schemas.ts b/products/feature_flags/frontend/generated/api.schemas.ts
index 9b312a28581f..f605a9301a92 100644
--- a/products/feature_flags/frontend/generated/api.schemas.ts
+++ b/products/feature_flags/frontend/generated/api.schemas.ts
@@ -493,7 +493,7 @@ export interface FeatureFlagRequestUsageItemApi {
sdk: string
/** Number of billable requests in this bucket. */
request_count: number
- /** Estimated billing units. Local evaluation requests count as 10 units each. */
+ /** Estimated billing units. Local evaluation requests count as 10 units each, except 304 Not Modified responses, which count as 1. */
billing_units: number
}
diff --git a/products/feature_flags/frontend/requestUsage/FeatureFlagRequestUsage.tsx b/products/feature_flags/frontend/requestUsage/FeatureFlagRequestUsage.tsx
index 4111dd46783e..6510b117aeee 100644
--- a/products/feature_flags/frontend/requestUsage/FeatureFlagRequestUsage.tsx
+++ b/products/feature_flags/frontend/requestUsage/FeatureFlagRequestUsage.tsx
@@ -90,11 +90,11 @@ export function FeatureFlagRequestUsage(): JSX.Element {
title: 'Billing units',
dataIndex: 'billingUnits',
align: 'right',
- render: (value, { remoteRequests, localRequests }) => (
+ render: (value, { remoteRequests, localBillingUnits }) => (
<span>
{(value ?? 0).toLocaleString()}{' '}
<span className="text-secondary whitespace-nowrap">
- ({remoteRequests.toLocaleString()} + 10 × {localRequests.toLocaleString()})
+ ({remoteRequests.toLocaleString()} remote + {localBillingUnits.toLocaleString()} local)
</span>
</span>
),
@@ -168,9 +168,9 @@ export function FeatureFlagRequestUsage(): JSX.Element {
<LemonBanner type="info">
This view shows billable requests, not every HTTP request. Remote requests include <code>/flags</code>{' '}
- and legacy <code>/decide</code> traffic. Local evaluation requests use 10 billing units each. Data can
- take about 30 minutes to appear, and hourly buckets reflect billing aggregation time rather than exact
- request time.{' '}
+ and legacy <code>/decide</code> traffic. Local evaluation requests use 10 billing units each, except
+ responses with no changes (304 Not Modified), which use 1. Data can take about 30 minutes to appear, and
+ hourly buckets reflect billing aggregation time rather than exact request time.{' '}
<Link to="https://posthog.com/docs/feature-flags/common-questions#billing--usage">
Learn how usage is calculated
</Link>
diff --git a/products/feature_flags/frontend/requestUsage/featureFlagRequestUsageLogic.test.ts b/products/feature_flags/frontend/requestUsage/featureFlagRequestUsageLogic.test.ts
index 4a36665b86f4..4d54fc583f64 100644
--- a/products/feature_flags/frontend/requestUsage/featureFlagRequestUsageLogic.test.ts
+++ b/products/feature_flags/frontend/requestUsage/featureFlagRequestUsageLogic.test.ts
@@ -39,20 +39,22 @@ describe('featureFlagRequestUsageLogic', () => {
bucket: '2026-08-20T00:00:00Z',
request_type: 'local_evaluation',
sdk: 'posthog-ruby',
- request_count: 3,
- billing_units: 30,
+ // Three full responses at 10 units and two 304s at 1 unit.
+ request_count: 5,
+ billing_units: 32,
},
],
})
await expectLogic(logic).toMatchValues({
- totalBillingUnits: 50,
+ totalBillingUnits: 52,
sdkTotals: [
{
sdk: 'posthog-ruby',
remoteRequests: 20,
- localRequests: 3,
- billingUnits: 50,
+ localRequests: 5,
+ localBillingUnits: 32,
+ billingUnits: 52,
billingUnitsShare: 100,
},
],
@@ -67,7 +69,7 @@ describe('featureFlagRequestUsageLogic', () => {
id: expect.any(Number),
label: 'posthog-ruby (local)',
dates: ['2026-08-20T00:00:00.000Z'],
- data: [3],
+ data: [5],
},
],
})
diff --git a/products/feature_flags/frontend/requestUsage/featureFlagRequestUsageLogic.ts b/products/feature_flags/frontend/requestUsage/featureFlagRequestUsageLogic.ts
index 03bc5466b6d0..caafa32d9bde 100644
--- a/products/feature_flags/frontend/requestUsage/featureFlagRequestUsageLogic.ts
+++ b/products/feature_flags/frontend/requestUsage/featureFlagRequestUsageLogic.ts
@@ -28,6 +28,7 @@ export interface FeatureFlagRequestUsageSdkTotal {
sdk: string
remoteRequests: number
localRequests: number
+ localBillingUnits: number
billingUnits: number
billingUnitsShare: number
}
@@ -299,6 +300,7 @@ export const featureFlagRequestUsageLogic = kea<featureFlagRequestUsageLogicType
sdk: item.sdk,
remoteRequests: 0,
localRequests: 0,
+ localBillingUnits: 0,
billingUnits: 0,
billingUnitsShare: 0,
}
@@ -306,6 +308,7 @@ export const featureFlagRequestUsageLogic = kea<featureFlagRequestUsageLogicType
current.remoteRequests += item.request_count
} else {
current.localRequests += item.request_count
+ current.localBillingUnits += item.billing_units
}
current.billingUnits += item.billing_units
totals.set(item.sdk, current)
diff --git a/rust/feature-flags/src/api/flag_definitions.rs b/rust/feature-flags/src/api/flag_definitions.rs
index 5ca325927428..0de719ec0506 100644
--- a/rust/feature-flags/src/api/flag_definitions.rs
+++ b/rust/feature-flags/src/api/flag_definitions.rs
@@ -10,7 +10,7 @@ use crate::{
metrics::consts::{
FLAG_DEFINITIONS_AUTH_COUNTER, FLAG_DEFINITIONS_CACHE_HIT_COUNTER,
FLAG_DEFINITIONS_CACHE_MISS_COUNTER, FLAG_DEFINITIONS_ETAG_COUNTER,
- FLAG_DEFINITIONS_REBUILD_REQUESTED_COUNTER,
+ FLAG_DEFINITIONS_NOT_MODIFIED_BILLING_COUNTER, FLAG_DEFINITIONS_REBUILD_REQUESTED_COUNTER,
},
router::State as AppState,
team::team_models::Team,
@@ -31,7 +31,7 @@ use serde_json::Value;
use sqlx::PgPool;
use std::collections::HashSet;
use std::sync::Arc;
-use std::time::{SystemTime, UNIX_EPOCH};
+use std::time::{Duration, SystemTime, UNIX_EPOCH};
use tracing::{info, warn};
const ALLOWLIST_TTL_SECS: u64 = 60;
@@ -125,6 +125,93 @@ pub async fn fetch_allowlist_from_db(pool: &PgPool) -> Result<Option<HashSet<Tea
/// This is returned as raw JSON from cache to avoid deserialization overhead
pub type FlagDefinitionsResponse = Value;
+/// Bound on the payload read a 304 makes when the memo has no entry for the
+/// current ETag. Matches the hypercache Redis timeout. A slow read delays the
+/// 304 by at most this long, and then the poll goes unbilled and the next one
+/// retries.
+const BILLABLE_LOOKUP_TIMEOUT: Duration = Duration::from_millis(500);
+
+/// How long a failed, slow, or non-Redis payload read is remembered. While
+/// the payload tier is degraded this caps the retries at one per pod per
+/// team per interval instead of one per poll, and every poll in between
+/// stays unbilled.
+const BILLABLE_MEMO_FAILURE_TTL: Duration = Duration::from_secs(30);
+
+type BillableMemoKey = (i32, String);
+
+/// A memoized answer outlives a mismatch between the ETag and the payload it
+/// was computed from for at most `answer_ttl`: the writer can publish a new
+/// version between the ETag read and the payload read of one request. A
+/// failure to answer is kept only briefly so the next poll retries soon.
+struct BillableMemoExpiry {
+ answer_ttl: Duration,
+ failure_ttl: Duration,
+}
+
+impl moka::Expiry<BillableMemoKey, Option<bool>> for BillableMemoExpiry {
+ fn expire_after_create(
+ &self,
+ _key: &BillableMemoKey,
+ value: &Option<bool>,
+ _created_at: std::time::Instant,
+ ) -> Option<Duration> {
+ Some(match value {
+ Some(_) => self.answer_ttl,
+ None => self.failure_ttl,
+ })
+ }
+}
+
+/// Memo of whether the definitions behind a `(team, etag)` contain a billable
+/// flag, so a 304 for definitions that hold only survey or product tour flags
+/// is not billed, without reading the payload the 304 deliberately skips.
+#[derive(Clone)]
+pub struct DefinitionsBillableCache {
+ entries: moka::future::Cache<BillableMemoKey, Option<bool>>,
+}
+
+impl DefinitionsBillableCache {
+ pub fn new(max_capacity: u64, answer_ttl: Duration) -> Self {
+ Self::with_failure_ttl(max_capacity, answer_ttl, BILLABLE_MEMO_FAILURE_TTL)
+ }
+
+ fn with_failure_ttl(max_capacity: u64, answer_ttl: Duration, failure_ttl: Duration) -> Self {
+ Self {
+ entries: moka::future::Cache::builder()
+ .max_capacity(max_capacity)
+ .expire_after(BillableMemoExpiry {
+ answer_ttl,
+ failure_ttl,
+ })
+ .build(),
+ }
+ }
+
+ pub async fn insert(&self, team_id: i32, etag: &str, billable: bool) {
+ self.entries
+ .insert((team_id, etag.to_string()), Some(billable))
+ .await;
+ }
+
+ /// Returns the memoized status, or runs `load` once for all concurrent
+ /// callers with the same key and stores the result. `None` when the load
+ /// returns `None`, fails, or exceeds the bound; that `None` is kept for
+ /// `BILLABLE_MEMO_FAILURE_TTL` before a later call retries.
+ pub async fn get_or_load<F>(&self, team_id: i32, etag: &str, load: F) -> Option<bool>
+ where
+ F: std::future::Future<Output = Option<bool>>,
+ {
+ self.entries
+ .get_with((team_id, etag.to_string()), async {
+ tokio::time::timeout(BILLABLE_LOOKUP_TIMEOUT, load)
+ .await
+ .ok()
+ .flatten()
+ })
+ .await
+ }
+}
+
/// Query parameters for the flag definitions endpoint
#[derive(Debug, Deserialize, Serialize)]
pub struct FlagDefinitionsQueryParams {
@@ -257,6 +344,15 @@ pub async fn flags_definitions(
&[("result".to_string(), "hit".to_string())],
1,
);
+ if !*state.config.skip_writes
+ && is_billable_for_etag(&state, &team_key, team.id, current_val).await
+ {
+ state.billing_aggregator.record(
+ team.id,
+ FlagRequestType::FlagDefinitionsNotModified,
+ Some(Library::from_headers(&headers)),
+ );
+ }
return Ok(not_modified_response(current_val));
}
}
@@ -281,12 +377,20 @@ pub async fn flags_definitions(
);
// Retrieve cached response from HyperCache (always with cohorts)
- let cached_response = get_from_cache(&state, &team_key, team.id).await?;
+ let (cached_response, cache_source) = get_from_cache(&state, &team_key, team.id).await?;
// Record usage for billing, filtering out non-billable flags (surveys, product tours).
- // Placed after the ETag/304 path intentionally: 304 responses skip billing,
- // matching Django's /local_evaluation behavior.
- if !*state.config.skip_writes && has_billable_flags(&cached_response) {
+ let billable = has_billable_flags(&cached_response);
+ if let (Some(etag), Some(billable)) = (
+ &current_etag,
+ billable_for_etag_from(&cached_response, cache_source),
+ ) {
+ state
+ .definitions_billable_cache
+ .insert(team.id, etag, billable)
+ .await;
+ }
+ if !*state.config.skip_writes && billable {
let library = Library::from_headers(&headers);
state
.billing_aggregator
@@ -432,6 +536,49 @@ fn cache_hit_metric_source_label(source: &CacheSource) -> &'static str {
}
}
+/// Only Redis holds the payload the ETag was published with. An S3 fallback
+/// may be another version, so it decides nothing.
+fn billable_for_etag_from(response: &FlagDefinitionsResponse, source: CacheSource) -> Option<bool> {
+ matches!(source, CacheSource::Redis).then(|| has_billable_flags(response))
+}
+
+/// Reads the payload at most once per pod per ETag per memo TTL. An
+/// unreadable, slow, or non-Redis payload is not memoized as an answer and
+/// counts as not billable, because over-billing is the worse failure.
+async fn is_billable_for_etag(
+ state: &AppState,
+ team_key: &KeyType,
+ team_id: i32,
+ etag: &str,
+) -> bool {
+ let billable = state
+ .definitions_billable_cache
+ .get_or_load(team_id, etag, async {
+ // Reads the hypercache directly: this lookup serves a 304, so it must not count as
+ // a definitions cache hit or queue a rebuild the way a full response does. It reads
+ // Redis only, because an S3 copy cannot decide (see `billable_for_etag_from`), so a
+ // fallback read would only add latency and S3 load. A Redis miss is not an answer.
+ let response = state
+ .flags_with_cohorts_hypercache_reader
+ .get_typed_from_redis::<FlagDefinitionsResponse>(team_key)
+ .await
+ .ok()?;
+ Some(response.is_some_and(|response| has_billable_flags(&response)))
+ })
+ .await;
+ let outcome = match billable {
+ Some(true) => "billable",
+ Some(false) => "not_billable",
+ None => "unknown",
+ };
+ inc(
+ FLAG_DEFINITIONS_NOT_MODIFIED_BILLING_COUNTER,
+ &[("outcome".to_string(), outcome.to_string())],
+ 1,
+ );
+ billable.unwrap_or(false)
+}
+
/// Retrieves the cached response using the pre-initialized HyperCacheReader
///
/// Always uses the cache with cohorts included to match Django's behavior and ensure
@@ -444,7 +591,7 @@ async fn get_from_cache(
state: &AppState,
team_key: &KeyType,
team_id: i32,
-) -> Result<FlagDefinitionsResponse, FlagError> {
+) -> Result<(FlagDefinitionsResponse, CacheSource), FlagError> {
let result = state
.flags_with_cohorts_hypercache_reader
.get_with_source(team_key)
@@ -472,7 +619,7 @@ async fn get_from_cache(
) {
enqueue_flag_definitions_rebuild(state, team_id, REBUILD_TRIGGER_S3_HIT);
}
- Ok(data)
+ Ok((data, source))
}
Err(e) => {
let reason = match &e {
@@ -720,6 +867,67 @@ mod tests {
}
}
+ #[tokio::test]
+ async fn test_billable_cache_memoizes_a_successful_load() {
+ let cache = DefinitionsBillableCache::new(10, Duration::from_secs(3600));
+ assert_eq!(
+ cache.get_or_load(1, "etag", async { Some(true) }).await,
+ Some(true)
+ );
+ // A later load is never run once the key is memoized.
+ assert_eq!(
+ cache.get_or_load(1, "etag", async { None }).await,
+ Some(true)
+ );
+ // A new etag for the same team is a fresh key.
+ assert_eq!(
+ cache.get_or_load(1, "etag2", async { Some(false) }).await,
+ Some(false)
+ );
+ }
+
+ #[tokio::test]
+ async fn test_billable_cache_memoizes_a_failed_load_until_the_failure_ttl() {
+ let cache = DefinitionsBillableCache::with_failure_ttl(
+ 10,
+ Duration::from_secs(3600),
+ Duration::from_millis(500),
+ );
+ assert_eq!(cache.get_or_load(1, "etag", async { None }).await, None);
+ assert_eq!(
+ cache.get_or_load(1, "etag", async { Some(true) }).await,
+ None
+ );
+ // moka expires entries on its own clock, so a paused tokio clock would not age this one.
+ tokio::time::sleep(Duration::from_secs(1)).await;
+ assert_eq!(
+ cache.get_or_load(1, "etag", async { Some(true) }).await,
+ Some(true)
+ );
+ }
+
+ #[tokio::test(start_paused = true)]
+ async fn test_billable_cache_bounds_a_slow_load() {
+ let cache = DefinitionsBillableCache::new(10, Duration::from_secs(3600));
+ assert_eq!(
+ cache.get_or_load(1, "etag", std::future::pending()).await,
+ None
+ );
+ }
+
+ #[rstest]
+ #[case::redis(CacheSource::Redis, Some(true))]
+ #[case::s3(CacheSource::S3, None)]
+ #[case::s3_after_redis_error(CacheSource::S3AfterRedisError, None)]
+ #[case::fallback(CacheSource::Fallback, None)]
+ fn test_billable_for_etag_from_only_trusts_redis(
+ #[case] source: CacheSource,
+ #[case] expected: Option<bool>,
+ ) {
+ let response = json!({"flags": [{"key": "billable-flag"}]});
+ assert_eq!(billable_for_etag_from(&response, source), expected);
+ }
+
#[test]
fn test_extract_etag_from_header_weak() {
let val = axum::http::HeaderValue::from_static("W/\"a1b2c3d4e5f6g7h8\"");
diff --git a/rust/feature-flags/src/billing/aggregator.rs b/rust/feature-flags/src/billing/aggregator.rs
index 71b2f004a5e6..85a273d82faa 100644
--- a/rust/feature-flags/src/billing/aggregator.rs
+++ b/rust/feature-flags/src/billing/aggregator.rs
@@ -78,6 +78,7 @@ use std::time::Duration;
use common_metrics::{gauge, histogram, inc};
use common_redis::{Client as RedisClient, CustomRedisError, PipelineCommand};
use rand::Rng;
+use strum::IntoEnumIterator;
use tokio::sync::Notify;
use tokio::task::JoinHandle;
@@ -328,6 +329,7 @@ struct Inner {
/// instead of an `inc()` call that allocates label strings every time.
record_count_decide: AtomicU64,
record_count_flag_definitions: AtomicU64,
+ record_count_flag_definitions_not_modified: AtomicU64,
shutdown_signal: Notify,
usage_reporter: Option<Arc<UsageReporter>>,
}
@@ -348,10 +350,21 @@ impl Inner {
last_successful_flush_epoch_ms: AtomicU64::new(0),
record_count_decide: AtomicU64::new(0),
record_count_flag_definitions: AtomicU64::new(0),
+ record_count_flag_definitions_not_modified: AtomicU64::new(0),
shutdown_signal: Notify::new(),
})
}
+ fn record_counter(&self, request_type: FlagRequestType) -> &AtomicU64 {
+ match request_type {
+ FlagRequestType::Decide => &self.record_count_decide,
+ FlagRequestType::FlagDefinitions => &self.record_count_flag_definitions,
+ FlagRequestType::FlagDefinitionsNotModified => {
+ &self.record_count_flag_definitions_not_modified
+ }
+ }
+ }
+
/// Sum any remaining entries in `pending` plus any drained-but-uncredited
/// counts from an interrupted flush, and emit them as shutdown drops.
fn record_shutdown_drops(&self) {
@@ -434,11 +447,9 @@ impl BillingAggregator {
// Bump the per-request-type atomic counter; the flusher emits the
// `FLAGS_BILLING_RECORDS` metric in batches at flush time so the
// hot path doesn't pay the per-call label clone in `inc()`.
- match request_type {
- FlagRequestType::Decide => &self.inner.record_count_decide,
- FlagRequestType::FlagDefinitions => &self.inner.record_count_flag_definitions,
- }
- .fetch_add(1, Ordering::Relaxed);
+ self.inner
+ .record_counter(request_type)
+ .fetch_add(1, Ordering::Relaxed);
let key = AggregationKey {
team_id,
@@ -764,23 +775,17 @@ async fn flush_once(inner: &Arc<Inner>, policy: FlushPolicy) {
// FLAGS_BILLING_RECORDS. Doing this here instead of in `record()`
// keeps the hot path free of the per-call `apply_label_filter`
// allocation that `inc()` triggers.
- let decide_records = inner.record_count_decide.swap(0, Ordering::Relaxed);
- if decide_records > 0 {
- inc(
- FLAGS_BILLING_RECORDS,
- &record_labels_for(FlagRequestType::Decide),
- decide_records,
- );
- }
- let flag_def_records = inner
- .record_count_flag_definitions
- .swap(0, Ordering::Relaxed);
- if flag_def_records > 0 {
- inc(
- FLAGS_BILLING_RECORDS,
- &record_labels_for(FlagRequestType::FlagDefinitions),
- flag_def_records,
- );
+ for request_type in FlagRequestType::iter() {
+ let records = inner
+ .record_counter(request_type)
+ .swap(0, Ordering::Relaxed);
+ if records > 0 {
+ inc(
+ FLAGS_BILLING_RECORDS,
+ &record_labels_for(request_type),
+ records,
+ );
+ }
}
// Swap pending and zero `pending_total` under the same lock so a
@@ -1495,6 +1500,7 @@ mod tests {
Some(Library::PosthogJs),
);
agg.record(7, FlagRequestType::Decide, None);
+ agg.record(9, FlagRequestType::FlagDefinitionsNotModified, None);
let bucket = current_bucket();
flush_once(&agg.inner, FlushPolicy::BailOnError).await;
@@ -1504,10 +1510,13 @@ mod tests {
let expected_sdk_42_def =
format!("posthog:local_evaluation_requests:sdk:42:posthog-js:{bucket}");
let expected_team_7_decide = format!("posthog:decide_requests:7:{bucket}");
- assert_eq!(calls.len(), 3);
+ let expected_team_9_not_modified =
+ format!("posthog:local_evaluation_not_modified_requests:9:{bucket}");
+ assert_eq!(calls.len(), 4);
assert!(calls.contains(&(expected_team_42_def, 1)));
assert!(calls.contains(&(expected_sdk_42_def, 1)));
assert!(calls.contains(&(expected_team_7_decide, 1)));
+ assert!(calls.contains(&(expected_team_9_not_modified, 1)));
// Belt-and-braces: the aggregator must never write `:shadow`-suffixed
// keys. A regression here would mean we're writing into the
diff --git a/rust/feature-flags/src/billing/usage_reporter.rs b/rust/feature-flags/src/billing/usage_reporter.rs
index cdacba264453..4f9db6c02ec3 100644
--- a/rust/feature-flags/src/billing/usage_reporter.rs
+++ b/rust/feature-flags/src/billing/usage_reporter.rs
@@ -280,13 +280,18 @@ fn dedup(teams: impl Iterator<Item = i64>) -> Vec<i64> {
teams
}
-/// The two billable request types are priced apart — the nightly report weighs one local
-/// evaluation as ten decide requests — so they cannot share a key. The weighting itself stays
-/// out of here: a producer reports what happened, and pricing is applied downstream.
+/// Each billable request type has its own key because the nightly report prices them apart:
+/// one local evaluation costs ten decide requests, and one 304 costs one. A 304 does not use
+/// the decide key even at the same price, because each key feeds its own usage counter and a
+/// shared key would count the 304 twice. The weighting itself stays out of here: a producer
+/// reports what happened, and pricing is applied downstream.
fn usage_key(request_type: FlagRequestType) -> &'static str {
match request_type {
FlagRequestType::Decide => "feature_flag_requests",
FlagRequestType::FlagDefinitions => "feature_flag_local_evaluation_requests",
+ FlagRequestType::FlagDefinitionsNotModified => {
+ "feature_flag_local_evaluation_not_modified_requests"
+ }
}
}
@@ -339,6 +344,13 @@ mod tests {
},
4,
),
+ (
+ AggregationKey {
+ request_type: FlagRequestType::FlagDefinitionsNotModified,
+ ..key(1, None)
+ },
+ 5,
+ ),
];
let records = build_records(&entries, &TeamIdCollection::All, 1);
@@ -349,7 +361,8 @@ mod tests {
.collect::<Vec<_>>(),
vec![
("feature_flag_requests", 3),
- ("feature_flag_local_evaluation_requests", 4)
+ ("feature_flag_local_evaluation_requests", 4),
+ ("feature_flag_local_evaluation_not_modified_requests", 5)
]
);
}
diff --git a/rust/feature-flags/src/config.rs b/rust/feature-flags/src/config.rs
index 5dd3d58c3ff8..7375ce10b7d0 100644
--- a/rust/feature-flags/src/config.rs
+++ b/rust/feature-flags/src/config.rs
@@ -984,6 +984,15 @@ pub struct Config {
#[envconfig(from = "TEAM_NEGATIVE_CACHE_TTL_SECONDS", default = "30")]
pub team_negative_cache_ttl_seconds: u64,
+ // In-memory memo of whether a team's flag definitions, keyed by ETag, hold a
+ // billable flag, so a /flags/definitions 304 is billed without a payload read.
+ // The TTL bounds how long a wrong answer can last, so it is tunable without a deploy.
+ #[envconfig(from = "DEFINITIONS_BILLABLE_CACHE_CAPACITY", default = "100000")]
+ pub definitions_billable_cache_capacity: u64,
+
+ #[envconfig(from = "DEFINITIONS_BILLABLE_CACHE_TTL_SECONDS", default = "3600")]
+ pub definitions_billable_cache_ttl_seconds: u64,
+
// Write an S3 hit back into Redis so the next reader for that key is served by Redis
// instead of paying another S3 read. Applies to the team metadata and remote config
// hypercaches, which have no in-process cache in front of them. 0 disables.
@@ -1407,6 +1416,8 @@ impl Config {
thread_pool_cores: 0,
team_negative_cache_capacity: 10_000,
team_negative_cache_ttl_seconds: 30,
+ definitions_billable_cache_capacity: 100_000,
+ definitions_billable_cache_ttl_seconds: 3_600,
hypercache_read_repair_ttl_seconds: 600,
skip_pg_team_fallback: FlexBool(false),
accept_team_secret_tokens: FlexBool(true),
diff --git a/rust/feature-flags/src/flags/flag_analytics.rs b/rust/feature-flags/src/flags/flag_analytics.rs
index fc8ccb31063d..128b8fc10ec3 100644
--- a/rust/feature-flags/src/flags/flag_analytics.rs
+++ b/rust/feature-flags/src/flags/flag_analytics.rs
@@ -24,6 +24,9 @@ pub fn get_team_request_key(team_id: i32, request_type: FlagRequestType) -> Stri
match request_type {
FlagRequestType::Decide => format!("posthog:decide_requests:{team_id}"),
FlagRequestType::FlagDefinitions => format!("posthog:local_evaluation_requests:{team_id}"),
+ FlagRequestType::FlagDefinitionsNotModified => {
+ format!("posthog:local_evaluation_not_modified_requests:{team_id}")
+ }
}
}
@@ -37,6 +40,9 @@ pub fn get_team_request_library_key(
FlagRequestType::FlagDefinitions => {
format!("posthog:local_evaluation_requests:sdk:{team_id}:{library}")
}
+ FlagRequestType::FlagDefinitionsNotModified => {
+ format!("posthog:local_evaluation_not_modified_requests:sdk:{team_id}:{library}")
+ }
}
}
@@ -54,6 +60,10 @@ mod tests {
get_team_request_key(456, FlagRequestType::FlagDefinitions),
"posthog:local_evaluation_requests:456"
);
+ assert_eq!(
+ get_team_request_key(456, FlagRequestType::FlagDefinitionsNotModified),
+ "posthog:local_evaluation_not_modified_requests:456"
+ );
}
#[tokio::test]
@@ -83,6 +93,14 @@ mod tests {
),
"posthog:local_evaluation_requests:sdk:101:posthog-elixir"
);
+ assert_eq!(
+ get_team_request_library_key(
+ 101,
+ FlagRequestType::FlagDefinitionsNotModified,
+ Library::PosthogElixir
+ ),
+ "posthog:local_evaluation_not_modified_requests:sdk:101:posthog-elixir"
+ );
// Test Other variant
assert_eq!(
get_team_request_library_key(102, FlagRequestType::Decide, Library::Other),
diff --git a/rust/feature-flags/src/flags/flag_request.rs b/rust/feature-flags/src/flags/flag_request.rs
index ddc8aa74c79b..6b1eac036b38 100644
--- a/rust/feature-flags/src/flags/flag_request.rs
+++ b/rust/feature-flags/src/flags/flag_request.rs
@@ -35,10 +35,12 @@ where
}))
}
-#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, strum::EnumIter)]
pub enum FlagRequestType {
Decide,
FlagDefinitions,
+ /// A `/flags/definitions` poll answered with 304, billed at the `/flags` rate.
+ FlagDefinitionsNotModified,
}
impl FlagRequestType {
@@ -46,6 +48,7 @@ impl FlagRequestType {
match self {
FlagRequestType::Decide => "decide",
FlagRequestType::FlagDefinitions => "flag_definitions",
+ FlagRequestType::FlagDefinitionsNotModified => "flag_definitions_not_modified",
}
}
}
diff --git a/rust/feature-flags/src/metrics/consts.rs b/rust/feature-flags/src/metrics/consts.rs
index 46ac7d9d080b..f5c101c9da54 100644
--- a/rust/feature-flags/src/metrics/consts.rs
+++ b/rust/feature-flags/src/metrics/consts.rs
@@ -383,6 +383,12 @@ pub const FLAG_DEFINITIONS_CACHE_MISS_COUNTER: &str = "flags_flag_definitions_ca
// redis_error = the etag read failed or the stored value did not decode)
pub const FLAG_DEFINITIONS_ETAG_COUNTER: &str = "flags_flag_definitions_etag_total";
+// Billing decision for a /flags/definitions 304. Labels: outcome (billable, not_billable,
+// unknown = the payload was unreadable, slow, or not served from Redis, so the poll went
+// unbilled). Compare with the etag `hit` counter to see how many 304s went unbilled.
+pub const FLAG_DEFINITIONS_NOT_MODIFIED_BILLING_COUNTER: &str =
+ "flags_flag_definitions_not_modified_billing_total";
+
// Per-pod resolved cluster for the /flags/definitions reader: 1 on the dedicated flags Redis,
// 0 on the shared one. Every emission carries a `reason` label, so
// `{reason="no_dedicated_client"}` separates a pod that wanted the dedicated cluster and could
diff --git a/rust/feature-flags/src/router.rs b/rust/feature-flags/src/router.rs
index 72ef0e5c8d5b..4be2d6e76d0c 100644
--- a/rust/feature-flags/src/router.rs
+++ b/rust/feature-flags/src/router.rs
@@ -42,6 +42,7 @@ use crate::{
body_read_metrics::{record_body_read, MAX_FLAGS_BODY_BYTES},
concurrency_metrics::{record_concurrency_enter, record_concurrency_wait},
endpoint, flag_definitions,
+ flag_definitions::DefinitionsBillableCache,
flag_definitions_rate_limiter::{FlagDefinitionsRateLimiter, RemoteConfigRateLimiter},
flags_rate_limiter::{FlagsRateLimiter, IpRateLimiter},
remote_config,
@@ -115,6 +116,10 @@ pub struct State {
/// Pre-initialized HyperCacheReader for feature flags with cohorts (flags_with_cohorts.json)
/// Used by the /flags/definitions endpoint
pub flags_with_cohorts_hypercache_reader: Arc<HyperCacheReader>,
+ /// Billable status of each team's current flag definitions, keyed by ETag, so a 304 on
+ /// /flags/definitions for survey-only or product-tour-only definitions goes unbilled
+ /// without a payload read on every poll
+ pub definitions_billable_cache: DefinitionsBillableCache,
/// Pre-initialized HyperCacheReader for team metadata (full_metadata.json)
/// Uses token-based lookup instead of team_id
pub team_hypercache_reader: Arc<HyperCacheReader>,
@@ -434,6 +439,10 @@ where
config_hypercache_reader,
rayon_dispatcher,
team_negative_cache,
+ definitions_billable_cache: DefinitionsBillableCache::new(
+ config.definitions_billable_cache_capacity,
+ std::time::Duration::from_secs(config.definitions_billable_cache_ttl_seconds),
+ ),
cohort_membership_provider,
auth_token_cache,
billing_aggregator,
diff --git a/rust/feature-flags/src/utils/test_utils.rs b/rust/feature-flags/src/utils/test_utils.rs
index 7c486bf4a33c..0beb39cbf298 100644
--- a/rust/feature-flags/src/utils/test_utils.rs
+++ b/rust/feature-flags/src/utils/test_utils.rs
@@ -2078,6 +2078,19 @@ impl TestContext {
.await
}
+ /// Populate cache for a team with the given flags payload and an ETag.
+ pub async fn populate_cache_for_team_with_flags_and_etag(
+ &self,
+ team_id: i32,
+ flags_data: serde_json::Value,
+ etag: &str,
+ ) -> Result<(), Error> {
+ self.populate_cache_for_team_with_flags(team_id, flags_data)
+ .await?;
+ let redis_client = setup_redis_client(Some(self.config.redis_url.clone())).await;
+ self.set_etag_for_team(redis_client, team_id, etag).await
+ }
+
/// Populate cache for a team and store an ETag alongside it, on the given Redis.
/// The ETag is stored at `{cache_key}:etag` using pickle serialization,
/// matching Django's HyperCache behavior.
@@ -2089,7 +2102,15 @@ impl TestContext {
) -> Result<(), Error> {
self.populate_flag_definitions_cache(redis_client.clone(), team_id)
.await?;
+ self.set_etag_for_team(redis_client, team_id, etag).await
+ }
+ async fn set_etag_for_team(
+ &self,
+ redis_client: Arc<dyn RedisClientTrait + Send + Sync>,
+ team_id: i32,
+ etag: &str,
+ ) -> Result<(), Error> {
let etag_key =
format!("posthog:1:cache/teams/{team_id}/feature_flags/flags_with_cohorts.json:etag");
let pickled_etag =
diff --git a/rust/feature-flags/tests/test_flag_definitions.rs b/rust/feature-flags/tests/test_flag_definitions.rs
index d610e965a9f7..18e2b635ddc0 100644
--- a/rust/feature-flags/tests/test_flag_definitions.rs
+++ b/rust/feature-flags/tests/test_flag_definitions.rs
@@ -3199,6 +3199,276 @@ async fn test_flag_definitions_billing_counter(#[case] skip_writes: bool) {
}
}
+/// A 304 lands on the not-modified counter and leaves the full local evaluation
+/// counter untouched. It stays unbilled for definitions that hold only survey or
+/// product tour flags, and when `skip_writes` is on.
+#[rstest::rstest]
+#[case::billable_flag("billable-flag", false, true)]
+#[case::survey_only("survey-targeting-only", false, false)]
+#[case::skip_writes("billable-flag", true, false)]
+#[tokio::test]
+async fn test_flag_definitions_304_records_not_modified_billing_counter(
+ #[case] flag_key: &str,
+ #[case] skip_writes: bool,
+ #[case] expect_billed: bool,
+) {
+ use feature_flags::config::FlexBool;
+ use feature_flags::flags::flag_analytics::{current_bucket, get_team_request_key};
+ use feature_flags::flags::flag_request::FlagRequestType;
+ use feature_flags::utils::test_utils::{setup_redis_client, TestContext};
+ use serde_json::json;
+
+ let mut config = feature_flags::config::Config::default_test_config();
+ config.skip_writes = FlexBool(skip_writes);
+ let context = TestContext::new(Some(&config)).await;
+ let (team, secret_token, _) = context
+ .create_team_with_secret_token(None, None, None)
+ .await
+ .unwrap();
+
+ let etag_value = "billed304etag0001";
+ context
+ .populate_cache_for_team_with_flags_and_etag(
+ team.id,
+ json!({
+ "flags": [{"key": flag_key, "active": true}],
+ "group_type_mapping": {},
+ "cohorts": {},
+ }),
+ etag_value,
+ )
+ .await
+ .unwrap();
+
+ let redis = setup_redis_client(Some(config.redis_url.clone())).await;
+ let not_modified_key =
+ get_team_request_key(team.id, FlagRequestType::FlagDefinitionsNotModified);
+ let full_response_key = get_team_request_key(team.id, FlagRequestType::FlagDefinitions);
+ redis.del(not_modified_key.clone()).await.unwrap();
+ redis.del(full_response_key.clone()).await.unwrap();
+
+ let server = common::ServerHandle::for_config(config).await;
+ let http = reqwest::Client::new();
+
+ let bucket_before = current_bucket();
+ let response = http
+ .get(format!(
+ "http://{}/flags/definitions?token={}",
+ server.addr, team.api_token
+ ))
+ .header("Authorization", format!("Bearer {secret_token}"))
+ .header("If-None-Match", format!("W/\"{etag_value}\""))
+ .send()
+ .await
+ .unwrap();
+ assert_eq!(response.status(), 304);
+ let bucket_after = current_bucket();
+
+ if expect_billed {
+ let counter = common::poll_for_billing_counter_across_buckets(
+ &redis,
+ &not_modified_key,
+ bucket_before,
+ bucket_after,
+ )
+ .await;
+ assert_eq!(
+ counter, "1",
+ "304 should be recorded once on the not-modified billing counter"
+ );
+ } else {
+ tokio::time::sleep(std::time::Duration::from_millis(500)).await;
+ for bucket in bucket_before..=bucket_after {
+ let counter = redis
+ .hget(not_modified_key.clone(), bucket.to_string())
+ .await;
+ assert!(
+ counter.is_err(),
+ "304 for non-billable flags must not be billed, got {counter:?}"
+ );
+ }
+ }
+ for bucket in bucket_before..=bucket_after {
+ let full = redis
+ .hget(full_response_key.clone(), bucket.to_string())
+ .await;
+ assert!(
+ full.is_err(),
+ "304 must not increment the full local evaluation counter, got {full:?}"
+ );
+ }
+}
+
+/// Only a payload served from Redis decides whether a 304 is billed. With the
+/// ETag in Redis but the payload only in S3, neither the full response that
+/// precedes the poll nor the poll's own read may memoize S3's answer, so the
+/// 304 stays unbilled even though S3 says the definitions are billable.
+#[tokio::test]
+async fn test_flag_definitions_304_is_not_billed_from_an_s3_payload() {
+ use feature_flags::flags::flag_analytics::{current_bucket, get_team_request_key};
+ use feature_flags::flags::flag_request::FlagRequestType;
+ use feature_flags::utils::test_utils::{setup_redis_client, static_s3_client, TestContext};
+ use serde_json::json;
+
+ let config = feature_flags::config::Config::default_test_config();
+ let context = TestContext::new(Some(&config)).await;
+ let (team, secret_token, _) = context
+ .create_team_with_secret_token(None, None, None)
+ .await
+ .unwrap();
+
+ let payload = json!({
+ "flags": [{"key": "billable-flag", "active": true}],
+ "group_type_mapping": {},
+ "cohorts": {},
+ });
+ let etag_value = "s3only304etag0001";
+ context
+ .populate_cache_for_team_with_flags_and_etag(team.id, payload.clone(), etag_value)
+ .await
+ .unwrap();
+
+ let redis = setup_redis_client(Some(config.redis_url.clone())).await;
+ // Leave the ETag in Redis and the payload only in S3.
+ redis
+ .del(format!(
+ "posthog:1:cache/teams/{}/feature_flags/flags_with_cohorts.json",
+ team.id
+ ))
+ .await
+ .unwrap();
+ let not_modified_key =
+ get_team_request_key(team.id, FlagRequestType::FlagDefinitionsNotModified);
+ redis.del(not_modified_key.clone()).await.unwrap();
+
+ let server = common::ServerHandle::for_config_with_s3(
+ config,
+ // S3 serves the same billable payload for every key, standing in for a copy
+ // that may lag the ETag Redis holds by a version.
+ Some(static_s3_client(&payload.to_string())),
+ )
+ .await;
+ let http = reqwest::Client::new();
+ let url = format!(
+ "http://{}/flags/definitions?token={}",
+ server.addr, team.api_token
+ );
+
+ let bucket_before = current_bucket();
+ let response = http
+ .get(&url)
+ .header("Authorization", format!("Bearer {secret_token}"))
+ .send()
+ .await
+ .unwrap();
+ assert_eq!(response.status(), 200);
+
+ let response = http
+ .get(&url)
+ .header("Authorization", format!("Bearer {secret_token}"))
+ .header("If-None-Match", format!("W/\"{etag_value}\""))
+ .send()
+ .await
+ .unwrap();
+ assert_eq!(response.status(), 304);
+ let bucket_after = current_bucket();
+
+ tokio::time::sleep(std::time::Duration::from_millis(500)).await;
+ for bucket in bucket_before..=bucket_after {
+ let counter = redis
+ .hget(not_modified_key.clone(), bucket.to_string())
+ .await;
+ assert!(
+ counter.is_err(),
+ "304 must not be billed from an S3 payload, got {counter:?}"
+ );
+ }
+}
+
+/// A full response served from Redis memoizes whether its ETag is billable, so
+/// a later 304 for that ETag is billed from the memo even after the Redis
+/// payload is gone and the 304's own read would find nothing.
+#[tokio::test]
+async fn test_flag_definitions_304_is_billed_from_the_full_response_memo() {
+ use feature_flags::flags::flag_analytics::{current_bucket, get_team_request_key};
+ use feature_flags::flags::flag_request::FlagRequestType;
+ use feature_flags::utils::test_utils::{setup_redis_client, TestContext};
+ use serde_json::json;
+
+ let config = feature_flags::config::Config::default_test_config();
+ let context = TestContext::new(Some(&config)).await;
+ let (team, secret_token, _) = context
+ .create_team_with_secret_token(None, None, None)
+ .await
+ .unwrap();
+
+ let etag_value = "memo304etag00001";
+ context
+ .populate_cache_for_team_with_flags_and_etag(
+ team.id,
+ json!({
+ "flags": [{"key": "billable-flag", "active": true}],
+ "group_type_mapping": {},
+ "cohorts": {},
+ }),
+ etag_value,
+ )
+ .await
+ .unwrap();
+
+ let redis = setup_redis_client(Some(config.redis_url.clone())).await;
+ let not_modified_key =
+ get_team_request_key(team.id, FlagRequestType::FlagDefinitionsNotModified);
+ redis.del(not_modified_key.clone()).await.unwrap();
+
+ let server = common::ServerHandle::for_config(config).await;
+ let http = reqwest::Client::new();
+ let url = format!(
+ "http://{}/flags/definitions?token={}",
+ server.addr, team.api_token
+ );
+
+ let response = http
+ .get(&url)
+ .header("Authorization", format!("Bearer {secret_token}"))
+ .send()
+ .await
+ .unwrap();
+ assert_eq!(response.status(), 200);
+
+ // Keep the ETag in Redis but drop the payload, so only the memo can bill the 304.
+ redis
+ .del(format!(
+ "posthog:1:cache/teams/{}/feature_flags/flags_with_cohorts.json",
+ team.id
+ ))
+ .await
+ .unwrap();
+
+ let bucket_before = current_bucket();
+ let response = http
+ .get(&url)
+ .header("Authorization", format!("Bearer {secret_token}"))
+ .header("If-None-Match", format!("W/\"{etag_value}\""))
+ .send()
+ .await
+ .unwrap();
+ assert_eq!(response.status(), 304);
+ let bucket_after = current_bucket();
+
+ let counter = common::poll_for_billing_counter_across_buckets(
+ &redis,
+ &not_modified_key,
+ bucket_before,
+ bucket_after,
+ )
+ .await;
+ assert_eq!(
+ counter, "1",
+ "304 should be billed from the memo the full response wrote"
+ );
+}
+
/// Poll the self-heal rebuild-requests set until it contains `team_id`, or return
/// false after ~2s. The enqueue runs in a background task, so a bounded retry is
/// needed rather than a single read.
diff --git a/rust/usage-ingestion/README.md b/rust/usage-ingestion/README.md
index 902c511ebac4..d6310cdc89c4 100644
--- a/rust/usage-ingestion/README.md
+++ b/rust/usage-ingestion/README.md
@@ -211,7 +211,7 @@ can count unmerged duplicates and is suitable only for an approximate read.
| `ai-ingestion` | `ai_events` | events | `{day}:{sha256 of event, distinct_id, uuid}` | AI ingestion consumer |
| `error-tracking` | `exceptions` | events | `{day}:{sha256 of event, distinct_id, uuid}` | error tracking server |
| `cdp` | `cdp_billable_invocations` | invocations | `event:{eventUuid}` / `flow:{invocationId}:{actionStepCount}:{kind}` / `webhook:{invocationId}` | CDP consumers |
-| `feature-flags` | `feature_flag_requests`, `feature_flag_local_evaluation_requests` | requests | fresh UUIDv7 per flush | feature flags service |
+| `feature-flags` | `feature_flag_requests`, `feature_flag_local_evaluation_requests`, `feature_flag_local_evaluation_not_modified_requests` | requests | fresh UUIDv7 per flush | feature flags service |
| `ingestion` | `survey_responses` | events | `{day}:{sha256 of event, distinct_id, uuid}` | ingestion consumers, ingestion API |
| `warehouse-sources` | `warehouse_rows_synced` | rows | the ExternalDataJob ID | warehouse sources worker |
| `batch-exports` | `batch_export_rows` | rows | the BatchExportRun ID | batch exports worker |
@@ -299,9 +299,11 @@ A bounded queue drained by one owned task sends records; shutdown closes and
awaits it within the aggregator flush timeout. A full queue drops and counts
the drop rather than growing, while Redis retains the authoritative count.
-The raw `feature_flag_requests` and
-`feature_flag_local_evaluation_requests` keys stay separate because the report
-prices them differently. Remote-config requests bill on neither side.
+The raw `feature_flag_requests`,
+`feature_flag_local_evaluation_requests`, and
+`feature_flag_local_evaluation_not_modified_requests` keys stay separate. The
+report prices full local evaluation responses differently, and each key has its
+own usage counter, so a 304 on the decide key would be counted twice. Remote-config requests bill on neither side.
### Trying it locally
diff --git a/services/mcp/src/api/generated.ts b/services/mcp/src/api/generated.ts
index eb44d44c910e..868ff846a1af 100644
--- a/services/mcp/src/api/generated.ts
+++ b/services/mcp/src/api/generated.ts
@@ -48264,7 +48264,7 @@ export namespace Schemas {
sdk: string;
/** Number of billable requests in this bucket. */
request_count: number;
- /** Estimated billing units. Local evaluation requests count as 10 units each. */
+ /** Estimated billing units. Local evaluation requests count as 10 units each, except 304 Not Modified responses, which count as 1. */
billing_units: number;
}