:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(growthbook): add experiment results, teams, namespaces and holdouts tables (#121103)
Tom Owers27d4ec56973d
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.mdindex 68a0189619f3..258302df4d9b 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md@@ -5618,12 +5618,12 @@ Today (12): `data_sources`, `dimensions`, `environments`, `experiments`, `fact_mDiffed against: <https://docs.growthbook.io/openapi.yaml>-- [ ] `Experiment results (GET /v1/experiments/results; GET /v1/experiments/{id}/results)` — analyze measured lift, uncertainty, sample sizes, and variation performance beyond experiment metadata and result summaries (high)-- [ ] `Teams (GET /v1/teams)` — resolve team IDs on synced members and compare experiment ownership across named teams (high)-- [ ] `Namespaces (GET /v1/namespaces)` — resolve namespace references in feature rules and explain mutually exclusive experiment allocation (high)+- [x] `Experiment results (GET /v1/experiments/results; GET /v1/experiments/{id}/results)` — analyze measured lift, uncertainty, sample sizes, and variation performance beyond experiment metadata and result summaries (high). Added as `experiment_results` from the list endpoint, which returns each experiment's latest snapshot; the per-experiment endpoint is only needed for older phases or dimension breakdowns+- [x] `Teams (GET /v1/teams)` — resolve team IDs on synced members and compare experiment ownership across named teams (high)+- [x] `Namespaces (GET /v1/namespaces)` — resolve namespace references in feature rules and explain mutually exclusive experiment allocation (high)- [ ] `Feature revisions (GET /v2/feature-revisions; GET /v1/revisions)` — track feature rule changes, approval state, and rollout history instead of keeping only current feature definitions (medium)- [ ] `Contextual Bandits (GET /v1/contextual-bandits; GET /v1/contextual-bandits/{id}/results)` — compare adaptive allocation and measured rewards across contexts and optimization strategies (medium)-- [ ] `Holdouts (GET /v1/holdouts)` — identify persistent control groups and explain long-term exposure differences across experiments (medium)+- [x] `Holdouts (GET /v1/holdouts)` — identify persistent control groups and explain long-term exposure differences across experiments (medium)- [ ] `Saved-group revisions (GET /v1/saved-groups-revisions)` — track targeting group changes and explain changes in experiment eligibility over time (medium)- [ ] `Reports (GET /v1/reports)` — bring saved analysis definitions and reporting context into portfolio comparisons (medium)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/growthbook/canonical_descriptions.py b/products/warehouse_sources/backend/temporal/data_imports/sources/growthbook/canonical_descriptions.pyindex 5944d554fdb3..ccadda754d2b 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/growthbook/canonical_descriptions.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/growthbook/canonical_descriptions.py@@ -23,4 +23,24 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {"saved_groups": {"description": "Reusable targeting groups defined by attribute values or conditions."},"data_sources": {"description": "Data source definitions with identifier types and experiment assignment queries."},"members": {"description": "Organization members with global roles, project roles, and environment access."},+ "teams": {"description": "Teams with their members, global roles, project roles, and environment access."},+ "namespaces": {+ "description": "Namespaces that split traffic so experiments in the same namespace stay mutually exclusive."+ },+ "holdouts": {+ "description": "Holdouts that keep a persistent control group out of features and experiments, with their linked experiments, features, and lifecycle stage."+ },+ "experiment_results": {+ "description": "The latest completed results snapshot for each experiment, with per-variation metric results and analysis settings.",+ "docs_url": "https://docs.growthbook.io/api/",+ "columns": {+ "id": "Unique identifier of the results snapshot.",+ "experimentId": "ID of the experiment the snapshot belongs to.",+ "phase": "Experiment phase the snapshot analyzes.",+ "dateStart": "Start of the analysis window.",+ "dateEnd": "End of the analysis window.",+ "dateUpdated": "When the snapshot was last updated.",+ "results": "Results per dimension, with total users, the sample ratio mismatch check, and metric analyses for each variation.",+ },+ },}diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/growthbook/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/growthbook/settings.pyindex 25335aef8291..fc3c91bd4624 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/growthbook/settings.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/growthbook/settings.py@@ -36,6 +36,13 @@ ENDPOINTS: dict[str, GrowthBookEndpoint] = {"saved_groups": GrowthBookEndpoint(path="v1/saved-groups", data_selector="savedGroups"),"data_sources": GrowthBookEndpoint(path="v1/data-sources", data_selector="dataSources"),"members": GrowthBookEndpoint(path="v1/members", data_selector="members", partition_key=None),+ "teams": GrowthBookEndpoint(path="v1/teams", data_selector="teams", paginated=False),+ "namespaces": GrowthBookEndpoint(path="v1/namespaces", data_selector="namespaces", partition_key=None),+ "holdouts": GrowthBookEndpoint(path="v1/holdouts", data_selector="holdouts", paginated=False),+ # One row per experiment holding its latest non-dimension snapshot; pages can be empty while hasMore is true.+ "experiment_results": GrowthBookEndpoint(+ path="v1/experiments/results", data_selector="experimentResults", partition_key=None+ ),}# Timestamp fields in responses do not imply server-side timestamp filtering.diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/growthbook/tests/test_growthbook.py b/products/warehouse_sources/backend/temporal/data_imports/sources/growthbook/tests/test_growthbook.pyindex 2404e4be59b6..22551a1d3c9d 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/growthbook/tests/test_growthbook.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/growthbook/tests/test_growthbook.py@@ -56,6 +56,25 @@ def test_pagination_and_resume(assert result.sort_mode is None+def test_experiment_results_continue_past_pages_without_snapshots(+ config: GrowthBookSourceConfig,+ inputs: SourceInputs,+ manager: MagicMock,+ http: responses.RequestsMock,+) -> None:+ inputs.schema_name = "experiment_results"+ url = "https://api.growthbook.io/api/v1/experiments/results"+ snapshot = {"id": "snp_one", "experimentId": "exp_one", "phase": "0", "results": []}+ http.add(responses.GET, url, json={"experimentResults": [], "count": 0, "hasMore": True, "nextOffset": 100})+ http.add(+ responses.GET, url, json={"experimentResults": [snapshot], "count": 1, "hasMore": False, "nextOffset": None}+ )+ result = GrowthBookSource().source_for_pipeline(config, manager, inputs)+ assert [row for batch in cast(Iterable[Any], result.items()) for row in batch] == [snapshot]+ assert [parse_qs(urlsplit(call.request.url).query)["offset"] for call in http.calls] == [["0"], ["100"]]+ assert result.partition_keys is None++@pytest.mark.parametrize("empty", [True, False])def test_empty_collection_or_missing_envelope(config: GrowthBookSourceConfig,