:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(warehouse-sources): add partnerstack transactions, rewards and stage tables (#124775)
Tom Owers88dba46cef58
diff --git a/posthog/test/repo_invariants/dataclass_frozen_baseline.txt b/posthog/test/repo_invariants/dataclass_frozen_baseline.txtindex 48ab07a5e8a6..d67d6e3eaece 100644--- a/posthog/test/repo_invariants/dataclass_frozen_baseline.txt+++ b/posthog/test/repo_invariants/dataclass_frozen_baseline.txt@@ -841,7 +841,6 @@1 products/warehouse_sources/backend/temporal/data_imports/sources/pardot/pardot.py1 products/warehouse_sources/backend/temporal/data_imports/sources/pardot/settings.py1 products/warehouse_sources/backend/temporal/data_imports/sources/partnerstack/partnerstack.py-1 products/warehouse_sources/backend/temporal/data_imports/sources/partnerstack/settings.py1 products/warehouse_sources/backend/temporal/data_imports/sources/pendo/pendo.py1 products/warehouse_sources/backend/temporal/data_imports/sources/pendo/settings.py1 products/warehouse_sources/backend/temporal/data_imports/sources/perigon/perigon.pydiff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.mdindex 326bac90a6a5..83a5a976c222 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md@@ -9178,15 +9178,15 @@ Note: The docs page embeds a full OpenAPI document (91 operationIds) which I par## PartnerStack — **thin**-Today (4): `customers`, `deals`, `leads`, `partnerships`+Today (8): `customers`, `deal_stages`, `deals`, `lead_stages`, `leads`, `partnerships`, `rewards`, `transactions`Diffed against: <https://docs.partnerstack.com/llms.txt>-- [ ] `GET /v2/transactions` — the commission ledger; without it partner revenue cannot be measured at all (high)-- [ ] `GET /v2/rewards` — reward records earned by partners, the payout-side counterpart to transactions (high)+- [x] `GET /v2/transactions` — the commission ledger; without it partner revenue cannot be measured at all (high)+- [x] `GET /v2/rewards` — reward records earned by partners, the payout-side counterpart to transactions (high)- [ ] `GET /v2/payouts` — money actually paid out to partners (high)- [ ] `GET /v2/actions` — tracked partner actions/events - the raw attribution events behind customers and deals (high)-- [ ] `GET /v2/sales-cycles/{object_type}/stages` — lookup resolving the stage ids carried on the deals and leads we already sync (high)+- [x] `GET /v2/sales-cycles/{object_type}/stages` — lookup resolving the stage ids carried on the deals and leads we already sync (high)- [ ] `GET /v2/groups` — lookup resolving the group each partnership belongs to (high)- [ ] `GET /v2/tiers (and /v2/tiers/collection)` — lookup resolving partnership tier ids, which drive commission rates (high)- [ ] `GET /v2/links/{partnership_identifier}` — per-partner referral links, the attribution key between traffic and customers (medium)@@ -9195,7 +9195,7 @@ Diffed against: <https://docs.partnerstack.com/llms.txt>- [ ] `GET /v2/product-book` — product book definitions that scope which products a partner can sell (medium)- [ ] `GET /v2/partner-team/{team_key}` — partner team membership resolving individual members behind a partnership (medium)-Note: Only 4 of roughly 20 listable v2 collections are exposed. Everything monetary - transactions, rewards, payouts - and every lookup that resolves ids on the objects we already sync (groups, tiers, sales cycle stages, products) is missing, so the current tables cannot be joined into a partner revenue model.+Note: The source syncs 8 tables. Transactions, rewards, and the deal and lead sales cycle stages are synced; the stages path is `/v2/sales_cycles/{object_type}/stages` (underscore). `GET /v2/payouts` exists only in the Partner API (partner bearer token), not the Vendor API this source authenticates against, so it stays unticked. The groups, tiers, and products lookups are still missing.## PayFit — gapsdiff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/partnerstack/canonical_descriptions.py b/products/warehouse_sources/backend/temporal/data_imports/sources/partnerstack/canonical_descriptions.pyindex e5b6f492d994..2961b98a13d2 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/partnerstack/canonical_descriptions.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/partnerstack/canonical_descriptions.py@@ -60,4 +60,76 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {"updated_at": "When the lead was last updated (epoch milliseconds).",},},+ "transactions": {+ "description": "A customer payment recorded in PartnerStack, used to calculate partner commissions.",+ "docs_url": "https://docs.partnerstack.com/reference/get_v2-transactions-2",+ "columns": {+ "key": "Key of the transaction record in PartnerStack.",+ "amount": "Transaction amount in cents for the specified currency.",+ "amount_usd": "Transaction amount in cents converted to USD.",+ "currency": "Three-letter ISO currency code of the transaction.",+ "customer_key": "Key of the customer who made the transaction.",+ "customer": "Customer info.",+ "product_key": "Key of the transaction product.",+ "category_key": "Key of the transaction category.",+ "metadata": "Additional data provided for the transaction on creation.",+ "approved": "Whether the transaction has been approved.",+ "archived": "Whether the transaction has been archived.",+ "test": "True if created in a test partnership or by a test customer.",+ "created_at": "When the transaction was created (epoch milliseconds).",+ "updated_at": "When the transaction was last updated (epoch milliseconds).",+ },+ },+ "rewards": {+ "description": "A reward earned by a partner, such as a commission on a transaction or a bounty for an action.",+ "docs_url": "https://docs.partnerstack.com/reference/get_v2-rewards-2",+ "columns": {+ "key": "The unique key of the reward.",+ "amount": "The amount of the reward in cents (USD).",+ "currency": "The currency of the reward.",+ "reward_status": "The status of the reward: hold, pending, approved, declined, or paid.",+ "description": "The description that the company gives to the reward.",+ "decline_reason": "The reason the reward has been held or declined.",+ "creation_source": "The source that created the reward, for example manual, offer_trigger, import, drip, or api_key.",+ "created_by": "The user key or 'system' that created the reward.",+ "partnership": "The partnership associated with the reward.",+ "customer": "The customer associated with the reward.",+ "transaction": "The transaction associated with the reward.",+ "action": "The action associated with the reward.",+ "offer": "The offer associated with the reward.",+ "invoice": "The invoice associated with the reward.",+ "created_at": "When the reward was created (epoch milliseconds).",+ "updated_at": "When the reward was last updated (epoch milliseconds).",+ },+ },+ "deal_stages": {+ "description": "A stage in your company's deal sales cycle, resolving the stage carried on each deal.",+ "docs_url": "https://docs.partnerstack.com/reference/get_v2-sales-cycles-object-type-stages",+ "columns": {+ "key": "The unique key of the stage.",+ "name": "The name of the stage.",+ "description": "The description of the stage.",+ "api_name": "The API name of the stage for non-PartnerStack systems.",+ "internal_status": "The PartnerStack status for the stage: open, closed, dead, closed won, closed lost, or custom.",+ "position": "The display position of the stage in the sales cycle.",+ "sales_cycle_key": "The unique key of the associated sales cycle.",+ "created_at": "When the stage was created (epoch milliseconds).",+ "updated_at": "When the stage was last updated (epoch milliseconds).",+ },+ },+ "lead_stages": {+ "description": "A stage in your company's lead sales cycle, resolving the stage carried on each lead.",+ "docs_url": "https://docs.partnerstack.com/reference/get_v2-sales-cycles-object-type-stages",+ "columns": {+ "key": "The unique key of the stage.",+ "name": "The name of the stage.",+ "description": "The description of the stage.",+ "api_name": "The API name of the stage for non-PartnerStack systems.",+ "internal_status": "The PartnerStack status for the stage: open, closed, dead, closed won, closed lost, or custom.",+ "position": "The display position of the stage in the sales cycle.",+ "sales_cycle_key": "The unique key of the associated sales cycle.",+ "created_at": "When the stage was created (epoch milliseconds).",+ "updated_at": "When the stage was last updated (epoch milliseconds).",+ },+ },}diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/partnerstack/partnerstack.py b/products/warehouse_sources/backend/temporal/data_imports/sources/partnerstack/partnerstack.pyindex c89df87c2fa9..7dc0bab99fb8 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/partnerstack/partnerstack.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/partnerstack/partnerstack.py@@ -119,11 +119,12 @@ def partnerstack_source("name": endpoint,"endpoint": {"path": config.path,- "params": {"limit": PAGE_SIZE},+ "params": {"limit": PAGE_SIZE} if config.paginated else {},"data_selector": "data.items",# A 200 whose body isn't the expected `{"data": {"items": [...]}}` envelope is# treated as transient and reissued, matching the hand-rolled source's retry."data_selector_malformed_retryable": True,+ **({} if config.paginated else {"paginator": "single_page"}),},}],diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/partnerstack/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/partnerstack/settings.pyindex 6464dcfbd194..73a4b185739e 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/partnerstack/settings.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/partnerstack/settings.py@@ -1,12 +1,14 @@from dataclasses import dataclass, field-@dataclass+@dataclass(frozen=True)class PartnerStackEndpointConfig:name: strpath: str# PartnerStack objects are identified by a globally unique `key`, so it is a safe primary key.primary_keys: list[str] = field(default_factory=lambda: ["key"])+ # Sales-cycle stage lookups return the whole list in one response, with no `limit` or cursor.+ paginated: bool = True# PartnerStack Vendor API v2 list endpoints. All are full-refresh only: while most objects expose@@ -18,6 +20,10 @@ PARTNERSTACK_ENDPOINTS: dict[str, PartnerStackEndpointConfig] = {"customers": PartnerStackEndpointConfig(name="customers", path="/customers"),"deals": PartnerStackEndpointConfig(name="deals", path="/deals"),"leads": PartnerStackEndpointConfig(name="leads", path="/leads"),+ "transactions": PartnerStackEndpointConfig(name="transactions", path="/transactions"),+ "rewards": PartnerStackEndpointConfig(name="rewards", path="/rewards"),+ "deal_stages": PartnerStackEndpointConfig(name="deal_stages", path="/sales_cycles/deal/stages", paginated=False),+ "lead_stages": PartnerStackEndpointConfig(name="lead_stages", path="/sales_cycles/lead/stages", paginated=False),}ENDPOINTS = tuple(PARTNERSTACK_ENDPOINTS.keys())diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/partnerstack/source.py b/products/warehouse_sources/backend/temporal/data_imports/sources/partnerstack/source.pyindex 0e0689f12b55..9526b5c0769a 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/partnerstack/source.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/partnerstack/source.py@@ -54,7 +54,7 @@ class PartnerStackSource(ResumableSource[PartnerStackSourceConfig, PartnerStackRreleaseStatus=ReleaseStatus.BETA,caption="""Enter your PartnerStack Vendor API keys to pull your partner program data into the PostHog Data warehouse.-You can find your **public key** and **private key** under **Settings → Integrations → PartnerStack API Keys** in [PartnerStack](https://dash.partnerstack.com/). The keys grant read access to your partnerships, customers, deals, and leads.+You can find your **public key** and **private key** under **Settings → Integrations → PartnerStack API Keys** in [PartnerStack](https://dash.partnerstack.com/). The keys grant read access to your partnerships, customers, deals, leads, transactions, rewards, and deal and lead sales-cycle stages.""",iconPath="/static/services/partnerstack.png",docsUrl="https://posthog.com/docs/cdp/sources/partnerstack",diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/partnerstack/tests/test_partnerstack.py b/products/warehouse_sources/backend/temporal/data_imports/sources/partnerstack/tests/test_partnerstack.pyindex 19d52ffd5af7..9a3d752dd99f 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/partnerstack/tests/test_partnerstack.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/partnerstack/tests/test_partnerstack.py@@ -73,6 +73,18 @@ class TestPagination:assert len(result.requests) == 1assert result.saved_states == []+ @parameterized.expand(+ [("deal_stages", "/api/v2/sales_cycles/deal/stages"), ("lead_stages", "/api/v2/sales_cycles/lead/stages")]+ )+ def test_stage_lookups_fetch_a_single_unpaginated_page(self, endpoint: str, path: str) -> None:+ result = _driver().run(endpoint, [_response([{"key": "stg_a"}, {"key": "stg_b"}], has_more=True)])++ assert result.raised is None+ assert result.rows == [{"key": "stg_a"}, {"key": "stg_b"}]+ assert [request.path for request in result.requests] == [path]+ assert result.params("limit") == [None]+ assert result.saved_states == []+class TestRetryAndFailure:@parameterized.expand([("rate_limited", 429), ("server_error", 500), ("bad_gateway", 503)])