:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(data-warehouse): add pabbly plans and commissions tables (#123993)
Tom Owersac68190da262
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.mdindex 55ee53d27045..c8efee260ab8 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md@@ -9041,10 +9041,10 @@ Today (13): `addon_list_category`, `addons`, `coupons`, `customers`, `invoices`,Diffed against: <https://apidocs.pabbly.com/pabbly/subscription-billing>-- [ ] `/plans (and /plans/{product_id})` — lookup table for the plan ids carried on every subscription and invoice - currently unresolvable (high)-- [ ] `/commissions` — affiliate commission records, the revenue-share ledger (high)-- [ ] `/mrrsubscription/` — vendor's headline MRR metric endpoint (high)-- [ ] `/revenuetransaction/` — net-revenue / ARPU report rows (high)+- [x] `/plans (and /plans/{product_id})` — lookup table for the plan ids carried on every subscription and invoice - currently unresolvable (high). Added as `plans` (the global `/plans` list). `/plans/{product_id}` skipped: it returns the same rows filtered per product.+- [x] `/commissions` — affiliate commission records, the revenue-share ledger (high). Added as `commissions`.+- ~~`/mrrsubscription/` — vendor's headline MRR metric endpoint (high)~~ — skipped: it is a `POST` report over a relative interval such as `last_30_days` for one product, and its rows carry no subscription or customer id to key on+- ~~`/revenuetransaction/` — net-revenue / ARPU report rows (high)~~ — skipped: it is a `POST` report over a relative interval for one product, and its rows repeat the invoices, refunds and transactions this source already syncs- [ ] `/commissions/clicks` — affiliate click events - the top of the partner funnel (medium)- [ ] `/affiliate/links` — lookup resolving affiliate link ids on commissions and clicks (medium)- [ ] `/scheduledchanges/{subscription_id}` — pending plan changes; needed for forward-looking churn/upgrade analysis (medium)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/pabbly_subscriptions_billing/canonical_descriptions.py b/products/warehouse_sources/backend/temporal/data_imports/sources/pabbly_subscriptions_billing/canonical_descriptions.pyindex 5d5d613e3f14..0f79f893a094 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/pabbly_subscriptions_billing/canonical_descriptions.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/pabbly_subscriptions_billing/canonical_descriptions.py@@ -187,6 +187,30 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {"updatedAt": "When the license batch was last updated.",},},+ "plans": {+ "description": "A plan of a product, with its price and billing cycle.",+ "docs_url": PABBLY_DOCS_URL,+ "columns": {+ "id": "The unique ID of the plan.",+ "product_id": "The ID of the product the plan belongs to.",+ "plan_name": "The name of the plan.",+ "plan_code": "The unique code of the plan.",+ "plan_active": "Whether the plan is active.",+ "price": "The price of the plan per billing period.",+ "currency_code": "The currency of the plan price.",+ "billing_period": "The unit of the billing period, for example m for month.",+ "billing_period_num": "The number of billing period units between charges.",+ "billing_cycle": "How long the plan bills for, for example lifetime or specific.",+ "billing_cycle_num": "The number of billing cycles when the plan bills for a fixed number of cycles.",+ "trial_period": "The length of the free trial, in days.",+ "setup_fee": "The one-time setup fee charged with the first payment.",+ "plan_description": "The description of the plan.",+ "checkout_page": "The URL of the plan's checkout page.",+ "redirect_url": "The URL the customer is sent to after checkout.",+ "createdAt": "When the plan was created.",+ "updatedAt": "When the plan was last updated.",+ },+ },"multiplans": {"description": "A multiplan checkout page combining several plans of a product.","docs_url": PABBLY_DOCS_URL,@@ -212,6 +236,29 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {"updatedAt": "When the payment method was last updated.",},},+ "commissions": {+ "description": "An affiliate commission earned on an invoice paid by a referred customer.",+ "docs_url": PABBLY_DOCS_URL,+ "columns": {+ "id": "The unique ID of the commission.",+ "affiliate_id": "The ID of the affiliate who earned the commission.",+ "affiliate_email": "The email address of the affiliate.",+ "customer_email": "The email address of the referred customer.",+ "invoice_id": "The ID of the invoice the commission was earned on.",+ "invoice_number": "The number of the invoice the commission was earned on.",+ "plan_id": "The ID of the plan the referred customer bought.",+ "product_name": "The name of the product the referred customer bought.",+ "click_id": "The ID of the affiliate link click that referred the customer.",+ "commission_amount": "The commission amount owed to the affiliate.",+ "currency": "The currency symbol of the commission amount.",+ "status": "The commission status, for example Paid, Unpaid or Refunded.",+ "is_paid": "Whether the commission was paid out to the affiliate.",+ "is_void": "Whether the commission was voided.",+ "payout_date": "When the commission was paid out, if it was.",+ "createdAt": "When the commission was created.",+ "updatedAt": "When the commission was last updated.",+ },+ },"payment_gateways": {"description": "A payment gateway connected to your Pabbly Subscription Billing account.","docs_url": PABBLY_DOCS_URL,diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/pabbly_subscriptions_billing/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/pabbly_subscriptions_billing/settings.pyindex b2823fc76e93..b80151ccbbe7 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/pabbly_subscriptions_billing/settings.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/pabbly_subscriptions_billing/settings.py@@ -48,6 +48,12 @@ PABBLY_ENDPOINTS: dict[str, PabblyEndpointConfig] = {# Product catalog metadata only (no customer content or credentials); safe to capture.capture_http_samples=True,),+ "plans": PabblyEndpointConfig(+ name="plans",+ path="/plans",+ # Plan catalog metadata only (no customer content or credentials); safe to capture.+ capture_http_samples=True,+ ),"multiplans": PabblyEndpointConfig(name="multiplans",path="/multiplans",@@ -55,6 +61,7 @@ PABBLY_ENDPOINTS: dict[str, PabblyEndpointConfig] = {capture_http_samples=True,),"payment_gateways": PabblyEndpointConfig(name="payment_gateways", path="/paymentgateways"),+ "commissions": PabblyEndpointConfig(name="commissions", path="/commissions"),"addons": PabblyEndpointConfig(name="addons",path="/addons/{parent_id}",diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/pabbly_subscriptions_billing/tests/test_pabbly_subscriptions_billing.py b/products/warehouse_sources/backend/temporal/data_imports/sources/pabbly_subscriptions_billing/tests/test_pabbly_subscriptions_billing.pyindex dd7e0ebe664c..b9af83bb25af 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/pabbly_subscriptions_billing/tests/test_pabbly_subscriptions_billing.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/pabbly_subscriptions_billing/tests/test_pabbly_subscriptions_billing.py@@ -232,6 +232,7 @@ class TestHttpSampleCapture:# bodies to HTTP samples. Everything carrying customer PII, payment data, redeemable# secrets, or bearer URLs the scrubbers can't recognise must stay out of captured samples.("products", True),+ ("plans", True),("multiplans", True),("addons", True),("addon_list_category", True),@@ -244,6 +245,7 @@ class TestHttpSampleCapture:("transactions", False),("licenses", False),("coupons", False),+ ("commissions", False),])def test_capture_enabled_only_for_catalog_metadata_endpoints(