:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(pardot): add email template, custom redirect, form handler and landing page tables (#125428)
Tom Owersdbe6e8cb9a77
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.mdindex 54e42114e085..75ae4834dda5 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md@@ -9147,10 +9147,10 @@ Diffed against: <https://developer.salesforce.com/docs/marketing/pardot/guide/ve- [x] `Lifecycle Stages (GET /api/v5/objects/lifecycle-stages)` — resolve lifecycleStageId values on prospects into sales journey stages (high)- [x] `Lifecycle Histories (GET /api/v5/objects/lifecycle-histories)` — measure time between funnel stages and progression from prospect to won opportunity (high)- [x] `Emails (GET /api/v5/objects/emails)` — resolve emailId values on visitor activities and analyze individual email performance (high)-- [ ] `Email Templates (GET /api/v5/objects/email-templates)` — resolve emailTemplateId values on visitor activities and list emails for performance comparisons (high)-- [ ] `Custom Redirects (GET /api/v5/objects/custom-redirects)` — resolve customRedirectId values on visitor activities into tracked links and campaigns (high)-- [ ] `Form Handlers (GET /api/v5/objects/form-handlers)` — resolve formHandlerId values on visitor activities for external form conversion analysis (high)-- [ ] `Landing Pages (GET /api/v5/objects/landing-pages)` — resolve landingPageId values on visitor activities into conversion pages (high)+- [x] `Email Templates (GET /api/v5/objects/email-templates)` — resolve emailTemplateId values on visitor activities and list emails for performance comparisons (high)+- [x] `Custom Redirects (GET /api/v5/objects/custom-redirects)` — resolve customRedirectId values on visitor activities into tracked links and campaigns (high)+- [x] `Form Handlers (GET /api/v5/objects/form-handlers)` — resolve formHandlerId values on visitor activities for external form conversion analysis (high)+- [x] `Landing Pages (GET /api/v5/objects/landing-pages)` — resolve landingPageId values on visitor activities into conversion pages (high)- [x] `Visitor Page Views (GET /api/v5/objects/visitor-page-views)` — resolve visitorPageViewId values on visitor activities into individual page visits (high)Note: Read the official v5 overview and each listed object reference, then checked their Query GET operations. Existing visitor activities carry IDs for several missing marketing assets. Lifecycle history adds stage transitions beyond the current prospect stage. Excluded exports, authentication, write operations, and layout configuration.diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/pardot/canonical_descriptions.py b/products/warehouse_sources/backend/temporal/data_imports/sources/pardot/canonical_descriptions.pyindex bfcdc2145b74..f877b85c0092 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/pardot/canonical_descriptions.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/pardot/canonical_descriptions.py@@ -292,4 +292,75 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {"durationInSeconds": "Length of the page view in seconds.",},},+ "email_templates": {+ "description": "A reusable email layout for engagement programs, autoresponders, one-to-one and list emails.",+ "docs_url": f"{_DOCS_BASE}/email-template-v5.html",+ "columns": {+ **_shared("email_templates"),+ "subject": "Subject line of the email template.",+ "type": "Content type of the template (html, text or htmlAndText).",+ "isOneToOneEmail": "Whether the template is available for one-to-one emails.",+ "isAutoResponderEmail": "Whether the template is available for autoresponder emails.",+ "isDripEmail": "Whether the template is available for Engagement Studio programs.",+ "isListEmail": "Whether the template is available for list emails.",+ "trackerDomainId": "ID of the tracker domain used for links in the template.",+ "tagReplacementLanguage": "Merge language of the template (pml, hml or undetermined).",+ },+ },+ "custom_redirects": {+ "description": "A tracked link that records a visitor activity when someone clicks it before sending them on.",+ "docs_url": f"{_DOCS_BASE}/custom-redirect-v5.html",+ "columns": {+ **_shared("custom_redirects"),+ "trackerDomainId": "ID of the tracker domain used in the redirect URL.",+ "destinationUrl": "URL the visitor is sent to after clicking the redirect.",+ "url": "Tracked URL of the redirect, without the vanity URL.",+ "vanityUrl": "Vanity URL of the redirect, including the tracker domain.",+ "vanityUrlPath": "Path part of the vanity URL.",+ "trackedUrl": "The vanity URL when one is set, otherwise the tracked URL.",+ "bitlyIsPersonalized": "Whether a custom Bitly URL is set.",+ "bitlyShortUrl": "Bitly short URL of the redirect, if set.",+ "gaSource": "Google Analytics campaign source.",+ "gaMedium": "Google Analytics campaign medium.",+ "gaTerm": "Google Analytics campaign keyword.",+ "gaContent": "Google Analytics campaign content.",+ "gaCampaign": "Google Analytics campaign ID.",+ },+ },+ "form_handlers": {+ "description": "An endpoint that receives submissions from a form hosted outside Account Engagement.",+ "docs_url": f"{_DOCS_BASE}/form-handler-v5.html",+ "columns": {+ **_shared("form_handlers"),+ "trackerDomainId": "ID of the tracker domain associated with the form handler.",+ "isDataForwarded": "Whether submitted data is forwarded to the success location.",+ "successLocation": "URL the visitor is sent to after a successful submission.",+ "errorLocation": "URL the visitor is sent to when a submission has an error.",+ "isAlwaysEmail": "Whether autoresponder emails are sent after every submission, without throttling.",+ "isCookieless": "Whether the form handler runs in kiosk mode and does not cookie the submitter.",+ "embedCode": "Code to embed the form handler in a web page.",+ },+ },+ "landing_pages": {+ "description": "A hosted landing page, usually with a form, used to convert visitors into prospects.",+ "docs_url": f"{_DOCS_BASE}/landing-page-v5.html",+ "columns": {+ **_shared("landing_pages"),+ "title": "Title of the landing page.",+ "description": "Meta description of the landing page, used for SEO.",+ "formId": "ID of the form on the landing page.",+ "layoutTemplateId": "ID of the layout template the landing page uses.",+ "trackerDomainId": "ID of the tracker domain associated with the landing page.",+ "layoutType": "Where the landing page was created.",+ "url": "URL of the landing page.",+ "vanityUrl": "Vanity URL of the landing page, if set.",+ "vanityUrlPath": "Path part of the vanity URL.",+ "bitlyShortUrl": "Bitly short URL of the landing page, if set.",+ "redirectLocation": "URL the prospect is redirected to instead of seeing the thank-you content.",+ "isUseRedirectLocation": "Whether the prospect is redirected instead of seeing the thank-you content.",+ "isDoNotIndex": "Whether the landing page is hidden from search engine indexing.",+ "archiveDate": "Date the landing page is archived.",+ "salesforceCmsId": "ID of the copy of the landing page in Salesforce CMS, if it was copied.",+ },+ },}diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/pardot/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/pardot/settings.pyindex 30f9aae29d25..5dfc4810baf7 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/pardot/settings.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/pardot/settings.py@@ -468,6 +468,119 @@ PARDOT_ENDPOINTS: dict[str, PardotEndpointConfig] = {"createdAt",),),+ "email_templates": PardotEndpointConfig(+ name="email_templates",+ path="email-templates",+ order_by="id",+ incremental_fields=_CREATED_AND_UPDATED,+ # htmlMessage/textMessage aren't queryable — v5 400s if either is in a query's+ # `fields` list.+ fields=(+ "id",+ "name",+ "subject",+ "type",+ "isOneToOneEmail",+ "isAutoResponderEmail",+ "isDripEmail",+ "isListEmail",+ "campaignId",+ "folderId",+ "trackerDomainId",+ "tagReplacementLanguage",+ "isDeleted",+ *_TIMESTAMPS,+ ),+ ),+ "custom_redirects": PardotEndpointConfig(+ name="custom_redirects",+ path="custom-redirects",+ # Custom redirects only sort and filter on id, so this table is full refresh.+ order_by="id",+ fields=(+ "id",+ "name",+ "campaignId",+ "folderId",+ "trackerDomainId",+ "destinationUrl",+ "url",+ "vanityUrl",+ "vanityUrlPath",+ "trackedUrl",+ "bitlyIsPersonalized",+ "bitlyShortUrl",+ # The ga* fields need a Google Analytics connector; business units without one+ # refuse them, and the transport drops a refused field and retries.+ "gaSource",+ "gaMedium",+ "gaTerm",+ "gaContent",+ "gaCampaign",+ "salesforceId",+ "isDeleted",+ *_TIMESTAMPS,+ ),+ ),+ "form_handlers": PardotEndpointConfig(+ name="form_handlers",+ path="form-handlers",+ # Form handlers have no updatedAt and only filter on createdAt, so an incremental+ # sync would never pick up edits — full refresh.+ order_by="id",+ fields=(+ "id",+ "name",+ "campaignId",+ "folderId",+ "trackerDomainId",+ "isDataForwarded",+ "successLocation",+ "errorLocation",+ "isAlwaysEmail",+ "isCookieless",+ "embedCode",+ "salesforceId",+ "isDeleted",+ "createdAt",+ "createdById",+ "updatedById",+ ),+ ),+ "landing_pages": PardotEndpointConfig(+ name="landing_pages",+ path="landing-pages",+ order_by="id",+ incremental_fields=_CREATED_AND_UPDATED,+ # The page content and CSS fields (content, regionContent, layoutCss, ...) aren't+ # queryable — v5 400s if any of them is in a query's `fields` list.+ fields=(+ "id",+ "name",+ "title",+ "description",+ "campaignId",+ "folderId",+ "formId",+ "layoutTemplateId",+ "trackerDomainId",+ "layoutType",+ "layoutTableBorder",+ "url",+ "vanityUrl",+ "vanityUrlPath",+ "bitlyIsPersonalized",+ "bitlyShortUrl",+ "redirectLocation",+ "isUseRedirectLocation",+ "isDoNotIndex",+ "archiveDate",+ "salesforceId",+ "salesforceCmsId",+ "isDeleted",+ *_TIMESTAMPS,+ ),+ ),}ENDPOINTS = tuple(PARDOT_ENDPOINTS.keys())diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/pardot/tests/test_pardot.py b/products/warehouse_sources/backend/temporal/data_imports/sources/pardot/tests/test_pardot.pyindex 6421d18ba748..ff3540e62e68 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/pardot/tests/test_pardot.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/pardot/tests/test_pardot.py@@ -144,6 +144,9 @@ class TestBuildQueryParams:("prospect_accounts", "createdAt"),# The requested field isn't one this endpoint advertises.("prospects", "lastActivityAt"),+ # Form handlers filter on createdAt but have no updatedAt, so a createdAt+ # watermark would never pick up edits.+ ("form_handlers", "createdAt"),],)def test_unsupported_cursor_falls_back_to_the_default_sort(self, endpoint: str, incremental_field: str) -> None:@@ -173,6 +176,11 @@ class TestFieldLists:# Email bodies aren't queryable and 400 the same way.("emails", "htmlMessage"),("emails", "textMessage"),+ ("email_templates", "htmlMessage"),+ ("email_templates", "textMessage"),+ ("landing_pages", "content"),+ ("landing_pages", "regionContent"),+ ("landing_pages", "layoutCss"),],)def test_does_not_request_write_only_fields(self, endpoint: str, write_only_field: str) -> None: