:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(data-warehouse): add 1 missing table to the Freshchat source (#125107)
Tom Owersfa066c36583a
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.mdindex 67308cc4f0fa..a0b67d8a2c75 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md@@ -4952,7 +4952,7 @@ Diffed against: <https://developers.freshchat.com/api/>- [x] `/v2/conversations/{conversation_id}/messages` — message-level data for response time, volume and agent workload analysis (high)- [x] `/v2/roles` — lookup resolving the role IDs carried on agents (high)- [ ] `/v2/reports/raw` — bulk raw data export - the practical way to land conversation/agent history at scale (medium) — not a readable collection: `POST /reports/raw` submits an async extraction job and `GET /reports/raw/{id}` returns a status plus a link to a generated CSV/JSON file. It needs a mandatory `event` type from a fixed enum, each of which yields a different report schema, over a window capped at one month (24 hours for chat transcripts) and no earlier than 15 months back. There is no single stable table behind it.-- [ ] `/v2/outbound-messages` — outbound campaign message sends and their delivery state (medium)+- [x] `/v2/outbound-messages` — outbound campaign message sends and their delivery state (medium)- [ ] `/v2/metrics/historical` — vendor-computed historical conversation/agent metrics (medium)Note: There is no top-level GET /v2/conversations list - conversations are only enumerable per user (/v2/users/{user_id}/conversations), so an implementation must fan out from the users table already synced, then fan out again for messages. /v2/agents/{id} status updates and business-hours/within-bh are write/check endpoints, not collections.diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/freshchat/canonical_descriptions.py b/products/warehouse_sources/backend/temporal/data_imports/sources/freshchat/canonical_descriptions.pyindex 72dd3e4b36bf..5ca088c7bc56 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/freshchat/canonical_descriptions.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/freshchat/canonical_descriptions.py@@ -87,6 +87,22 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {"description": "Description of what the role covers.",},},+ "outbound_messages": {+ "description": "A templated outbound message sent from your Freshchat account, such as a WhatsApp notification.",+ "docs_url": "https://developers.freshchat.com/api/#outbound-messages",+ "columns": {+ "message_id": "Unique identifier for the outbound message.",+ "request_id": "Identifier of the send request that produced the message.",+ "from": "The business number the message was sent from.",+ "to": "The recipient numbers or handles the message was sent to.",+ "provider": "Channel the message was sent through, such as whatsapp.",+ "data": "The message template and the values used to fill it.",+ "status": "Delivery state of the message: Accepted, Sent, Delivered or Failed.",+ "failure_code": "Code that identifies why the message failed, when status is Failed.",+ "failure_reason": "Reason why the message failed, when status is Failed.",+ "created_on": "Time at which the send request was triggered, in epoch format.",+ },+ },"user_conversations": {"description": ("Links each user to the conversations they take part in. Freshchat lists conversations "diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/freshchat/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/freshchat/settings.pyindex 6352267b6f08..6d8c1f578708 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/freshchat/settings.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/freshchat/settings.py@@ -97,8 +97,8 @@ class FreshchatEndpointConfig:## The public REST API exposes no server-side updated_since / created_since cursor on the top-level# list endpoints, so every endpoint is full refresh only (no `supports_incremental`). Pagination is-# resumable by page number via the ResumableSource manager. The outbound-messages and metrics-# endpoints require time-window params and are not covered.+# resumable by page number via the ResumableSource manager. The metrics endpoints require a metric+# name and time-window params, and raw reports are async export jobs, so neither is covered.FRESHCHAT_ENDPOINTS: dict[str, FreshchatEndpointConfig] = {"agents": FreshchatEndpointConfig(name="agents",@@ -134,6 +134,14 @@ FRESHCHAT_ENDPOINTS: dict[str, FreshchatEndpointConfig] = {paginated=False,single_object=True,),+ "outbound_messages": FreshchatEndpointConfig(+ name="outbound_messages",+ path="/outbound-messages",+ data_key="outbound_messages",+ # The endpoint documents no page / items_per_page params and returns every message as one+ # array; page params on it could loop on the same array forever.+ paginated=False,+ ),"user_conversations": FreshchatEndpointConfig(name="user_conversations",path="/users/{user_id}/conversations",@@ -166,6 +174,7 @@ PRIMARY_KEYS: dict[str, list[str]] = {"channels": ["id"],"roles": ["id"],"accounts_configuration": ["app_id"],+ "outbound_messages": ["message_id"],"user_conversations": ["user_id", "id"],"conversation_messages": ["conversation_id", "id"],}diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/freshchat/source.py b/products/warehouse_sources/backend/temporal/data_imports/sources/freshchat/source.pyindex 69a8dcee1a0e..4c8080b9eb3d 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/freshchat/source.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/freshchat/source.py@@ -35,9 +35,10 @@ from products.warehouse_sources.backend.temporal.data_imports.sources.generated_from products.warehouse_sources.backend.types import ExternalDataSourceType# Covers Freshchat's top-level v2 list endpoints (Agents, Users, Groups, Channels, Roles), the-# single-row account configuration, and the two fan-out tables that reach conversations and their-# messages — Freshchat exposes no top-level conversations list, so both hang off Users. The-# outbound-messages and metrics endpoints require time-window params and are not covered.+# single-row account configuration, outbound messages, and the two fan-out tables that reach+# conversations and their messages — Freshchat exposes no top-level conversations list, so both hang+# off Users. The metrics endpoints require a metric name and time-window params, and raw reports are+# async export jobs, so neither is covered.# Freshchat has no server-side incremental cursor on these endpoints, so every endpoint is full# refresh, resumable by page number.diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/freshchat/tests/test_freshchat.py b/products/warehouse_sources/backend/temporal/data_imports/sources/freshchat/tests/test_freshchat.pyindex 2e220fc0902f..54f62e8d945a 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/freshchat/tests/test_freshchat.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/freshchat/tests/test_freshchat.py@@ -65,6 +65,22 @@ class TestGetRows:# State saved once, pointing at page 2 (the next page after the first was written).assert result.saved_states == [FreshchatResumeConfig(page=2)]+ def test_outbound_messages_is_one_unpaginated_request(self) -> None:+ # The endpoint documents no paging and returns every message at once. Page params would+ # make the paginator re-request the same array until it hit an empty page that never comes.+ messages = [+ {"message_id": "m1", "request_id": "r1", "status": "Delivered", "created_on": 1700000000},+ {"message_id": "m2", "request_id": "r1", "status": "Failed", "failure_code": "OBM001"},+ ]+ result = _driver().run("outbound_messages", [ScriptedResponse(json={"outbound_messages": messages})])++ assert result.raised is None+ assert result.rows == messages+ assert len(result.requests) == 1+ assert result.requests[0].path == "/v2/outbound-messages"+ assert result.requests[0].query == {}+ assert result.saved_states == []+def test_bearer_token_is_redacted_from_samples(self) -> None:# The token rides in the Authorization header; it must be value-redacted from captured# HTTP samples via the tracked session's redact_values.