:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(data-warehouse): add demodesk recording details and transcripts tables (#119237)
Tom Owers2a1228206aec
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.mdindex b2f317980bf8..e954181e87d0 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md@@ -3663,12 +3663,12 @@ Note: DeepSource is a Relay-style GraphQL API at https://api.deepsource.com/grap## Demodesk — gaps-Today (6): `demo_templates`, `demos`, `recording_scorecards`, `recording_summaries`, `recordings`, `users`+Today (8): `demo_templates`, `demos`, `recording_details`, `recording_scorecards`, `recording_summaries`, `recording_transcripts`, `recordings`, `users`Diffed against: <https://demodesk.com/api/docs/v2/generated.yaml>-- [ ] `Recording transcripts (GET /api/v2/recordings/{token}/transcript)` — analyze conversation content, speaker contributions, and topics across imported meetings (high)-- [ ] `Recording details, participants, and statistics (GET /api/v2/recordings/{token})` — analyze participant attendance, engagement scores, and feedback scores absent from recording listings (medium)+- [x] `Recording transcripts (GET /api/v2/recordings/{token}/transcript)` — analyze conversation content, speaker contributions, and topics across imported meetings (high)+- [x] `Recording details, participants, and statistics (GET /api/v2/recordings/{token})` — analyze participant attendance, engagement scores, and feedback scores absent from recording listings (medium)Note: Parsed the v2 OpenAPI paths and response schemas, then read the official legacy v1 reference. Existing tables cover meetings, templates, users, recordings, summaries, and scorecards. Missing content needs recording-token fan-out; transcript responses contain paragraph and sentence collections. Auth, webhooks, and writes were excluded; detailed participant and statistics objects were checked separately from list metadata.diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/demodesk/canonical_descriptions.py b/products/warehouse_sources/backend/temporal/data_imports/sources/demodesk/canonical_descriptions.pyindex 12a6795ee3a5..6930a7062312 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/demodesk/canonical_descriptions.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/demodesk/canonical_descriptions.py@@ -98,4 +98,39 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {"updatedAt": "When the scorecard was last updated.",},},+ "recording_details": {+ "description": "Detailed metadata for a recording, including the host, participants, attendance, and engagement and feedback statistics.",+ "docs_url": "https://demodesk.com/api/docs/index.html",+ "columns": {+ "recordingId": "Internal recording id.",+ "recordingToken": "Public recording token.",+ "demoId": "Internal id of the recorded meeting.",+ "userId": "Id of the meeting host user.",+ "name": "Recording name.",+ "status": "Recording status: pending, ready, failure, cancelled, or expired.",+ "postprocessingStatus": "Post-processing status (transcription, statistics, summaries): pending, done, failed, or too_short.",+ "access": "Access permission level: company_wide, selected_groups, host_and_participants, or host_only.",+ "createdAt": "When the recording was created.",+ "updatedAt": "When the recording was last updated.",+ "demoStartDate": "Start time of the recorded meeting.",+ "durationMs": "Recording duration in milliseconds.",+ "host": "The meeting host, or null when no host is assigned.",+ "audioOnly": "Whether the recording has audio only.",+ "attendeeClassification": "Whether the meeting was internal or external.",+ "meetingLocation": "Where the meeting took place.",+ "participants": "Meeting participants, each with display name, email, permission, whether they attended, and CRM links.",+ "groups": "Groups assigned to the meeting. Only set when access is selected_groups.",+ "statistics": "Derived meeting statistics: engagement score (0 to 10) and feedback score.",+ "crmLinks": "CRM records the meeting is linked to.",+ },+ },+ "recording_transcripts": {+ "description": "The transcript of a recording, grouped into speaker paragraphs and sentences.",+ "docs_url": "https://demodesk.com/api/docs/index.html",+ "columns": {+ "recordingToken": "Token of the recording the transcript belongs to.",+ "language": "Language of the transcript.",+ "paragraphs": "Transcript paragraphs, each with start and end time in seconds, speaker, and sentences.",+ },+ },}diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/demodesk/demodesk.py b/products/warehouse_sources/backend/temporal/data_imports/sources/demodesk/demodesk.pyindex 4e4c8d4971e3..8c937aec60c9 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/demodesk/demodesk.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/demodesk/demodesk.py@@ -42,7 +42,7 @@ class DemodeskResumeConfig:# v1 endpoints checkpoint a page number, v2 endpoints an opaque cursor; one field is set.page: int | None = Nonecursor: str | None = None- # `recording_summaries`/`recording_scorecards` fan out over `recordings`; this is the+ # The `recording_*` endpoints fan out over `recordings`; this is the# `build_dependent_resource` checkpoint (which parents finished, where the current one# stopped), mutually exclusive with `page`/`cursor` above.fanout_state: dict[str, Any] | None = None@@ -116,9 +116,9 @@ def demodesk_client_config(api_key: str, api_version: DemodeskApiVersion) -> Cli# `Authorization` is, so `allow_redirects=False` is the only thing that keeps either auth# scheme's credential from following a spoofed 3xx off `BASE_URL`."allow_redirects": False,- # `recording_summaries` bodies carry free-text AI meeting summaries and `recordings`- # carries signed `temporaryDirectUrl` links; keep both out of the shared HTTP sample- # store (still metered and logged) rather than trust the name-based scrubbers on+ # `recording_summaries` and `recording_transcripts` bodies carry free-text meeting content+ # and `recordings` carries signed `temporaryDirectUrl` links; keep them out of the shared+ # HTTP sample store (still metered and logged) rather than trust the name-based scrubbers on# unstructured content."capture": False,}diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/demodesk/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/demodesk/settings.pyindex 1d8834eb4bff..5c1afe3796d2 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/demodesk/settings.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/demodesk/settings.py@@ -115,6 +115,42 @@ DEMODESK_ENDPOINTS: dict[str, DemodeskEndpointConfig] = {child_response_actions=[{"status_code": 404, "action": "ignore"}],),),+ "recording_details": DemodeskEndpointConfig(+ name="recording_details",+ path="/v2/recordings/{recording_token}",+ api_version="v2",+ primary_keys=["recordingId"],+ partition_key="createdAt",+ # Full refresh only: CRM link changes on the recording or its participants do not advance+ # `updatedAt`, so bounding the parent listing by that watermark would leave them stale.+ fanout=DependentEndpointConfig(+ parent_name="recordings",+ resolve_param="recording_token",+ resolve_field="recordingToken",+ # The detail row already carries its own `recordingToken`.+ include_from_parent=[],+ child_response_actions=[{"status_code": 404, "action": "ignore"}],+ ),+ ),+ "recording_transcripts": DemodeskEndpointConfig(+ name="recording_transcripts",+ path="/v2/recordings/{recording_token}/transcript",+ api_version="v2",+ # One transcript per recording. It carries no timestamp, so the table is full refresh only.+ primary_keys=["recordingToken"],+ fanout=DependentEndpointConfig(+ parent_name="recordings",+ resolve_param="recording_token",+ resolve_field="recordingToken",+ include_from_parent=["recordingToken"],+ parent_field_renames={"recordingToken": "recordingToken"},+ child_response_actions=[+ {"status_code": 404, "action": "ignore"},+ # 202 means the transcript is still being processed; the next sync picks it up.+ {"status_code": 202, "action": "ignore"},+ ],+ ),+ ),}ENDPOINTS = tuple(DEMODESK_ENDPOINTS.keys())diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/demodesk/tests/test_demodesk.py b/products/warehouse_sources/backend/temporal/data_imports/sources/demodesk/tests/test_demodesk.pyindex cb2e036694c8..0497320c8ea4 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/demodesk/tests/test_demodesk.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/demodesk/tests/test_demodesk.py@@ -159,6 +159,37 @@ class TestDemodeskFanout:assert result.paths == ["/api/v2/recordings", "/api/v2/recordings/tokB/summaries"]assert [row["summaryId"] for row in result.rows] == ["sB"]+ def test_recording_transcripts_skip_unready_transcripts(self) -> None:+ result = _driver().run(+ "recording_transcripts",+ route(+ {+ "/v2/recordings": [+ ScriptedResponse(+ json={+ "data": [+ {"recordingToken": "tokA"},+ {"recordingToken": "tokB"},+ {"recordingToken": "tokC"},+ ],+ "meta": {"hasNext": False, "limit": 100},+ }+ )+ ],+ "/v2/recordings/tokA/transcript": [+ ScriptedResponse(json={"data": {"language": "en", "paragraphs": []}})+ ],+ "/v2/recordings/tokB/transcript": [+ ScriptedResponse(status=202, json={"error": {"message": "processing"}})+ ],+ "/v2/recordings/tokC/transcript": [ScriptedResponse(status=204)],+ }+ ),+ )++ assert result.raised is None+ assert [(row["recordingToken"], row["language"]) for row in result.rows] == [("tokA", "en")]+class TestValidateCredentials:@pytest.mark.parametrize(