:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(rocketlane): add 4 missing tables to the rocketlane source (#128352)
Tom Owers5c927f04cfe0
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.mdindex 82e5b668caf6..e5c72813d8a6 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md@@ -10389,10 +10389,10 @@ Diffed against: <https://developer.rocketlane.com/reference/get-all-projects>- [x] `invoices/{id}/line-items (get-invoice-line-items)` — line-item revenue breakdown against projects and tasks (high). Synced as `invoice_line_items` from `/invoices/{invoiceId}/lines`, one fetch per invoice.- [x] `time-entry-categories (get-time-entry-categories)` — lookup table resolving the category IDs carried on time_entries we already sync (high). Synced as `time_entry_categories` from `/time-entries/categories`.- [x] `phases (get-all-phases)` — lookup table resolving the phase IDs on tasks and projects we already sync (high). Synced as `phases`, one fetch per project, because `/phases` requires a `projectId` filter.-- [ ] `resource-allocations (get-all-resource-allocations)` — planned capacity to compare against logged time entries (high)-- [ ] `invoices/{id}/payments (get-invoice-payments)` — cash collection and AR aging (medium)-- [ ] `time-offs (get-all-timeoffs)` — needed for correct utilization and availability math (medium)-- [ ] `spaces (get-all-spaces)` — lookup resolving the space grouping projects and documents belong to (medium)+- [x] `resource-allocations (get-all-resource-allocations)` — planned capacity to compare against logged time entries (high). Synced as `resource_allocations` with one wide `startDate`/`endDate` window, because the endpoint requires both.+- [x] `invoices/{id}/payments (get-invoice-payments)` — cash collection and AR aging (medium). Synced as `invoice_payments`, one fetch per invoice.+- [x] `time-offs (get-all-timeoffs)` — needed for correct utilization and availability math (medium). Synced as `time_offs`.+- [x] `spaces (get-all-spaces)` — lookup resolving the space grouping projects and documents belong to (medium). Synced as `spaces`, one fetch per project, because `/spaces` requires a `projectId` filter.- [ ] `placeholders (get-placeholders)` — unfilled roles on allocations, useful for staffing gap reporting (low)- [ ] `space-documents (get-all-space-documents)` — document inventory per space (low)diff --git a/products/warehouse_sources/sources/rocketlane/canonical_descriptions.py b/products/warehouse_sources/sources/rocketlane/canonical_descriptions.pyindex 606c3657ec09..2855960fd192 100644--- a/products/warehouse_sources/sources/rocketlane/canonical_descriptions.py+++ b/products/warehouse_sources/sources/rocketlane/canonical_descriptions.py@@ -165,4 +165,76 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {"updatedAt": "When the phase was last modified (epoch milliseconds).",},},+ "resource_allocations": {+ "description": "A Rocketlane resource allocation, which plans a team member's or placeholder's time on a project over a date range.",+ "docs_url": "https://developer.rocketlane.com/reference/get-all-resource-allocations",+ "columns": {+ "allocationId": "The unique, system-generated identifier of the resource allocation.",+ "startDate": "The date the allocation starts (YYYY-MM-DD).",+ "endDate": "The date the allocation ends (YYYY-MM-DD).",+ "secondsPerDay": "The allocated time per day, in seconds.",+ "minutesPerDay": "The allocated time per day, in minutes.",+ "hoursPerDay": "The allocated time per day, in hours.",+ "duration": "The total allocated seconds, minutes and hours between the start and end dates.",+ "allocationType": "The type of the allocation.",+ "allocationFor": "Whether the allocation is for a team member (TEAM_MEMBER) or a placeholder (PLACEHOLDER).",+ "project": "The project the allocation is for.",+ "tasks": "The tasks the allocation is for.",+ "member": "The team member allocated, when allocationFor is TEAM_MEMBER.",+ "placeholder": "The placeholder allocated, when allocationFor is PLACEHOLDER.",+ "workId": "The identifier of the work item (task, phase or milestone) the allocation is for.",+ "workType": "The type of the work item (TASK, PHASE or MILESTONE).",+ "includeSaturday": "Whether Saturday counts as a working day for the allocation.",+ "includeSunday": "Whether Sunday counts as a working day for the allocation.",+ "fields": "The custom fields and their values on the allocation.",+ "budget": "The financial budget linked to the allocation.",+ "createdBy": "The team member who created the allocation.",+ "updatedBy": "The team member who last updated the allocation.",+ "createdAt": "When the allocation was created (epoch milliseconds).",+ "updatedAt": "When the allocation was last modified (epoch milliseconds).",+ },+ },+ "time_offs": {+ "description": "A Rocketlane time-off, a period when a team member is not available for work.",+ "docs_url": "https://developer.rocketlane.com/reference/get-all-timeoffs",+ "columns": {+ "timeOffId": "The unique, system-generated identifier of the time-off.",+ "user": "The team member who takes the time-off.",+ "note": "The note about the time-off.",+ "startDate": "The first day of the time-off (YYYY-MM-DD).",+ "endDate": "The last day of the time-off (YYYY-MM-DD).",+ "durationInMinutes": "The time-off per day, in minutes.",+ "type": "The type of the time-off, such as a full day or a half day.",+ "notifyUsers": "The users notified about the time-off.",+ "createdBy": "The team member who created the time-off.",+ "createdAt": "When the time-off was created (epoch milliseconds).",+ },+ },+ "spaces": {+ "description": "A Rocketlane space, an area of a project that holds its documents.",+ "docs_url": "https://developer.rocketlane.com/reference/get-all-spaces",+ "columns": {+ "spaceId": "The unique, system-generated identifier of the space.",+ "spaceName": "The name of the space.",+ "project": "The project the space belongs to.",+ "private": "Whether the space is private or shared.",+ "createdBy": "The team member who created the space.",+ "updatedBy": "The team member who last updated the space.",+ "createdAt": "When the space was created (epoch milliseconds).",+ "updatedAt": "When the space was last modified (epoch milliseconds).",+ },+ },+ "invoice_payments": {+ "description": "A payment record on a Rocketlane invoice, such as a payment received or a write-off.",+ "docs_url": "https://developer.rocketlane.com/reference/get-invoice-payments",+ "columns": {+ "invoiceId": "The identifier of the invoice the payment belongs to.",+ "paymentId": "The identifier of the payment record.",+ "paymentRecordType": "The type of the payment record (for example PAID or WRITE_OFF).",+ "currency": "The currency of the payment amount.",+ "paymentDate": "The date the payment was made.",+ "amount": "The amount of the payment.",+ "notes": "The notes about the payment.",+ },+ },}diff --git a/products/warehouse_sources/sources/rocketlane/rocketlane.py b/products/warehouse_sources/sources/rocketlane/rocketlane.pyindex 663dcae19e73..ca0d9da09134 100644--- a/products/warehouse_sources/sources/rocketlane/rocketlane.py+++ b/products/warehouse_sources/sources/rocketlane/rocketlane.py@@ -1,4 +1,5 @@import dataclasses+from datetime import datefrom typing import Any, Optional, castfrom requests import Request, Response@@ -25,6 +26,12 @@ ROCKETLANE_BASE_URL = "https://api.rocketlane.com/api/1.0"# validates access to every list endpoint.DEFAULT_PROBE_PATH = "/users"+# Endpoints with a required date window return rows that start on or after `startDate` and end on or+# before `endDate`. Rocketlane documents no maximum range, so the source sends one window wide enough+# to hold every past and planned row.+DATE_WINDOW_START = "2000-01-01"+DATE_WINDOW_YEARS_AHEAD = 10+@dataclasses.dataclass # nosemgrep: prefer-frozen-dataclasses -- pre-existing; new field shifts the backlog finding into diffclass RocketlaneResumeConfig:@@ -113,6 +120,18 @@ def _client_config(api_key: str) -> ClientConfig:}+def _date_window(today: date) -> dict[str, str]:+ end = date(today.year + DATE_WINDOW_YEARS_AHEAD, 12, 31)+ return {"startDate": DATE_WINDOW_START, "endDate": end.isoformat()}+++def _endpoint_params(config: RocketlaneEndpointConfig, today: date) -> dict[str, Any]:+ params: dict[str, Any] = {"pageSize": PAGE_SIZE, **config.params}+ if config.requires_date_window:+ params.update(_date_window(today))+ return params++def _source_response(endpoint: str, config: RocketlaneEndpointConfig, resource: Resource) -> SourceResponse:return SourceResponse(name=endpoint,@@ -192,7 +211,7 @@ def rocketlane_source("name": endpoint,"endpoint": {"path": config.path,- "params": {"pageSize": PAGE_SIZE},+ "params": _endpoint_params(config, date.today()),"data_selector": "data",# `data` is always present in a well-formed list response; a 200 body without it# means the shape changed — fail loud rather than silently syncing 0 rows.diff --git a/products/warehouse_sources/sources/rocketlane/settings.py b/products/warehouse_sources/sources/rocketlane/settings.pyindex 0e8fb2bd05e2..3c64cda6582f 100644--- a/products/warehouse_sources/sources/rocketlane/settings.py+++ b/products/warehouse_sources/sources/rocketlane/settings.py@@ -25,6 +25,10 @@ class RocketlaneEndpointConfig:page_size: int = PAGE_SIZEincremental_fields: list[Any] = field(default_factory=list)default_incremental_field: Optional[str] = None+ # Extra query params sent on every request to the endpoint.+ params: dict[str, Any] = field(default_factory=dict)+ # Set for endpoints that require a `startDate`/`endDate` window.+ requires_date_window: bool = False# All endpoints are full-refresh only: Rocketlane's list endpoints expose per-field filters but no@@ -78,6 +82,53 @@ ROCKETLANE_ENDPOINTS: dict[str, RocketlaneEndpointConfig] = {child_response_actions=[{"status_code": 404, "action": "ignore"}],),),+ # Without `includeAllFields` the API leaves out the member, placeholder, tasks and duration.+ # `startDate`/`endDate` are required, so the source sends one wide window (see `rocketlane.py`).+ "resource_allocations": RocketlaneEndpointConfig(+ name="resource_allocations",+ path="/resource-allocations",+ primary_keys=["allocationId"],+ partition_key="createdAt",+ params={"includeAllFields": "true"},+ requires_date_window=True,+ ),+ "time_offs": RocketlaneEndpointConfig(+ name="time_offs",+ path="/time-offs",+ primary_keys=["timeOffId"],+ partition_key="createdAt",+ params={"includeAllFields": "true"},+ ),+ # `/spaces` requires a `projectId` filter. Each space row already carries its `project`, and+ # `spaceId` is documented as globally unique.+ "spaces": RocketlaneEndpointConfig(+ name="spaces",+ path="/spaces?projectId={projectId}",+ primary_keys=["spaceId"],+ partition_key="createdAt",+ fanout=DependentEndpointConfig(+ parent_name="projects",+ resolve_param="projectId",+ resolve_field="projectId",+ include_from_parent=[],+ child_response_actions=[{"status_code": 404, "action": "ignore"}],+ ),+ ),+ # Payments carry no creation timestamp and the API does not document `paymentId` as globally+ # unique, so the parent invoice id is part of the key.+ "invoice_payments": RocketlaneEndpointConfig(+ name="invoice_payments",+ path="/invoices/{invoiceId}/payments",+ primary_keys=["invoiceId", "paymentId"],+ fanout=DependentEndpointConfig(+ parent_name="invoices",+ resolve_param="invoiceId",+ resolve_field="invoiceId",+ include_from_parent=["invoiceId"],+ parent_field_renames={"invoiceId": "invoiceId"},+ child_response_actions=[{"status_code": 404, "action": "ignore"}],+ ),+ ),}ENDPOINTS = tuple(ROCKETLANE_ENDPOINTS.keys())diff --git a/products/warehouse_sources/sources/rocketlane/source.py b/products/warehouse_sources/sources/rocketlane/source.pyindex 0f1de6116b22..4e5b96eafc19 100644--- a/products/warehouse_sources/sources/rocketlane/source.py+++ b/products/warehouse_sources/sources/rocketlane/source.py@@ -48,7 +48,7 @@ class RocketlaneSource(ResumableSource[RocketlaneSourceConfig, RocketlaneResumeCreleaseStatus=ReleaseStatus.BETA,caption="""Enter your Rocketlane API key to pull your Rocketlane data into the PostHog Data warehouse.-You can generate an API key under **Settings → API** in the [Rocketlane app](https://app.rocketlane.com/). This single key grants read access to your projects, phases, tasks, time entries, users, fields, and invoices.+You can generate an API key under **Settings → API** in the [Rocketlane app](https://app.rocketlane.com/). This single key grants read access to your projects, phases, spaces, tasks, time entries, time-offs, resource allocations, users, fields, and invoices.""",iconPath="/static/services/rocketlane.png",docsUrl="https://posthog.com/docs/cdp/sources/rocketlane",diff --git a/products/warehouse_sources/sources/rocketlane/tests/test_rocketlane.py b/products/warehouse_sources/sources/rocketlane/tests/test_rocketlane.pyindex 26ee24d06d84..22091fb69450 100644--- a/products/warehouse_sources/sources/rocketlane/tests/test_rocketlane.py+++ b/products/warehouse_sources/sources/rocketlane/tests/test_rocketlane.py@@ -1,5 +1,7 @@from typing import Any+import time_machine+from parameterized import parameterizedfrom products.warehouse_sources.sources.rocketlane._config import RocketlaneSourceConfig@@ -84,6 +86,27 @@ class TestFanout:[None, None, None, None],[{"invoiceLineItemId": 10, "invoiceId": 1}, {"invoiceLineItemId": 11, "invoiceId": 1}],),+ (+ "spaces",+ "projects",+ "projectId",+ ["/api/1.0/projects", "/api/1.0/spaces", "/api/1.0/spaces", "/api/1.0/spaces"],+ [None, "1", "1", "2"],+ [{"spaceId": 10}, {"spaceId": 11}],+ ),+ (+ "invoice_payments",+ "invoices",+ "invoiceId",+ [+ "/api/1.0/invoices",+ "/api/1.0/invoices/1/payments",+ "/api/1.0/invoices/1/payments",+ "/api/1.0/invoices/2/payments",+ ],+ [None, None, None, None],+ [{"paymentId": 10, "invoiceId": 1}, {"paymentId": 11, "invoiceId": 1}],+ ),])def test_walks_child_pages_per_parent_and_skips_missing_parent(@@ -155,6 +178,39 @@ class TestFanout:assert result.saved_states[-1].fanout_state["completed"] == ["/phases?projectId=1", "/phases?projectId=2"]+class TestRequestParams:+ @parameterized.expand(+ [+ ("projects", None, None, None),+ ("resource_allocations", "true", "2000-01-01", "2036-12-31"),+ ("time_offs", "true", None, None),+ ]+ )+ @time_machine.travel("2026-11-03", tick=False)+ def test_sends_endpoint_params(+ self,+ endpoint: str,+ include_all_fields: str | None,+ start_date: str | None,+ end_date: str | None,+ ) -> None:+ result = _driver().run(+ endpoint,+ [+ _response([{"id": 1}], has_more=True, next_token="t2"),+ _response([{"id": 2}], has_more=False, next_token=None),+ ],+ )++ assert result.raised is None+ assert result.params("pageSize") == ["100", "100"]+ assert result.params("pageToken") == [None, "t2"]+ # Required params must stay on every page, not only the first.+ assert result.params("includeAllFields") == [include_all_fields] * 2+ assert result.params("startDate") == [start_date] * 2+ assert result.params("endDate") == [end_date] * 2++class TestRocketlaneSourceResponse:@parameterized.expand([@@ -167,6 +223,10 @@ class TestRocketlaneSourceResponse:("invoice_line_items", ["invoiceId", "invoiceLineItemId"], None),("time_entry_categories", ["categoryId"], None),("phases", ["phaseId"], "createdAt"),+ ("resource_allocations", ["allocationId"], "createdAt"),+ ("time_offs", ["timeOffId"], "createdAt"),+ ("spaces", ["spaceId"], "createdAt"),+ ("invoice_payments", ["invoiceId", "paymentId"], None),])def test_response_uses_endpoint_primary_key(