:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(data-warehouse): add 4 missing tables to the Harvest source (#121119)
Tom Owers6bfcf60cb02c
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.mdindex c334952419ab..fcb64729c16e 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md@@ -5712,18 +5712,18 @@ Note: Downloaded the official bundled OpenAPI specification and enumerated GET c## Harvest — gaps-Today (15): `clients`, `contacts`, `estimate_item_categories`, `estimates`, `expense_categories`, `expenses`, `invoice_item_categories`, `invoices`, `projects`, `roles`, `task_assignments`, `tasks`, `time_entries`, `user_assignments`, `users`+Today (19): `clients`, `contacts`, `estimate_item_categories`, `estimates`, `expense_categories`, `expenses`, `invoice_item_categories`, `invoice_payments`, `invoices`, `projects`, `pto_requests`, `pto_types`, `roles`, `task_assignments`, `tasks`, `time_entries`, `user_assignments`, `user_cost_rates`, `users`Diffed against: <https://help.getharvest.com/api-v2/>-- [ ] `Invoice payments (GET /v2/invoices/{INVOICE_ID}/payments)` — reconcile invoice revenue with actual cash receipts, partial payments, and payment dates (high)-- [ ] `User cost rates (GET /v2/users/{USER_ID}/cost_rates)` — track effective cost rate changes and analyze labor costs, including periods without logged time (medium)+- [x] `Invoice payments (GET /v2/invoices/{INVOICE_ID}/payments)` — reconcile invoice revenue with actual cash receipts, partial payments, and payment dates (high) Shipped as `invoice_payments`, fanned out over `invoices`.+- [x] `User cost rates (GET /v2/users/{USER_ID}/cost_rates)` — track effective cost rate changes and analyze labor costs, including periods without logged time (medium) Shipped as `user_cost_rates`, fanned out over `users` (full refresh only).- [ ] `User billable rates (GET /v2/users/{USER_ID}/billable_rates)` — track billing rate changes and compare historical pricing across users and projects (medium)- [ ] `Invoice messages (GET /v2/invoices/{INVOICE_ID}/messages)` — analyze invoice communication, payment reminders, and collection follow-up activity (medium)- [ ] `Estimate messages (GET /v2/estimates/{estimate_ID}/messages)` — connect estimate follow-up activity with acceptance outcomes and sales cycle duration (medium)-- [ ] `Time off requests (GET /v2/pto/requests)` — analyze approved leave, pending requests, absence patterns, and approval delays (medium)+- [x] `Time off requests (GET /v2/pto/requests)` — analyze approved leave, pending requests, absence patterns, and approval delays (medium) Shipped as `pto_requests`.- [ ] `Time off balances (GET /v2/pto/balances)` — measure accrued, used, pending, and remaining leave by employee and policy (medium)-- [ ] `Time off policies (GET /v2/pto/types)` — identify leave categories and interpret policy allocation and accrual rules (medium)+- [x] `Time off policies (GET /v2/pto/types)` — identify leave categories and interpret policy allocation and accrual rules (medium) Shipped as `pto_types` (active policies, full refresh).- [ ] `Work schedules (GET /v2/pto/work_schedules)` — compare tracked hours and leave against expected working days and capacity (medium)Note: Read the official API index and selected pages, extracting documented GET collection paths. Compared them with fifteen static tables; time entries include applied rates, while rate collections add effective history. Invoice, estimate, and rate collections need parent fan-out; leave collections require appropriate permissions. Excluded writes, settings, duplicate assignment views, and reports derivable from existing detail tables.diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/harvest/canonical_descriptions.py b/products/warehouse_sources/backend/temporal/data_imports/sources/harvest/canonical_descriptions.pyindex fcc92024ab99..e3ed0faa5424 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/harvest/canonical_descriptions.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/harvest/canonical_descriptions.py@@ -122,6 +122,22 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {use_as_expense="Whether the category is used for expenses when generating an invoice.",),},+ "invoice_payments": {+ "description": "A payment recorded against an invoice. An invoice can have several partial payments.",+ "docs_url": "https://help.getharvest.com/api-v2/invoices-api/invoices/invoice-payments/",+ "columns": _columns(+ invoice_id="ID of the invoice the payment was recorded against.",+ amount="The amount of the payment.",+ paid_at="Date and time the payment was made.",+ paid_date="Date the payment was made.",+ recorded_by="Name of the person who recorded the payment.",+ recorded_by_email="Email of the person who recorded the payment.",+ notes="Notes associated with the payment.",+ transaction_id="Card authorization or PayPal transaction ID.",+ payment_gateway="The ID and name of the payment gateway used to process the payment.",+ created_at="Date and time the payment was recorded.",+ ),+ },"invoices": {"description": "An invoice issued to a client, including its line items.","docs_url": "https://help.getharvest.com/api-v2/invoices-api/invoices/invoices/",@@ -158,6 +174,47 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {closed_at="Date and time the invoice was closed.",),},+ "pto_requests": {+ "description": "A time off request and its review status.",+ "docs_url": "https://help.getharvest.com/api-v2/pto-api/pto/time-off-requests/",+ "columns": _columns(+ status="Status of the request: pending, approved, declined, changes_requested, or cancelled.",+ start_date="Start date of the request.",+ end_date="End date of the request.",+ days="Total business days requested.",+ business_days_at_submission="Working days at the time the request was submitted.",+ start_time_minutes="Starting minute offset for a partial-day request.",+ end_time_minutes="Ending minute offset for a partial-day request.",+ notes="Notes from the person who submitted the request.",+ review_reason="Reason given by the manager or administrator who reviewed the request.",+ reviewed_at="Date and time the request was reviewed.",+ user="The ID and name of the user the request is for.",+ pto_type="The time off policy the request is booked against.",+ reviewer="The ID and name of the reviewer, or null.",+ request_days="Daily breakdown of the request, with the hours and day fraction for each date.",+ coverage_warnings="Coworkers on shared projects who are also away on the requested dates.",+ ),+ },+ "pto_types": {+ "description": "A time off policy (leave category) and its accrual rules.",+ "docs_url": "https://help.getharvest.com/api-v2/pto-api/pto/time-off-policies/",+ "columns": _columns(+ name="Name of the time off policy, for example Vacation or Sick Leave.",+ color="Color assigned to the policy in the UI.",+ icon="Icon assigned to the policy in the UI.",+ accrual_type="How the allowance is granted: fixed (front-loaded), accrued (earned over time), or unlimited.",+ accrual_schedule="How often leave accrues: monthly, biweekly, semimonthly, or hourly.",+ accrual_first_payday="Anchor date for biweekly accruals.",+ accrual_leave_hours="Leave hours earned per worked-hours base, for hourly accruals.",+ accrual_worked_hours="Worked-hours base, for hourly accruals.",+ accrual_max_hours_per_year="Maximum accrued hours per year, for hourly accruals.",+ default_days_per_year="Default days per year credited to teammates.",+ max_carryover_days="Maximum days that carry over between years.",+ requires_approval="Whether requests under this policy need manager or administrator review.",+ visible_to_all="Whether every teammate can see and book this policy.",+ is_active="Whether the policy is active, or deactivated.",+ ),+ },"projects": {"description": "A project that time and expenses are tracked against.","docs_url": "https://help.getharvest.com/api-v2/projects-api/projects/projects/",@@ -261,6 +318,16 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {budget="Budget in hours or currency for this user on the project.",),},+ "user_cost_rates": {+ "description": "A cost rate for a user, with the dates it was in effect.",+ "docs_url": "https://help.getharvest.com/api-v2/users-api/users/cost-rates/",+ "columns": _columns(+ user_id="ID of the user the cost rate belongs to.",+ amount="The amount of the cost rate.",+ start_date="Date the cost rate takes effect.",+ end_date="Date the cost rate stops being in effect, calculated by Harvest. Null for the current rate.",+ ),+ },"users": {"description": "A person in the Harvest account.","docs_url": "https://help.getharvest.com/api-v2/users-api/users/users/",diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/harvest/harvest.py b/products/warehouse_sources/backend/temporal/data_imports/sources/harvest/harvest.pyindex 79f941dc370e..235c5cb2be98 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/harvest/harvest.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/harvest/harvest.py@@ -1,15 +1,23 @@import datetimeimport dataclasses-from typing import Any, Optional+from collections.abc import Iterable+from typing import Any, Optional, castfrom products.warehouse_sources.backend.temporal.data_imports.sources.common.http import make_tracked_sessionfrom products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source import (RESTAPIConfig,rest_api_resource,)+from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.fanout import (+ build_dependent_resource,+)from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.paginators import (JSONResponsePaginator,)+from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.typing import (+ ClientConfig,+ IncrementalConfig,+)from products.warehouse_sources.backend.temporal.data_imports.sources.common.resumable import ResumableSourceManagerfrom products.warehouse_sources.backend.temporal.data_imports.sources.common.source_helpers import validate_via_probefrom products.warehouse_sources.backend.temporal.data_imports.sources.common.typings import SourceResponse@@ -27,12 +35,15 @@ HARVEST_USER_AGENT = "PostHog (https://posthog.com)"REQUEST_TIMEOUT = (10.0, 60.0)[email protected](frozen=True)class HarvestResumeConfig:# Absolute URL of the next page, taken from the response's `links.next`. Harvest's own# docs tell integrators to follow that link rather than rebuild page/cursor params, since# it picks whichever pagination style is cheapest for the endpoint.- next_url: str+ next_url: Optional[str] = None+ # Per-parent sub-resources (invoice payments, user cost rates) checkpoint the shape+ # `build_dependent_resource` emits: which parents finished and where the current one stopped.+ fanout_state: Optional[dict[str, Any]] = Nonedef _to_iso8601(value: Any) -> Optional[str]:@@ -77,6 +88,88 @@ def _build_params(endpoint: str, should_use_incremental_field: bool) -> dict[strreturn params+def _client_config(account_id: str, access_token: str, api_version: str) -> ClientConfig:+ return {+ "base_url": f"{HARVEST_BASE_URL}/{api_version}",+ "headers": _build_headers(account_id),+ "auth": {"type": "bearer", "token": access_token},+ # Harvest returns a self-contained absolute next-page URL, which already carries+ # `per_page` and `updated_since`, so the filter survives every page.+ "paginator": JSONResponsePaginator(next_url_path="links.next"),+ # Pagination and resume follow URLs supplied by the response body, so pin them to+ # the API host and refuse redirects - a tampered link must not carry the bearer+ # token somewhere else.+ "allowed_hosts": [HARVEST_API_HOST],+ "allow_redirects": False,+ "request_timeout": REQUEST_TIMEOUT,+ }+++def _updated_since(cursor_path: str) -> IncrementalConfig:+ return {"cursor_path": cursor_path, "start_param": "updated_since", "convert": _to_iso8601}+++def _fanout_source(+ endpoint: str,+ client_config: ClientConfig,+ team_id: int,+ job_id: str,+ resumable_source_manager: ResumableSourceManager[HarvestResumeConfig],+ use_incremental: bool,+ db_incremental_field_last_value: Optional[Any],+) -> SourceResponse:+ config = HARVEST_ENDPOINTS[endpoint]+ assert config.fanout is not None+ parent_config = HARVEST_ENDPOINTS[config.fanout.parent_name]++ initial_state: Optional[dict[str, Any]] = None+ if resumable_source_manager.can_resume():+ resume = resumable_source_manager.load_state()+ if resume is not None:+ initial_state = resume.fanout_state++ def save_checkpoint(state: Optional[dict[str, Any]]) -> None:+ if state:+ resumable_source_manager.save_state(HarvestResumeConfig(fanout_state=state))++ # The parent listing always walks the full collection: a child can change without its+ # parent's `updated_at` moving, so the child's own `updated_since` does the narrowing.+ child = cast(+ Iterable[Any],+ build_dependent_resource(+ endpoint_configs=HARVEST_ENDPOINTS,+ child_endpoint=endpoint,+ fanout=config.fanout,+ client_config=client_config,+ path_format_values={},+ team_id=team_id,+ job_id=job_id,+ db_incremental_field_last_value=db_incremental_field_last_value if use_incremental else None,+ should_use_incremental_field=use_incremental,+ incremental_field="updated_at",+ incremental_config_factory=_updated_since,+ page_size_param="per_page",+ parent_endpoint_extra={"data_selector": parent_config.data_key, "data_selector_required": True},+ child_endpoint_extra={"data_selector": config.data_key, "data_selector_required": True},+ resume_hook=save_checkpoint,+ initial_paginator_state=initial_state,+ ),+ )++ return SourceResponse(+ name=endpoint,+ items=lambda: child,+ primary_keys=config.primary_keys,+ partition_count=1,+ partition_size=1,+ partition_mode="datetime" if config.partition_key else None,+ partition_format="month" if config.partition_key else None,+ partition_keys=[config.partition_key] if config.partition_key else None,+ # Rows arrive one parent at a time, so they are not one ordered series.+ sort_mode=None,+ )++def harvest_source(account_id: str,access_token: str,@@ -90,22 +183,21 @@ def harvest_source() -> SourceResponse:config = HARVEST_ENDPOINTS[endpoint]use_incremental = should_use_incremental_field and config.supports_updated_since+ client_config = _client_config(account_id, access_token, api_version)++ if config.fanout is not None:+ return _fanout_source(+ endpoint,+ client_config,+ team_id,+ job_id,+ resumable_source_manager,+ use_incremental,+ db_incremental_field_last_value,+ )rest_config: RESTAPIConfig = {- "client": {- "base_url": f"{HARVEST_BASE_URL}/{api_version}",- "headers": _build_headers(account_id),- "auth": {"type": "bearer", "token": access_token},- # Harvest returns a self-contained absolute next-page URL, which already carries- # `per_page` and `updated_since`, so the filter survives every page.- "paginator": JSONResponsePaginator(next_url_path="links.next"),- # Pagination and resume follow URLs supplied by the response body, so pin them to- # the API host and refuse redirects - a tampered link must not carry the bearer- # token somewhere else.- "allowed_hosts": [HARVEST_API_HOST],- "allow_redirects": False,- "request_timeout": REQUEST_TIMEOUT,- },+ "client": client_config,"resources": [{"name": endpoint,@@ -124,7 +216,7 @@ def harvest_source(initial_paginator_state: Optional[dict[str, Any]] = Noneif resumable_source_manager.can_resume():resume = resumable_source_manager.load_state()- if resume is not None:+ if resume is not None and resume.next_url:initial_paginator_state = {"next_url": resume.next_url}def save_checkpoint(state: Optional[dict[str, Any]]) -> None:diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/harvest/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/harvest/settings.pyindex 476604772b24..c7073b383583 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/harvest/settings.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/harvest/settings.py@@ -1,6 +1,10 @@from dataclasses import dataclass, fieldfrom typing import Optional+from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.fanout import (+ DependentEndpointConfig,+)+from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.typing import ResponseActionfrom products.warehouse_sources.backend.temporal.data_imports.sources.common.schema import incremental_fieldfrom products.warehouse_sources.backend.types import IncrementalField@@ -28,6 +32,21 @@ class HarvestEndpointConfig:# partitioning only adds overhead.partition_key: Optional[str] = Nonedescription: Optional[str] = None+ # Set for sub-resources that Harvest only lists per parent object (e.g. an invoice's payments).+ fanout: Optional[DependentEndpointConfig] = None++ @property+ def incremental_fields(self) -> list[IncrementalField]:+ return [incremental_field("updated_at")] if self.supports_updated_since else []++ @property+ def default_incremental_field(self) -> Optional[str]:+ return "updated_at" if self.supports_updated_since else None+++# A parent deleted between the parent listing and its child fetch answers 404. Treat it as an+# empty child so one deleted invoice or user doesn't abort the whole fan-out.+_MISSING_PARENT_IGNORE: list[ResponseAction] = [{"status_code": 404, "action": "ignore"}]HARVEST_ENDPOINTS: dict[str, HarvestEndpointConfig] = {@@ -76,6 +95,23 @@ HARVEST_ENDPOINTS: dict[str, HarvestEndpointConfig] = {data_key="invoice_item_categories",description="Categories available for invoice line items.",),+ "invoice_payments": HarvestEndpointConfig(+ name="invoice_payments",+ path="invoices/{invoice_id}/payments",+ data_key="invoice_payments",+ primary_keys=["invoice_id", "id"],+ partition_key="created_at",+ # Payment rows don't carry their invoice, so the parent id is injected as `invoice_id`.+ fanout=DependentEndpointConfig(+ parent_name="invoices",+ resolve_param="invoice_id",+ resolve_field="id",+ include_from_parent=["id"],+ parent_field_renames={"id": "invoice_id"},+ child_response_actions=_MISSING_PARENT_IGNORE,+ ),+ description="Payments recorded against each invoice, including partial payments.",+ ),"invoices": HarvestEndpointConfig(name="invoices",path="invoices",@@ -84,6 +120,22 @@ HARVEST_ENDPOINTS: dict[str, HarvestEndpointConfig] = {partition_key="created_at",description="Invoices issued to clients, including their line items.",),+ "pto_requests": HarvestEndpointConfig(+ name="pto_requests",+ path="pto/requests",+ data_key="pto_requests",+ partition_key="created_at",+ description="Time off requests and their review status. Non-administrators only see their own requests.",+ ),+ "pto_types": HarvestEndpointConfig(+ name="pto_types",+ path="pto/types",+ data_key="pto_types",+ # The listing only returns active policies by default, so an incremental merge would never+ # see a policy get deactivated. The table is tiny, so full refresh keeps it accurate.+ supports_updated_since=False,+ description="Time off policies (leave categories) and their accrual rules. Only active policies are listed.",+ ),"projects": HarvestEndpointConfig(name="projects",path="projects",@@ -123,6 +175,23 @@ HARVEST_ENDPOINTS: dict[str, HarvestEndpointConfig] = {data_key="user_assignments",description="Users assigned to projects, with their rate and budget.",),+ "user_cost_rates": HarvestEndpointConfig(+ name="user_cost_rates",+ path="users/{user_id}/cost_rates",+ data_key="cost_rates",+ primary_keys=["user_id", "id"],+ supports_updated_since=False,+ fanout=DependentEndpointConfig(+ parent_name="users",+ resolve_param="user_id",+ resolve_field="id",+ include_from_parent=["id"],+ parent_field_renames={"id": "user_id"},+ child_response_actions=_MISSING_PARENT_IGNORE,+ ),+ description="Each user's cost rate history, with the dates each rate was in effect. Full refresh "+ "only - the endpoint has no server-side modified-since filter. Needs an administrator.",+ ),"users": HarvestEndpointConfig(name="users",path="users",@@ -137,8 +206,7 @@ ENDPOINTS = tuple(HARVEST_ENDPOINTS.keys())# via `updated_since`. Endpoints without that filter advertise no incremental field, so# `build_endpoint_schemas` marks them full refresh only.INCREMENTAL_FIELDS: dict[str, list[IncrementalField]] = {- name: [incremental_field("updated_at")] if config.supports_updated_since else []- for name, config in HARVEST_ENDPOINTS.items()+ name: config.incremental_fields for name, config in HARVEST_ENDPOINTS.items()}DESCRIPTIONS: dict[str, str] = {diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/harvest/tests/test_harvest.py b/products/warehouse_sources/backend/temporal/data_imports/sources/harvest/tests/test_harvest.pyindex 8a14f402dd0f..00ce763a1ab0 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/harvest/tests/test_harvest.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/harvest/tests/test_harvest.py@@ -6,6 +6,7 @@ from parameterized import parameterizedfrom products.warehouse_sources.backend.temporal.data_imports.sources.common.testing import (ScriptedResponse,SourceDriver,+ route,scripted_network,)from products.warehouse_sources.backend.temporal.data_imports.sources.generated_configs.harvest import (@@ -37,6 +38,9 @@ def _page(return ScriptedResponse(json=body)+_TOP_LEVEL_ENDPOINTS = [name for name in ENDPOINTS if HARVEST_ENDPOINTS[name].fanout is None]++def _driver() -> SourceDriver:return SourceDriver(HarvestSource(), HarvestSourceConfig(account_id="123456", access_token="pat-secret"))@@ -136,7 +140,7 @@ class TestHarvestTransport:assert isinstance(result.raised, Exception)assert "time_entries" in str(result.raised)- @parameterized.expand(ENDPOINTS)+ @parameterized.expand(_TOP_LEVEL_ENDPOINTS)def test_every_endpoint_reads_its_own_envelope_and_path(self, endpoint: str) -> None:config = HARVEST_ENDPOINTS[endpoint]result = _driver().run(endpoint, [_page([{"id": 1}], data_key=config.data_key)])@@ -150,6 +154,92 @@ class TestHarvestTransport:assert response.name == endpointassert response.primary_keys == config.primary_keys+ @parameterized.expand(+ [+ ("invoice_payments", "invoices", "invoice_id", True, None),+ ("invoice_payments", "invoices", "invoice_id", False, None),+ ("user_cost_rates", "users", "user_id", True, datetime.datetime(2026, 5, 4, tzinfo=datetime.UTC)),+ (+ "invoice_payments",+ "invoices",+ "invoice_id",+ True,+ datetime.datetime(2026, 5, 4, 9, 30, tzinfo=datetime.UTC),+ ),+ ]+ )+ def test_fanout_walks_every_parent_and_tags_rows_with_the_parent_id(+ self, endpoint: str, parent: str, parent_key: str, incremental: bool, last_value: Any+ ) -> None:+ config = HARVEST_ENDPOINTS[endpoint]+ parent_data_key = HARVEST_ENDPOINTS[parent].data_key+ parent_next = f"https://{HARVEST_API_HOST}/v2/{parent}?cursor=p2"+ result = _driver().run(+ endpoint,+ route(+ {+ f"/v2/{parent}": [+ _page([{"id": 10}], data_key=parent_data_key, next_url=parent_next),+ _page([{"id": 20}], data_key=parent_data_key),+ ],+ f"/v2/{config.path.format(**{parent_key: 10})}": [_page([{"id": 1}], data_key=config.data_key)],+ f"/v2/{config.path.format(**{parent_key: 20})}": [_page([{"id": 2}], data_key=config.data_key)],+ }+ ),+ should_use_incremental_field=incremental,+ db_incremental_field_last_value=last_value,+ )++ assert result.raised is None+ # Child ids are only unique per parent, so the parent id must land in the row for the key.+ assert sorted((row[parent_key], row["id"]) for row in result.rows) == [(10, 1), (20, 2)]+ response = result.response+ assert response is not None+ assert response.primary_keys == [parent_key, "id"]+ assert response.sort_mode is None+ # Filtering the parent listing would skip children that changed under an unchanged parent.+ parent_requests = [r for r in result.requests if r.path == f"/v2/{parent}"]+ assert [r.param("updated_since") for r in parent_requests] == [None, None]+ child_requests = [r for r in result.requests if r.path != f"/v2/{parent}"]+ expected = "2026-05-04T09:30:00Z" if last_value is not None and config.supports_updated_since else None+ assert [r.param("updated_since") for r in child_requests] == [expected, expected]++ @parameterized.expand([("invoice_payments", "invoices", "invoice_id"), ("user_cost_rates", "users", "user_id")])+ def test_fanout_skips_a_parent_deleted_mid_sync(self, endpoint: str, parent: str, parent_key: str) -> None:+ config = HARVEST_ENDPOINTS[endpoint]+ result = _driver().run(+ endpoint,+ route(+ {+ f"/v2/{parent}": [_page([{"id": 10}, {"id": 20}], data_key=HARVEST_ENDPOINTS[parent].data_key)],+ f"/v2/{config.path.format(**{parent_key: 10})}": [ScriptedResponse(status=404)],+ f"/v2/{config.path.format(**{parent_key: 20})}": [_page([{"id": 2}], data_key=config.data_key)],+ }+ ),+ )++ # One 404 must not abort the fan-out and drop every later parent's rows.+ assert result.raised is None+ assert [(row[parent_key], row["id"]) for row in result.rows] == [(20, 2)]++ def test_fanout_resume_skips_completed_parents(self) -> None:+ result = _driver().run(+ "invoice_payments",+ route(+ {+ "/v2/invoices": [_page([{"id": 10}, {"id": 20}], data_key="invoices")],+ "/v2/invoices/20/payments": [_page([{"id": 2}], data_key="invoice_payments")],+ }+ ),+ resume_state=HarvestResumeConfig(+ fanout_state={"completed": ["invoices/10/payments"], "current": None, "child_state": None}+ ),+ )++ assert result.raised is None+ assert "/v2/invoices/10/payments" not in result.paths+ assert [row["id"] for row in result.rows] == [2]+@parameterized.expand([("time_entries", "created_at"), ("invoices", "created_at"), ("roles", None)])def test_partitioning_matches_the_endpoint_config(self, endpoint: str, partition_key: str | None) -> None:config = HARVEST_ENDPOINTS[endpoint]