:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(data-warehouse): add 3 missing tables to the Wasabi source (#125276)
Tom Owersb54ee5825977
diff --git a/posthog/test/repo_invariants/dataclass_frozen_baseline.txt b/posthog/test/repo_invariants/dataclass_frozen_baseline.txtindex 8491e938bfc3..c547dd5dec09 100644--- a/posthog/test/repo_invariants/dataclass_frozen_baseline.txt+++ b/posthog/test/repo_invariants/dataclass_frozen_baseline.txt@@ -1135,7 +1135,6 @@1 products/warehouse_sources/backend/temporal/data_imports/sources/vercel/settings.py1 products/warehouse_sources/backend/temporal/data_imports/sources/vercel/vercel.py1 products/warehouse_sources/backend/temporal/data_imports/sources/vultr/settings.py-1 products/warehouse_sources/backend/temporal/data_imports/sources/wasabi/settings.py1 products/warehouse_sources/backend/temporal/data_imports/sources/wasabi/wasabi.py1 products/warehouse_sources/backend/temporal/data_imports/sources/watchmode/watchmode.py1 products/warehouse_sources/backend/temporal/data_imports/sources/webflow/settings.pydiff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.mdindex 10c32d2cc83a..dfb8c7bf6af3 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md@@ -13086,9 +13086,9 @@ Today (4): `accounts`, `bucket_utilizations`, `sub_account_invoices`, `utilizatiDiffed against: <https://docs.wasabi.com/llms.txt>-- [ ] `GET /v1/accounts/{AcctNum}/invoices/{SubInvoiceNum}/regional` — per-region cost and usage breakdown of the sub-invoices we already sync; Wasabi explicitly recommends this over the flat detail endpoint (high)-- [ ] `GET /v1/accounts/{AcctNum}/invoices/{SubInvoiceNum}` — sub-invoice line detail (unit cost, resource type); being deprecated in favor of the regional variant but still the only flat line-item view (medium)-- [ ] `GET /v1/accounts/{AcctNum}/air-jobs` — Wasabi AiR job history per sub-account, an activity table with no equivalent today (low)+- [x] `GET /v1/accounts/{AcctNum}/invoices/{SubInvoiceNum}/regional` — per-region cost and usage breakdown of the sub-invoices we already sync; Wasabi explicitly recommends this over the flat detail endpoint (high)+- [x] `GET /v1/accounts/{AcctNum}/invoices/{SubInvoiceNum}` — sub-invoice line detail (unit cost, resource type); being deprecated in favor of the regional variant but still the only flat line-item view (medium)+- [x] `GET /v1/accounts/{AcctNum}/air-jobs` — Wasabi AiR job history per sub-account, an activity table with no equivalent today (low)Note: Coverage is close to complete for the Wasabi Account Control API (partner.wasabisys.com). I verified from the endpoint docs that /v1/utilizations and /v1/utilizations/buckets already return BOTH control and sub-account rows, so /v1/control-account/utilizations, /v1/control-account/bucket-utilizations, /v1/accounts/{acct}/utilizations and /v1/utilizations/status are all redundant with what is already synced and I deliberately excluded them. Wasabi also ships two other API families on different hosts and credentials — the WACM Connect API (api.wacm.wasabisys.com: control accounts, channel accounts, members, standalone accounts, /v1/invoices, /v1/usages) and the Stats API (/v1/standalone/utilizations) — which would be separate sources, not gaps in this one. Wasabi publishes a docs llms.txt plus per-page .md, which is the fastest way to enumerate.diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/wasabi/canonical_descriptions.py b/products/warehouse_sources/backend/temporal/data_imports/sources/wasabi/canonical_descriptions.pyindex 12e1a527e820..db705fa8497d 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/wasabi/canonical_descriptions.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/wasabi/canonical_descriptions.py@@ -28,6 +28,23 @@ _UTILIZATION_METRIC_COLUMNS = {"DeleteBytes": "Bytes deleted in the period.",}+_SUB_INVOICE_ITEM_COLUMNS = {+ "SubInvoiceItemNum": "Unique identifier of the sub-invoice line item.",+ "SubInvoiceNum": "Identifier of the sub-invoice the line item belongs to.",+ "Type": "Resource type the line item charges for.",+ "DisplayName": "Display name of the line item.",+ "Description": "Description of the line item.",+ "Qty": "Billed quantity of the resource.",+ "UnitCost": "Cost per unit of the resource.",+ "Total": "Total amount of the line item.",+ "Currency": "Currency of the line item amounts.",+ "AcctNum": "Account number of the sub-account the sub-invoice belongs to.",+ "InvoiceNum": "Identifier of the Control Account invoice the sub-invoice rolls up into.",+ "CreateTime": "When the sub-invoice was created.",+ "PeriodStart": "Start of the billing period the sub-invoice covers.",+ "PeriodEnd": "End of the billing period the sub-invoice covers.",+}+CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {"accounts": {"description": "Sub-accounts associated with the Wasabi Control Account, with summary profile information.",@@ -82,4 +99,25 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {"Status": "Payment status of the sub-invoice.",},},+ "sub_account_invoice_items": {+ "description": "Line items of each sub-invoice, such as storage and egress charges. Where a sub-account has buckets in regions with different prices, the unit cost is the weighted average across those regions.",+ "docs_url": "https://docs.wasabi.com/apidocs/get-details-for-a-sub-invoice",+ "columns": _SUB_INVOICE_ITEM_COLUMNS,+ },+ "sub_account_invoice_regional_items": {+ "description": "Line items of each sub-invoice, broken down by Wasabi region.",+ "docs_url": "https://docs.wasabi.com/apidocs/get-details-and-regional-usage-for-a-sub-invoice",+ "columns": _SUB_INVOICE_ITEM_COLUMNS,+ },+ "air_jobs": {+ "description": "Finished Wasabi AiR jobs for each sub-account. Jobs that are still in progress are not listed.",+ "docs_url": "https://docs.wasabi.com/apidocs/get-air-jobs-for-a-sub-account",+ "columns": {+ "JobId": "Unique identifier of the AiR job.",+ "AcctNum": "Account number of the sub-account that ran the job.",+ "MediaRuntimeTotalSeconds": "Total runtime of the processed media, in seconds.",+ "CreatedAt": "When the job was created.",+ "FinishedAt": "When the job finished.",+ },+ },}diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/wasabi/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/wasabi/settings.pyindex 41934db971f4..2afa025bb942 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/wasabi/settings.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/wasabi/settings.py@@ -17,15 +17,29 @@ UTILIZATION_WINDOW_DAYS = 30# (the merge on primary keys dedupes the overlap).UTILIZATION_LOOKBACK_SECONDS = 3 * 24 * 60 * 60+# AiR jobs are listed only once finished, and Wasabi doesn't say whether from/to filter on the+# start or the finish time, so re-read a trailing window to pick up jobs that ran across days.+AIR_JOBS_LOOKBACK_SECONDS = 3 * 24 * 60 * 60++# Wasabi documents no maximum page size for AiR jobs.+AIR_JOBS_PAGE_SIZE = 100+START_TIME_INCREMENTAL: IncrementalField = incremental_field("StartTime", IncrementalFieldType.DateTime)+FINISHED_AT_INCREMENTAL: IncrementalField = incremental_field("FinishedAt", IncrementalFieldType.DateTime)ACCOUNTS = "accounts"UTILIZATIONS = "utilizations"BUCKET_UTILIZATIONS = "bucket_utilizations"SUB_ACCOUNT_INVOICES = "sub_account_invoices"+SUB_ACCOUNT_INVOICE_ITEMS = "sub_account_invoice_items"+SUB_ACCOUNT_INVOICE_REGIONAL_ITEMS = "sub_account_invoice_regional_items"+AIR_JOBS = "air_jobs"++# Sub-invoice line items carry only SubInvoiceNum, so the account and billing period come from the parent sub-invoice.+SUB_INVOICE_PARENT_FIELDS = ["AcctNum", "InvoiceNum", "CreateTime", "PeriodStart", "PeriodEnd"]-@dataclass+@dataclass(frozen=True)class WasabiEndpointConfig:name: strpath: str@@ -34,8 +48,13 @@ class WasabiEndpointConfig:incremental_fields: list[IncrementalField] = field(default_factory=list)# Endpoints that accept from/to date filters are walked in ascending date windows.date_windowed: bool = False- # Fan-out endpoints iterate this parent endpoint and resolve {acct_num} per parent row.+ # Fan-out endpoints iterate this parent endpoint and resolve each path param per parent row.parent: str | None = None+ resolve_fields: dict[str, str] = field(default_factory=lambda: {"acct_num": "AcctNum"})+ include_from_parent: list[str] = field(default_factory=list)+ data_selector: str = "$"+ # Endpoints paged by a 0-based pageNumber query param.+ page_size: int | None = NoneWASABI_ENDPOINTS: dict[str, WasabiEndpointConfig] = {@@ -72,6 +91,37 @@ WASABI_ENDPOINTS: dict[str, WasabiEndpointConfig] = {partition_key="CreateTime",parent=ACCOUNTS,),+ SUB_ACCOUNT_INVOICE_ITEMS: WasabiEndpointConfig(+ name=SUB_ACCOUNT_INVOICE_ITEMS,+ path="/v1/accounts/{acct_num}/invoices/{sub_invoice_num}",+ primary_keys=["AcctNum", "SubInvoiceNum", "SubInvoiceItemNum"],+ partition_key="CreateTime",+ parent=SUB_ACCOUNT_INVOICES,+ resolve_fields={"acct_num": "AcctNum", "sub_invoice_num": "SubInvoiceNum"},+ include_from_parent=SUB_INVOICE_PARENT_FIELDS,+ data_selector="SubInvoiceItems",+ ),+ SUB_ACCOUNT_INVOICE_REGIONAL_ITEMS: WasabiEndpointConfig(+ name=SUB_ACCOUNT_INVOICE_REGIONAL_ITEMS,+ path="/v1/accounts/{acct_num}/invoices/{sub_invoice_num}/regional",+ primary_keys=["AcctNum", "SubInvoiceNum", "SubInvoiceItemNum"],+ partition_key="CreateTime",+ parent=SUB_ACCOUNT_INVOICES,+ resolve_fields={"acct_num": "AcctNum", "sub_invoice_num": "SubInvoiceNum"},+ include_from_parent=SUB_INVOICE_PARENT_FIELDS,+ data_selector="SubInvoiceItems",+ ),+ AIR_JOBS: WasabiEndpointConfig(+ name=AIR_JOBS,+ path="/v1/accounts/{acct_num}/air-jobs",+ primary_keys=["AcctNum", "JobId"],+ partition_key="CreatedAt",+ incremental_fields=[FINISHED_AT_INCREMENTAL],+ parent=ACCOUNTS,+ # Wasabi documents the body as an array of {Pagination, Records} pages; also accept a bare object.+ data_selector="$..Records[*]",+ page_size=AIR_JOBS_PAGE_SIZE,+ ),}ENDPOINTS = tuple(WASABI_ENDPOINTS)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/wasabi/source.py b/products/warehouse_sources/backend/temporal/data_imports/sources/wasabi/source.pyindex 6361b7342e57..ac80b3c8bbb5 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/wasabi/source.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/wasabi/source.py@@ -21,6 +21,8 @@ from products.warehouse_sources.backend.temporal.data_imports.sources.common.schfrom products.warehouse_sources.backend.temporal.data_imports.sources.common.typings import SourceInputs, SourceResponsefrom products.warehouse_sources.backend.temporal.data_imports.sources.generated_configs.wasabi import WasabiSourceConfigfrom products.warehouse_sources.backend.temporal.data_imports.sources.wasabi.settings import (+ AIR_JOBS,+ AIR_JOBS_LOOKBACK_SECONDS,BUCKET_UTILIZATIONS,ENDPOINTS,INCREMENTAL_FIELDS,@@ -69,17 +71,19 @@ class WasabiSource(ResumableSource[WasabiSourceConfig, WasabiResumeConfig]):force_refresh: bool = False,api_version: str | None = None,) -> list[SourceSchema]:- # The date-window walk re-reads a boundary day each run, so append syncs would- # duplicate rows — utilization endpoints are merge-only.+ # The date-window walk and the AiR jobs date filter re-read a boundary day each run,+ # so append syncs would duplicate rows — these endpoints are merge-only.schemas = build_endpoint_schemas(ENDPOINTS,INCREMENTAL_FIELDS,names,- merge_only=(UTILIZATIONS, BUCKET_UTILIZATIONS),+ merge_only=(UTILIZATIONS, BUCKET_UTILIZATIONS, AIR_JOBS),)for schema in schemas:if schema.name in (UTILIZATIONS, BUCKET_UTILIZATIONS):schema.default_incremental_lookback_seconds = UTILIZATION_LOOKBACK_SECONDS+ elif schema.name == AIR_JOBS:+ schema.default_incremental_lookback_seconds = AIR_JOBS_LOOKBACK_SECONDSreturn schemasdef validate_credentials(diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/wasabi/tests/test_wasabi.py b/products/warehouse_sources/backend/temporal/data_imports/sources/wasabi/tests/test_wasabi.pyindex 278081b368e4..abbaabdb385f 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/wasabi/tests/test_wasabi.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/wasabi/tests/test_wasabi.py@@ -1,6 +1,7 @@from datetime import UTC, date, datetime, timedeltafrom typing import Any, cast+import time_machinefrom unittest import mockfrom parameterized import parameterized@@ -10,6 +11,7 @@ from products.warehouse_sources.backend.temporal.data_imports.sources.common.tesScriptedResponse,SourceDriver,always,+ route,)from products.warehouse_sources.backend.temporal.data_imports.sources.generated_configs.wasabi import WasabiSourceConfigfrom products.warehouse_sources.backend.temporal.data_imports.sources.wasabi.settings import (@@ -224,3 +226,141 @@ class TestWasabiSourceTransport:"field": "AcctNum",}assert child_resource["write_disposition"] == "replace"+++_SUB_INVOICE_PARENT = {+ "AcctNum": 101,+ "InvoiceNum": 900,+ "CreateTime": "2026-02-01T00:00:00Z",+ "PeriodStart": "2026-01-01T00:00:00Z",+ "PeriodEnd": "2026-01-31T23:59:59Z",+}+++def _sub_invoice_item(item_num: int, sub_invoice_num: int) -> dict[str, Any]:+ return {+ "SubInvoiceItemNum": item_num,+ "SubInvoiceNum": sub_invoice_num,+ "Type": "storage",+ "DisplayName": "Timed active storage",+ "Qty": 1.5,+ "UnitCost": 6.99,+ "Total": 10.49,+ "Currency": "USD",+ }+++def _air_job(job_id: str) -> dict[str, Any]:+ return {+ "JobId": job_id,+ "AcctNum": 101,+ "MediaRuntimeTotalSeconds": 120,+ "CreatedAt": "2026-03-05 10:00:00",+ "FinishedAt": "2026-03-05 10:05:00",+ }+++class TestWasabiFanOutEndpoints:+ @parameterized.expand(+ [+ ("details", "sub_account_invoice_items", ""),+ ("regional", "sub_account_invoice_regional_items", "/regional"),+ ]+ )+ def test_sub_invoice_items_walk_accounts_then_sub_invoices(self, _name: str, schema: str, suffix: str) -> None:+ result = _driver().run(+ schema,+ route(+ {+ "/v1/accounts": [ScriptedResponse(json=[{"AcctNum": 101}, {"AcctNum": 102}])],+ "/v1/accounts/101/invoices": [+ ScriptedResponse(json=[{**_SUB_INVOICE_PARENT, "SubInvoiceNum": 5001}])+ ],+ "/v1/accounts/102/invoices": [+ ScriptedResponse(json=[{**_SUB_INVOICE_PARENT, "AcctNum": 102, "SubInvoiceNum": 5002}])+ ],+ f"/v1/accounts/101/invoices/5001{suffix}": [+ ScriptedResponse(+ json={+ "SubInvoice": {**_SUB_INVOICE_PARENT, "SubInvoiceNum": 5001},+ "SubInvoiceItems": [_sub_invoice_item(1, 5001), _sub_invoice_item(2, 5001)],+ }+ )+ ],+ # A sub-invoice removed between its listing and the detail fetch.+ f"/v1/accounts/102/invoices/5002{suffix}": [+ ScriptedResponse(status=404, json={"Msg": "Not Found"})+ ],+ }+ ),+ )++ assert result.raised is None+ assert result.rows == [+ {**_sub_invoice_item(1, 5001), **_SUB_INVOICE_PARENT},+ {**_sub_invoice_item(2, 5001), **_SUB_INVOICE_PARENT},+ ]++ @parameterized.expand(+ [+ ("full_refresh", None, None, None),+ ("incremental", "2026-03-05 10:05:00", "2026-03-05", (TODAY + timedelta(days=1)).isoformat()),+ ]+ )+ def test_air_jobs_pages_each_account_until_an_empty_page(+ self, _name: str, last_value: str | None, expected_from: str | None, expected_to: str | None+ ) -> None:+ with time_machine.travel(datetime.combine(TODAY, datetime.min.time(), UTC), tick=False):+ result = _driver().run(+ "air_jobs",+ route(+ {+ "/v1/accounts": [ScriptedResponse(json=[{"AcctNum": 101}, {"AcctNum": 102}])],+ "/v1/accounts/101/air-jobs": [+ ScriptedResponse(+ json=[+ {+ "Pagination": {+ "PageSize": 100,+ "PageNumber": 0,+ "TotalRecords": 2,+ "TotalPages": 2,+ },+ "Records": [_air_job("job-1")],+ }+ ]+ ),+ # Wasabi reports TotalPages as -1 on every page after the first.+ ScriptedResponse(+ json=[+ {+ "Pagination": {+ "PageSize": 100,+ "PageNumber": 1,+ "TotalRecords": -1,+ "TotalPages": -1,+ },+ "Records": [_air_job("job-2")],+ }+ ]+ ),+ ScriptedResponse(json=[{"Pagination": {}, "Records": []}]),+ ],+ # A bare object body is accepted too.+ "/v1/accounts/102/air-jobs": [ScriptedResponse(json={"Pagination": {}, "Records": []})],+ }+ ),+ should_use_incremental_field=last_value is not None,+ incremental_field="FinishedAt",+ db_incremental_field_last_value=last_value,+ )++ assert result.raised is None+ assert result.rows == [_air_job("job-1"), _air_job("job-2")]+ assert result.response is not None+ # Rows restart at each account, so the run is not one ascending series.+ assert result.response.sort_mode is None+ job_requests = [r for r in result.requests if r.path.endswith("/air-jobs")]+ assert [r.param("pageNumber") for r in job_requests] == ["0", "1", "2", "0"]+ assert {r.param("from") for r in job_requests} == {expected_from}+ assert {r.param("to") for r in job_requests} == {expected_to}diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/wasabi/wasabi.py b/products/warehouse_sources/backend/temporal/data_imports/sources/wasabi/wasabi.pyindex 9234201763e9..f8620385eafe 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/wasabi/wasabi.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/wasabi/wasabi.py@@ -14,12 +14,17 @@ from products.warehouse_sources.backend.temporal.data_imports.sources.common.resrest_api_resource,rest_api_resources,)+from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.fanout import (+ build_chained_resource,+)from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.paginators import (BasePaginator,+ PageNumberPaginator,SinglePagePaginator,)from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.typing import (ClientConfig,+ Endpoint,EndpointResource,)from products.warehouse_sources.backend.temporal.data_imports.sources.common.resumable import ResumableSourceManager@@ -117,19 +122,101 @@ def _client_config(api_key: str) -> ClientConfig:}-def _make_source_response(endpoint_config: WasabiEndpointConfig, resource: Any) -> SourceResponse:+def _make_source_response(+ endpoint_config: WasabiEndpointConfig, resource: Any, supports_resume: bool = True+) -> SourceResponse:return SourceResponse(name=endpoint_config.name,items=lambda: resource,primary_keys=endpoint_config.primary_keys,# Date windows walk oldest-first, so the per-batch max of the incremental field- # advances monotonically across batches.- sort_mode="asc",+ # advances monotonically across batches. Fan-out rows restart per account.+ sort_mode="asc" if endpoint_config.date_windowed else None,partition_count=1,partition_size=1,partition_mode="datetime" if endpoint_config.partition_key else None,partition_format="month" if endpoint_config.partition_key else None,partition_keys=[endpoint_config.partition_key] if endpoint_config.partition_key else None,+ supports_resume=supports_resume,+ )+++def _paginator(endpoint_config: WasabiEndpointConfig) -> BasePaginator:+ if endpoint_config.page_size is not None:+ # TotalPages is only returned on page 0 (later pages report -1), so stop on an empty page instead.+ return PageNumberPaginator(base_page=0, page_param="pageNumber")+ return SinglePagePaginator()+++def _child_endpoint(+ endpoint_config: WasabiEndpointConfig,+ parent_name: str,+ should_use_incremental_field: bool,+ db_incremental_field_last_value: Optional[Any],+) -> Endpoint:+ params: dict[str, Any] = {+ param: {"type": "resolve", "resource": parent_name, "field": field_name}+ for param, field_name in endpoint_config.resolve_fields.items()+ }+ if endpoint_config.page_size is not None:+ params["pageSize"] = endpoint_config.page_size+ params["sortOrder"] = "asc"+ if endpoint_config.incremental_fields and should_use_incremental_field:+ params["from"] = _start_date(should_use_incremental_field, db_incremental_field_last_value).isoformat()+ # `to` is exclusive, so tomorrow keeps jobs that finished today.+ params["to"] = (datetime.now(UTC).date() + timedelta(days=1)).isoformat()+ return {+ "path": endpoint_config.path,+ "params": params,+ "data_selector": endpoint_config.data_selector,+ "paginator": _paginator(endpoint_config),+ }+++def _parent_resource(parent_config: WasabiEndpointConfig, endpoint: Endpoint) -> EndpointResource:+ return {+ "name": parent_config.name,+ "table_name": parent_config.name,+ "write_disposition": "replace",+ "endpoint": endpoint,+ "table_format": "delta",+ }+++def _chained_resource(+ endpoint_config: WasabiEndpointConfig,+ parent_config: WasabiEndpointConfig,+ api_key: str,+ team_id: int,+ job_id: str,+) -> Any:+ """Build the accounts -> sub-invoices -> sub-invoice detail chain and return the detail resource."""+ assert parent_config.parent is not None+ root_config = WASABI_ENDPOINTS[parent_config.parent]+ root_resource = _parent_resource(+ root_config,+ {"path": root_config.path, "data_selector": "$", "paginator": SinglePagePaginator()},+ )+ middle_resource = _parent_resource(parent_config, _child_endpoint(parent_config, root_config.name, False, None))+ child_endpoint = _child_endpoint(endpoint_config, parent_config.name, False, None)+ # A sub-invoice or sub-account removed between its listing and this fetch must not sink the run.+ child_endpoint["response_actions"] = [{"status_code": 404, "action": "ignore"}]+ child_resource: EndpointResource = {+ "name": endpoint_config.name,+ "table_name": endpoint_config.name,+ "write_disposition": "replace",+ "include_from_parent": endpoint_config.include_from_parent,+ "endpoint": child_endpoint,+ "table_format": "delta",+ }+ return build_chained_resource(+ resources=[root_resource, middle_resource, child_resource],+ child_name=endpoint_config.name,+ parent_name=parent_config.name,+ parent_field_renames={field_name: field_name for field_name in endpoint_config.include_from_parent},+ client_config=_client_config(api_key),+ team_id=team_id,+ job_id=job_id,)@@ -157,32 +244,30 @@ def wasabi_source(if endpoint_config.parent is not None:parent_config = WASABI_ENDPOINTS[endpoint_config.parent]+ if parent_config.parent is not None:+ # The chained walk saves no checkpoints, so a restart would repeat it from the start.+ return _make_source_response(+ endpoint_config,+ _chained_resource(endpoint_config, parent_config, api_key, team_id, job_id),+ supports_resume=False,+ )+# Fan-out: iterate the sub-account list and fetch the child endpoint per account.# Child rows natively carry AcctNum, so no parent-field injection is needed.resources: list[str | EndpointResource] = [- {- "name": parent_config.name,- "table_name": parent_config.name,- "write_disposition": "replace",- "endpoint": {- "path": parent_config.path,- "data_selector": "$",- "paginator": SinglePagePaginator(),- },- "table_format": "delta",- },+ _parent_resource(+ parent_config,+ {"path": parent_config.path, "data_selector": "$", "paginator": SinglePagePaginator()},+ ),{"name": endpoint_config.name,"table_name": endpoint_config.name,- "write_disposition": "replace",- "endpoint": {- "path": endpoint_config.path,- "params": {- "acct_num": {"type": "resolve", "resource": parent_config.name, "field": "AcctNum"},- },- "data_selector": "$",- "paginator": SinglePagePaginator(),- },+ "write_disposition": {"disposition": "merge", "strategy": "upsert"}+ if should_use_incremental_field+ else "replace",+ "endpoint": _child_endpoint(+ endpoint_config, parent_config.name, should_use_incremental_field, db_incremental_field_last_value+ ),"table_format": "delta",},]