:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(data-warehouse): add 3 missing tables to the Cisco Meraki source (#124451)
Tom Owersf8c0879d64c2
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.mdindex 5e657c88ee4e..847b2db287d9 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md@@ -2561,10 +2561,10 @@ Diffed against: <https://raw.githubusercontent.com/meraki/openapi/master/openapi- [x] `Network clients (GET /api/v1/networks/{networkId}/clients)` — measure active clients and join client activity with imported networks and devices (high)- [x] `Network events (GET /api/v1/networks/{networkId}/events)` — analyze connectivity failures, authentication events, and operational history (high)-- [ ] `Client usage histories (GET /api/v1/networks/{networkId}/clients/usageHistories)` — measure bandwidth consumption by client over time (high)-- [ ] `Uplink loss and latency history (GET /api/v1/devices/{serial}/lossAndLatencyHistory)` — measure packet loss and latency beyond the current uplink status (high)-- [ ] `Device statuses (GET /api/v1/organizations/{organizationId}/devices/statuses)` — monitor device availability and last contact across the imported inventory (medium)-- [ ] `Uplink usage history (GET /api/v1/networks/{networkId}/appliance/uplinks/usageHistory)` — track sent and received traffic for each network uplink (medium)+- [x] `Client usage histories (GET /api/v1/networks/{networkId}/clients/usageHistories)` — measure bandwidth consumption by client over time (high) → `client_usage_histories` (client keys come from the network clients listing, in batches of 100)+- [ ] `Uplink loss and latency history (GET /api/v1/devices/{serial}/lossAndLatencyHistory)` — measure packet loss and latency beyond the current uplink status (high) — skipped: each request needs one device, one uplink, and a required destination IP, so a table needs a device × uplink × destination fan-out+- [x] `Device statuses (GET /api/v1/organizations/{organizationId}/devices/statuses)` — monitor device availability and last contact across the imported inventory (medium) → `device_availabilities` (implemented against the non-deprecated `GET /organizations/{organizationId}/devices/availabilities`; the spec marks `devices/statuses` deprecated)+- [x] `Uplink usage history (GET /api/v1/networks/{networkId}/appliance/uplinks/usageHistory)` — track sent and received traffic for each network uplink (medium) → `uplink_usage_history`- [ ] `Network traffic (GET /api/v1/networks/{networkId}/traffic)` — identify applications that drive network traffic and capacity requirements (medium)- [ ] `Wireless client connection statistics (GET /api/v1/networks/{networkId}/wireless/clients/connectionStats)` — compare connection failures and wireless reliability across clients (medium)- [ ] `Organization security events (GET /api/v1/organizations/{organizationId}/appliance/security/events)` — measure security threats and affected networks over time (medium)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/cisco_meraki/canonical_descriptions.py b/products/warehouse_sources/backend/temporal/data_imports/sources/cisco_meraki/canonical_descriptions.pyindex 7bdea351dd69..6113ea150d06 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/cisco_meraki/canonical_descriptions.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/cisco_meraki/canonical_descriptions.py@@ -100,4 +100,40 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {"eventData": "Additional data about the event.",},},+ "device_availabilities": {+ "description": "Availability status of each device in the selected Meraki organization. Meraki updates this data every 5 minutes.",+ "docs_url": "https://developer.cisco.com/meraki/api-v1/get-organization-devices-availabilities/",+ "columns": {+ "serial": "Serial number that identifies the device.",+ "mac": "MAC address of the device.",+ "name": "Name of the device.",+ "network": "Network that contains the device.",+ "productType": "Product type of the device.",+ "status": "Status of the device: online, alerting, offline, or dormant.",+ "tags": "Tags assigned to the device.",+ },+ },+ "uplink_usage_history": {+ "description": "Data sent and received by each uplink of the security appliance networks in the selected Meraki organization, for each hour.",+ "docs_url": "https://developer.cisco.com/meraki/api-v1/get-network-appliance-uplinks-usage-history/",+ "columns": {+ "networkId": "Identifier of the network.",+ "startTime": "Start time of the interval.",+ "endTime": "End time of the interval.",+ "byInterface": "Data sent and received by each uplink interface in the interval, in bytes.",+ },+ },+ "client_usage_histories": {+ "description": "Data sent and received by each client of each network in the selected Meraki organization, for each day.",+ "docs_url": "https://developer.cisco.com/meraki/api-v1/get-network-clients-usage-histories/",+ "columns": {+ "networkId": "Identifier of the network that the client used.",+ "clientId": "Identifier of the client. It is unique within the network.",+ "clientIp": "IP address of the client.",+ "clientMac": "MAC address of the client.",+ "ts": "Start of the day that the usage covers.",+ "sent": "Data that the client sent on the day, in kilobytes.",+ "received": "Data that the client received on the day, in kilobytes.",+ },+ },}diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/cisco_meraki/cisco_meraki.py b/products/warehouse_sources/backend/temporal/data_imports/sources/cisco_meraki/cisco_meraki.pyindex 65b31e595bc4..c39872f10a48 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/cisco_meraki/cisco_meraki.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/cisco_meraki/cisco_meraki.py@@ -1,7 +1,9 @@import reimport json+import mathimport hashlib-from datetime import datetime+from collections.abc import Iterator+from datetime import UTC, datetimefrom typing import TYPE_CHECKING, Any, castfrom urllib.parse import parse_qs, urlsplit@@ -11,6 +13,9 @@ from posthog.dataclasses import frozenfrom products.warehouse_sources.backend.temporal.data_imports.sources.cisco_meraki.settings import (AUTH_ERROR,+ CLIENT_USAGE_BATCH_SIZE,+ CLIENT_USAGE_MIN_TIMESPAN_SECONDS,+ CLIENTS_TIMESPAN_SECONDS,ENDPOINTS,EVENT_LOG_FLOOR,EVENT_PRODUCT_TYPES,@@ -19,6 +24,8 @@ from products.warehouse_sources.backend.temporal.data_imports.sources.cisco_meraPERMISSION_ERROR,REGION_ERROR,REGION_HOSTS,+ UPLINK_USAGE_MAX_TIMESPAN_SECONDS,+ UPLINK_USAGE_MIN_TIMESPAN_SECONDS,MerakiEndpoint,)from products.warehouse_sources.backend.temporal.data_imports.sources.common.datetime_utils import parse_datetime_value@@ -189,6 +196,19 @@ def _network_product_types(network: dict[str, Any]) -> list[dict[str, Any]]:]+def _appliance_networks(network: dict[str, Any]) -> list[dict[str, Any]]:+ # Only security appliance networks have uplink usage. Other networks reject the request.+ return [network] if "appliance" in (network.get("productTypes") or []) else []+++def _timespan_since(last_value: Any, minimum: int, maximum: int) -> int:+ since = parse_datetime_value(last_value) if last_value is not None else None+ if since is None:+ return maximum+ elapsed = math.ceil((datetime.now(UTC) - since).total_seconds())+ return max(minimum, min(maximum, elapsed))++def _with_event_id(event: dict[str, Any]) -> dict[str, Any]:content = json.dumps(event, sort_keys=True, separators=(",", ":"), default=str)return {"id": hashlib.sha256(content.encode()).hexdigest(), **event}@@ -223,11 +243,20 @@ def _network_source()is_event_log = endpoint.name == "network_events"- incremental = is_event_log and should_use_incremental_field- stop_before = parse_datetime_value(db_incremental_field_last_value) if incremental else None+ is_uplink_usage = endpoint.name == "uplink_usage_history"+ incremental = bool(endpoint.incremental_fields) and should_use_incremental_field+ last_value = db_incremental_field_last_value if incremental else Nonechild_endpoint_extra: dict[str, Any] = {"data_selector": endpoint.data_selector}if is_event_log:- child_endpoint_extra["paginator"] = _EventLogPaginator(stop_before)+ child_endpoint_extra["paginator"] = _EventLogPaginator(parse_datetime_value(last_value))+ if not endpoint.paginated:+ child_endpoint_extra["paginator"] = "single_page"+ child_params_extra = (+ {"timespan": _timespan_since(last_value, UPLINK_USAGE_MIN_TIMESPAN_SECONDS, UPLINK_USAGE_MAX_TIMESPAN_SECONDS)}+ if is_uplink_usage+ else None+ )+ parent_data_map = _network_product_types if is_event_log else _appliance_networks if is_uplink_usage else Noneresource = cast(Resource,@@ -241,12 +270,13 @@ def _network_source(job_id=job_id,db_incremental_field_last_value=None,should_use_incremental_field=incremental,- # The event log has no time filter, so the paginator bounds an incremental read instead.+ # The event log paginator and the uplink usage timespan bound an incremental read instead.incremental_config_factory=lambda _field: None,parent_endpoint_extra={"data_selector": "$"},child_endpoint_extra=cast(Any, child_endpoint_extra),- parent_data_map=_network_product_types if is_event_log else None,- page_size_param="perPage",+ child_params_extra=child_params_extra,+ parent_data_map=parent_data_map,+ page_size_param="perPage" if endpoint.paginated else None,resume_hook=save_checkpoint,initial_paginator_state=initial_state,),@@ -260,10 +290,111 @@ def _network_source(primary_keys=list(endpoint.primary_keys),on_complete=manager.clear_state,# Each network's event log arrives newest first, so the watermark advances only when the sync completes.- sort_mode="desc" if is_event_log else "asc",- partition_mode="datetime" if is_event_log else None,- partition_format="month" if is_event_log else None,- partition_keys=["occurredAt"] if is_event_log else None,+ # Uplink usage arrives as one series for each network, so it has no order across the table.+ sort_mode="desc" if is_event_log else None if is_uplink_usage else "asc",+ **_partitioning(endpoint),+ )+++def _partitioning(endpoint: MerakiEndpoint) -> dict[str, Any]:+ if endpoint.partition_key is None:+ return {}+ return {"partition_mode": "datetime", "partition_format": "month", "partition_keys": [endpoint.partition_key]}+++def _usage_history_rows(network_id: str, histories: list[Any]) -> list[dict[str, Any]]:+ return [+ {+ "networkId": network_id,+ "clientId": history.get("clientId"),+ "clientIp": history.get("clientIp"),+ "clientMac": history.get("clientMac"),+ **usage,+ }+ for history in histories+ if isinstance(history, dict)+ for usage in history.get("usageHistory") or []+ if isinstance(usage, dict)+ ]+++def _client_usage_pages(+ config: "CiscoMerakiSourceConfig",+ endpoint: MerakiEndpoint,+ api_version: str,+ manager: "ResumableSourceManager[CiscoMerakiResumeConfig]",+ timespan: int,+) -> Iterator[list[dict[str, Any]]]:+ """Read usage histories for the clients of each network, a batch of client keys at a time.++ The endpoint returns usage only for the clients that a request names, so the network clients+ listing supplies the keys. The resume state is the list of networks that are complete.+ """+ settings = client_config(config, api_version)+ client = RESTClient(+ base_url=settings["base_url"],+ auth=APIKeyAuth(api_key=config.api_key, name="X-Cisco-Meraki-API-Key"),+ paginator=HeaderLinkPaginator(),+ session=settings["session"],+ request_timeout=settings["request_timeout"],+ )+ resume = manager.load_state() if manager.can_resume() else None+ completed = list(resume.completed_paths or []) if resume is not None else []+ networks = ENDPOINTS["networks"]++ for network_page in client.paginate(+ _organization_path(networks.path, config), params={"perPage": networks.page_size}+ ):+ manager.safe_point()+ for network in network_page:+ network_id = str(network["id"])+ path = endpoint.path.replace("{networkId}", network_id)+ if path in completed:+ continue+ manager.save_state(CiscoMerakiResumeConfig(completed_paths=list(completed)))+ manager.safe_point()+ for client_page in client.paginate(+ f"networks/{network_id}/clients", params={"perPage": endpoint.page_size, "timespan": timespan}+ ):+ keys = [str(row["id"]) for row in client_page if isinstance(row, dict) and row.get("id")]+ if not keys:+ manager.safe_point()+ for start in range(0, len(keys), CLIENT_USAGE_BATCH_SIZE):+ params = {+ "clients": ",".join(keys[start : start + CLIENT_USAGE_BATCH_SIZE]),+ "perPage": endpoint.page_size,+ "timespan": timespan,+ }+ for history_page in client.paginate(path, params=params):+ rows = _usage_history_rows(network_id, history_page)+ if rows:+ yield rows+ else:+ manager.safe_point()+ completed.append(path)+++def _client_usage_source(+ config: "CiscoMerakiSourceConfig",+ endpoint: MerakiEndpoint,+ api_version: str,+ manager: "ResumableSourceManager[CiscoMerakiResumeConfig]",+ should_use_incremental_field: bool,+ db_incremental_field_last_value: Any,+) -> SourceResponse:+ timespan = _timespan_since(+ db_incremental_field_last_value if should_use_incremental_field else None,+ CLIENT_USAGE_MIN_TIMESPAN_SECONDS,+ CLIENTS_TIMESPAN_SECONDS,+ )+ return SourceResponse(+ name=endpoint.name,+ items=lambda: _client_usage_pages(config, endpoint, api_version, manager, timespan),+ primary_keys=list(endpoint.primary_keys),+ on_complete=manager.clear_state,+ # Usage arrives as one series for each batch of clients, so it has no order across the table.+ sort_mode=None,+ **_partitioning(endpoint),)@@ -278,6 +409,10 @@ def cisco_meraki_source(db_incremental_field_last_value: Any = None,) -> SourceResponse:endpoint = schema_for_resource(ENDPOINTS, endpoint_name)+ if endpoint.name == "client_usage_histories":+ return _client_usage_source(+ config, endpoint, api_version, manager, should_use_incremental_field, db_incremental_field_last_value+ )if endpoint.fanout is not None:return _network_source(config,diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/cisco_meraki/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/cisco_meraki/settings.pyindex 6a5000cfc0ca..5983b59fd4b5 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/cisco_meraki/settings.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/cisco_meraki/settings.py@@ -19,6 +19,19 @@ REGION_HOSTS = {# The maximum lookback that the network clients endpoint accepts (31 days).CLIENTS_TIMESPAN_SECONDS = 31 * 24 * 60 * 60+# Client usage histories hold one row per client per day. An incremental read always covers at least one day, so+# it reads the current day again.+CLIENT_USAGE_MIN_TIMESPAN_SECONDS = 24 * 60 * 60++# The uplink usage history endpoint accepts a lookback of 30 days at most.+UPLINK_USAGE_MAX_TIMESPAN_SECONDS = 30 * 24 * 60 * 60+UPLINK_USAGE_RESOLUTION_SECONDS = 60 * 60+# An incremental read covers at least two intervals, so it reads the current partial hour again.+UPLINK_USAGE_MIN_TIMESPAN_SECONDS = 2 * UPLINK_USAGE_RESOLUTION_SECONDS++# Client keys go into the query string, so a request names a bounded number of clients.+CLIENT_USAGE_BATCH_SIZE = 100+# Product types that the network events endpoint accepts. A network can contain others, such as sensor.EVENT_PRODUCT_TYPES = frozenset({@@ -51,6 +64,17 @@ class MerakiEndpoint:fanout: DependentEndpointConfig | None = Noneincremental_fields: list[IncrementalField] = field(default_factory=list)default_incremental_field: str | None = None+ paginated: bool = True+ partition_key: str | None = None+++def _datetime_field(name: str) -> IncrementalField:+ return {+ "label": name,+ "type": IncrementalFieldType.DateTime,+ "field": name,+ "field_type": IncrementalFieldType.DateTime,+ }_NETWORK_FANOUT_PARENT = "networks"@@ -74,6 +98,11 @@ ENDPOINTS = {primary_keys=("id",),page_size=300,),+ MerakiEndpoint(+ name="device_availabilities",+ path="organizations/{organization_id}/devices/availabilities",+ primary_keys=("serial",),+ ),MerakiEndpoint(name="network_clients",path="networks/{networkId}/clients",@@ -105,15 +134,43 @@ ENDPOINTS = {"productType": {"type": "resolve", "resource": _NETWORK_FANOUT_PARENT, "field": "productType"}},),- incremental_fields=[- {- "label": "occurredAt",- "type": IncrementalFieldType.DateTime,- "field": "occurredAt",- "field_type": IncrementalFieldType.DateTime,- }- ],+ incremental_fields=[_datetime_field("occurredAt")],default_incremental_field="occurredAt",+ partition_key="occurredAt",+ ),+ MerakiEndpoint(+ name="uplink_usage_history",+ path="networks/{networkId}/appliance/uplinks/usageHistory",+ primary_keys=("networkId", "startTime"),+ paginated=False,+ fanout=DependentEndpointConfig(+ parent_name=_NETWORK_FANOUT_PARENT,+ resolve_param="networkId",+ resolve_field="id",+ include_from_parent=["id"],+ parent_field_renames={"id": "networkId"},+ parent_params={"perPage": 1000},+ child_params={"resolution": UPLINK_USAGE_RESOLUTION_SECONDS},+ ),+ incremental_fields=[_datetime_field("startTime")],+ default_incremental_field="startTime",+ partition_key="startTime",+ ),+ MerakiEndpoint(+ name="client_usage_histories",+ path="networks/{networkId}/clients/usageHistories",+ primary_keys=("networkId", "clientId", "ts"),+ # Each request names a batch of clients from the network clients table, so the source reads it itself.+ fanout=DependentEndpointConfig(+ parent_name=_NETWORK_FANOUT_PARENT,+ resolve_param="networkId",+ resolve_field="id",+ include_from_parent=["id"],+ parent_field_renames={"id": "networkId"},+ ),+ incremental_fields=[_datetime_field("ts")],+ default_incremental_field="ts",+ partition_key="ts",),)}diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/cisco_meraki/source.py b/products/warehouse_sources/backend/temporal/data_imports/sources/cisco_meraki/source.pyindex 7d2ea7213c47..bed2344168a3 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/cisco_meraki/source.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/cisco_meraki/source.py@@ -136,8 +136,10 @@ class CiscoMerakiSource(ResumableSource[CiscoMerakiSourceConfig, CiscoMerakiResucaption=("In Meraki Dashboard, open **Organization > API & Webhooks > API keys and access**. Create an API key. ""Enable API access for your organization. Use an administrator account with read access to the selected tables. "- "Each connection imports one organization. Network events support incremental sync, and other tables use full refresh. "- "Network clients include clients seen in the last 31 days. Health alerts include active alerts only."+ "Each connection imports one organization. Network events, uplink usage history, and client usage histories support incremental sync. "+ "Other tables use full refresh. Network clients include clients seen in the last 31 days. "+ "The first sync of client usage histories reads the last 31 days, and the first sync of uplink usage history reads the last 30 days. "+ "Health alerts include active alerts only."),fields=cast(list[FieldType],diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/cisco_meraki/tests/test_cisco_meraki.py b/products/warehouse_sources/backend/temporal/data_imports/sources/cisco_meraki/tests/test_cisco_meraki.pyindex 03401131cbb6..8655e89bf4c0 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/cisco_meraki/tests/test_cisco_meraki.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/cisco_meraki/tests/test_cisco_meraki.py@@ -4,6 +4,7 @@ from typing import Any, castfrom urllib.parse import parse_qs, urlsplitimport pytest+import time_machinefrom unittest.mock import MagicMockimport responses@@ -72,6 +73,7 @@ def rows(response: SourceResponse) -> list[dict[str, Any]]:("inventory_devices", "inventory/devices", "serial", 1000),("uplink_statuses", "uplinks/statuses", "serial", 1000),("assurance_alerts", "assurance/alerts", "id", 300),+ ("device_availabilities", "devices/availabilities", "serial", 1000),],)@responses.activate@@ -390,3 +392,215 @@ def test_network_events_walk_back_per_product_type(assert {request.param("productType") for request in result.requests[1:]} == {"appliance", "wireless"}assert {row["productType"] for row in result.rows} == {"appliance"}assert len({row["id"] for row in result.rows}) == len(result.rows) == 4+++NOW = datetime(2026, 3, 1, 12, 0, tzinfo=UTC)+NETWORKS_PATH = "/api/v1/organizations/123456/networks"+++def _usage_history(client_id: str, ts: str) -> dict[str, Any]:+ return {+ "clientId": client_id,+ "clientIp": "192.0.2.10",+ "clientMac": "00:00:5e:00:53:01",+ "usageHistory": [{"ts": ts, "sent": 10.0, "received": 20.0}],+ }+++@time_machine.travel(NOW, tick=False)+ ("endpoint", "child_path", "watermark", "expected_timespan"),+ [+ ("uplink_usage_history", "/api/v1/networks/N_1/appliance/uplinks/usageHistory", None, "2592000"),+ (+ "uplink_usage_history",+ "/api/v1/networks/N_1/appliance/uplinks/usageHistory",+ datetime(2026, 2, 26, 12, 0, tzinfo=UTC),+ "259200",+ ),+ (+ "uplink_usage_history",+ "/api/v1/networks/N_1/appliance/uplinks/usageHistory",+ datetime(2026, 3, 1, 11, 50, tzinfo=UTC),+ "7200",+ ),+ (+ "uplink_usage_history",+ "/api/v1/networks/N_1/appliance/uplinks/usageHistory",+ "2025-12-01T00:00:00Z",+ "2592000",+ ),+ ("client_usage_histories", "/api/v1/networks/N_1/clients/usageHistories", None, "2678400"),+ (+ "client_usage_histories",+ "/api/v1/networks/N_1/clients/usageHistories",+ datetime(2026, 2, 26, 12, 0, tzinfo=UTC),+ "259200",+ ),+ (+ "client_usage_histories",+ "/api/v1/networks/N_1/clients/usageHistories",+ "2026-03-01T00:00:00Z",+ "86400",+ ),+ (+ "client_usage_histories",+ "/api/v1/networks/N_1/clients/usageHistories",+ datetime(2025, 12, 1, tzinfo=UTC),+ "2678400",+ ),+ ],+)+def test_usage_history_timespan_follows_watermark(+ endpoint: str, child_path: str, watermark: datetime | str | None, expected_timespan: str+) -> None:+ result = make_driver().run(+ endpoint,+ network_api(+ {+ (NETWORKS_PATH, None): ScriptedResponse(json=[{"id": "N_1", "productTypes": ["appliance"]}]),+ ("/api/v1/networks/N_1/clients", None): ScriptedResponse(json=[{"id": "k1"}]),+ (child_path, None): ScriptedResponse(json=[]),+ }+ ),+ incremental_field="startTime" if endpoint == "uplink_usage_history" else "ts",+ should_use_incremental_field=watermark is not None,+ db_incremental_field_last_value=watermark,+ )++ assert result.raised is None+ timespans = {request.path: request.param("timespan") for request in result.requests[1:]}+ assert timespans[child_path] == expected_timespan+ if endpoint == "client_usage_histories":+ assert timespans["/api/v1/networks/N_1/clients"] == expected_timespan+++def test_full_refresh_ignores_stored_watermark() -> None:+ result = make_driver().run(+ "uplink_usage_history",+ network_api(+ {+ (NETWORKS_PATH, None): ScriptedResponse(json=[{"id": "N_1", "productTypes": ["appliance"]}]),+ ("/api/v1/networks/N_1/appliance/uplinks/usageHistory", None): ScriptedResponse(json=[]),+ }+ ),+ incremental_field="startTime",+ should_use_incremental_field=False,+ db_incremental_field_last_value=datetime.now(UTC),+ )++ assert result.raised is None+ assert result.requests[-1].param("timespan") == "2592000"+++def test_uplink_usage_history_reads_only_appliance_networks() -> None:+ interval = {+ "startTime": "2026-01-01T00:00:00Z",+ "endTime": "2026-01-01T01:00:00Z",+ "byInterface": [{"interface": "wan1", "sent": 100, "received": 200}],+ }+ result = make_driver().run(+ "uplink_usage_history",+ network_api(+ {+ (NETWORKS_PATH, None): ScriptedResponse(+ json=[+ {"id": "N_1", "productTypes": ["appliance", "switch"]},+ {"id": "N_2", "productTypes": ["wireless"]},+ {"id": "N_3", "productTypes": []},+ ]+ ),+ # A Link header on this endpoint must not start a second request.+ ("/api/v1/networks/N_1/appliance/uplinks/usageHistory", None): ScriptedResponse(+ json=[interval],+ headers={"Link": f'<{API_URL}/networks/N_1/appliance/uplinks/usageHistory?page=2>; rel="next"'},+ ),+ }+ ),+ )++ assert result.raised is None+ assert result.rows == [{**interval, "networkId": "N_1"}]+ assert [request.path for request in result.requests] == [+ NETWORKS_PATH,+ "/api/v1/networks/N_1/appliance/uplinks/usageHistory",+ ]+ assert result.requests[1].query == {"timespan": ("2592000",), "resolution": ("3600",)}+ assert result.response is not None+ assert result.response.sort_mode is None+ assert result.response.partition_keys == ["startTime"]+++def test_client_usage_histories_batch_client_keys_for_each_network() -> None:+ usage_path = "/api/v1/networks/N_1/clients/usageHistories"+ next_clients = f"{API_URL}/networks/N_1/clients?perPage=1000×pan=2678400&startingAfter=k120"++ def answer(request: RecordedRequest) -> ScriptedResponse:+ if request.path == usage_path:+ keys = (request.param("clients") or "").split(",")+ return ScriptedResponse(json=[_usage_history(key, "2026-01-01T00:00:00Z") for key in keys[:1]])+ return network_api(+ {+ (NETWORKS_PATH, None): ScriptedResponse(+ json=[{"id": "N_1"}, {"id": "N_2"}, {"id": "N_3"}],+ headers={"Link": f'<{API_URL}/organizations/123456/networks?startingAfter=N_3>; rel="next"'},+ ),+ (NETWORKS_PATH, "N_3"): ScriptedResponse(json=[]),+ ("/api/v1/networks/N_1/clients", None): ScriptedResponse(+ json=[{"id": f"k{index}"} for index in range(120)],+ headers={"Link": f'<{next_clients}>; rel="next"'},+ ),+ ("/api/v1/networks/N_1/clients", "k120"): ScriptedResponse(json=[{"id": "k120"}]),+ ("/api/v1/networks/N_2/clients", None): ScriptedResponse(json=[]),+ ("/api/v1/networks/N_3/clients", None): ScriptedResponse(json=[{"id": "k1"}]),+ ("/api/v1/networks/N_3/clients/usageHistories", None): ScriptedResponse(+ json=[{"clientId": "k1", "usageHistory": []}]+ ),+ }+ )(request)++ result = make_driver().run("client_usage_histories", answer)++ assert result.raised is None+ usage_requests = [request for request in result.requests if request.path.endswith("/usageHistories")]+ assert [len((request.param("clients") or "").split(",")) for request in usage_requests] == [100, 20, 1, 1]+ assert [request.param("clients") for request in usage_requests][2:] == ["k120", "k1"]+ assert [(row["networkId"], row["clientId"], row["ts"]) for row in result.rows] == [+ ("N_1", "k0", "2026-01-01T00:00:00Z"),+ ("N_1", "k100", "2026-01-01T00:00:00Z"),+ ("N_1", "k120", "2026-01-01T00:00:00Z"),+ ]+ assert result.rows[0] == {+ "networkId": "N_1",+ "clientId": "k0",+ "clientIp": "192.0.2.10",+ "clientMac": "00:00:5e:00:53:01",+ "ts": "2026-01-01T00:00:00Z",+ "sent": 10.0,+ "received": 20.0,+ }+ assert result.completed_state is None+++def test_client_usage_histories_resume_skips_completed_networks() -> None:+ result = make_driver().run(+ "client_usage_histories",+ network_api(+ {+ (NETWORKS_PATH, None): ScriptedResponse(json=[{"id": "N_1"}, {"id": "N_2"}, {"id": "N_3"}]),+ ("/api/v1/networks/N_2/clients", None): ScriptedResponse(json=[{"id": "k9"}]),+ ("/api/v1/networks/N_2/clients/usageHistories", None): ScriptedResponse(+ json=[_usage_history("k9", "2026-01-02T00:00:00Z")]+ ),+ ("/api/v1/networks/N_3/clients", None): ScriptedResponse(json=[]),+ }+ ),+ resume_state=CiscoMerakiResumeConfig(completed_paths=["networks/N_1/clients/usageHistories"]),+ )++ assert result.raised is None+ assert [row["clientId"] for row in result.rows] == ["k9"]+ assert not any(request.path.startswith("/api/v1/networks/N_1/") for request in result.requests)+ assert result.final_state == CiscoMerakiResumeConfig(+ completed_paths=["networks/N_1/clients/usageHistories", "networks/N_2/clients/usageHistories"]+ )