:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(scalr): add users, plans and state versions tables (#129125)
Tom Owersa984818c3e57
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.mdindex 35abe8c08985..e006ffe65bf0 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md@@ -10646,11 +10646,11 @@ Today (3): `environments`, `runs`, `workspaces`Diffed against: <https://docs.scalr.io/llms.txt>-- [ ] `Users (GET /api/iacp/v3/users)` — resolve created-by, updated-by, and locked-by IDs on runs and workspaces (high)+- [x] `Users (GET /api/iacp/v3/users)` — resolve created-by, updated-by, and locked-by IDs on runs and workspaces (high)- [ ] `Tags (GET /api/iacp/v3/tags)` — resolve tag IDs on imported runs and workspaces for ownership and cost grouping (high)- [ ] `Cost estimates (GET /api/iacp/v3/cost-estimates/{cost_estimate})` — resolve run cost-estimate IDs and compare current, proposed, and changed monthly infrastructure costs (high)-- [ ] `Plans (GET /api/iacp/v3/plans/{plan})` — resolve run plan IDs and measure resource additions, changes, and destruction (high)-- [ ] `State versions (GET /api/iacp/v3/state-versions)` — resolve run state-version IDs and analyze infrastructure state history (high)+- [x] `Plans (GET /api/iacp/v3/plans/{plan})` — resolve run plan IDs and measure resource additions, changes, and destruction (high)+- [x] `State versions (GET /api/iacp/v3/state-versions)` — resolve run state-version IDs and analyze infrastructure state history (high)- [ ] `Billing usages (GET /api/iacp/v3/reports/billing)` — analyze billed run usage by account, environment, workspace, and period (high)- [ ] `Policy checks (GET /api/iacp/v3/runs/{run}/policy-checks)` — analyze policy failures and overrides associated with infrastructure changes (medium)- [ ] `Drifted workspaces (GET /api/iacp/v3/reports/environments/{environment}/drifted-workspaces)` — measure infrastructure drift and identify affected workspaces and resources (medium)diff --git a/products/warehouse_sources/sources/scalr/api_inventory.md b/products/warehouse_sources/sources/scalr/api_inventory.mdindex 920630553aa0..9eaab9430a07 100644--- a/products/warehouse_sources/sources/scalr/api_inventory.md+++ b/products/warehouse_sources/sources/scalr/api_inventory.md@@ -5,16 +5,30 @@ All requests use a customer token with Bearer authentication and an account ID fConnections use HTTPS. The host check rejects internal addresses under the shared cloud policy.The source rejects redirects and reads page numbers from `meta.pagination.next-page`.-| Table | GET path | Required access | Key | Sync | Partition |-| ------------ | --------------- | -------------------------------- | ---- | --------------------------- | ------------ |-| environments | `/environments` | Read environments in the account | `id` | Full refresh | `created_at` |-| workspaces | `/workspaces` | Read workspaces in the account | `id` | Incremental on `updated_at` | `created_at` |-| runs | `/runs` | Read runs in the account | `id` | Full refresh | `created_at` |+| Table | GET path | Required access | Key | Sync | Partition |+| -------------- | ---------------------------------------- | ------------------------------------- | ---- | --------------------------- | ------------ |+| environments | `/environments` | Read environments in the account | `id` | Full refresh | `created_at` |+| workspaces | `/workspaces` | Read workspaces in the account | `id` | Incremental on `updated_at` | `created_at` |+| runs | `/runs` | Read runs in the account | `id` | Full refresh | `created_at` |+| users | `/users` | Read users in the account | `id` | Full refresh | `created_at` |+| plans | `/runs?include=plan` | Read runs in the account | `id` | Full refresh | None |+| state_versions | `/state-versions?filter[workspace]=<id>` | Read state versions of each workspace | `id` | Full refresh | `created_at` |Workspace requests use `sort=updated-at` and `filter[updated-at]=gte:<timestamp>` after the first sync.Full refresh omits the time filter. Environment requests use `sort=created-at`.The runs endpoint documents a creation filter but no update filter or sort parameter.Runs use full refresh because a creation watermark would miss later status changes.+Users and state versions use `sort=created-at`. Neither route documents an account filter, so their requests send none.+Plans have only a by-ID route (`/plans/{plan}`), so the source reads them from the `included` part of the run list.+State versions require a workspace filter, so the source lists the workspaces and then reads each one.+A workspace that returns 404 during this fan-out is skipped. A resumed fan-out skips the workspaces it finished.+State version requests are left out of HTTP sample capture, because the response holds the state file.+The source removes the base64 `state` attribute and the values of sensitive outputs, because they can hold secrets.++The billing usage report (`/reports/billing`) is not a table.+Its rows have no ID and no period, and a required date range sets what each row counts.+The runs table holds the same run counts per workspace and period.+JSON:API attributes become columns with underscores. Relationship references remain in the `relationships` column.The shared REST framework handles retries and stores page checkpoints through its resume hook.@@ -31,4 +45,8 @@ Sources:- [Environment list](https://docs.scalr.io/reference/list_environments)- [Workspace list](https://docs.scalr.io/reference/get_workspaces)- [Run list](https://docs.scalr.io/reference/get_runs)+- [User list](https://docs.scalr.io/reference/get_users)+- [Plan](https://docs.scalr.io/reference/get_plan)+- [State version list](https://docs.scalr.io/reference/list_state_versions)+- [Billing usage report](https://docs.scalr.io/reference/list_billing_usage)- [API tokens](https://docs.scalr.io/reference/api-tokens)diff --git a/products/warehouse_sources/sources/scalr/canonical_descriptions.py b/products/warehouse_sources/sources/scalr/canonical_descriptions.pyindex f99706eb5384..a40290e02aed 100644--- a/products/warehouse_sources/sources/scalr/canonical_descriptions.py+++ b/products/warehouse_sources/sources/scalr/canonical_descriptions.py@@ -38,4 +38,47 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {"relationships": "References to the workspace, plan, apply, and other related resources.",},},+ "users": {+ "description": "IAM users who can sign in to the Scalr account.",+ "docs_url": "https://docs.scalr.io/reference/get_users",+ "columns": {+ "id": "Identifier of the user.",+ "email": "Email address of the user.",+ "full_name": "Full name of the user.",+ "username": "Username of the user.",+ "status": "Current status of the user.",+ "created_at": "Time when Scalr created the user.",+ "last_login_at": "Time of the last sign-in of the user.",+ "relationships": "References to the identity providers and teams of the user.",+ },+ },+ "plans": {+ "description": "Terraform and OpenTofu plan stages of runs, with the count of proposed resource changes.",+ "docs_url": "https://docs.scalr.io/reference/get_plan",+ "columns": {+ "id": "Identifier of the plan.",+ "status": "Current stage or final result of the plan.",+ "status_timestamps": "Times when the plan entered its current and previous states.",+ "has_changes": "Whether the plan proposes resource changes.",+ "resource_additions": "Number of resources that the plan adds.",+ "resource_changes": "Number of resources that the plan updates in place.",+ "resource_destructions": "Number of resources that the plan destroys.",+ "execution_details": "Details about where and how the plan ran.",+ },+ },+ "state_versions": {+ "description": "Versions of the Terraform state of each workspace. The state file itself is not imported.",+ "docs_url": "https://docs.scalr.io/reference/list_state_versions",+ "columns": {+ "id": "Identifier of the state version.",+ "created_at": "Time when Scalr created the state version.",+ "serial": "Serial number of the state version, from the state file.",+ "lineage": "Lineage of the state, from the state file.",+ "status": "Processing status of the state version.",+ "size": "Size of the state file in bytes.",+ "outputs": "Output values of the state. Sensitive values are removed.",+ "resources": "Resources in the state. Scalr marks this attribute as deprecated.",+ "relationships": "References to the workspace, run, and previous and next state versions.",+ },+ },}diff --git a/products/warehouse_sources/sources/scalr/scalr.py b/products/warehouse_sources/sources/scalr/scalr.pyindex 62c6d1906e96..17b460461ae4 100644--- a/products/warehouse_sources/sources/scalr/scalr.py+++ b/products/warehouse_sources/sources/scalr/scalr.py@@ -3,9 +3,12 @@ from datetime import datetimefrom typing import Anyfrom products.warehouse_sources.sources.scalr._config import ScalrSourceConfig-from products.warehouse_sources.sources.scalr.settings import ENDPOINTS+from products.warehouse_sources.sources.scalr.settings import ENDPOINTS, ScalrEndpointfrom products.warehouse_sources.sources.sdk import (BearerTokenAuth,+ Endpoint,+ EndpointResource,+ JSONResponseCursorPaginatorConfig,RESTAPIConfig,RESTClient,ResumableSourceManager,@@ -14,14 +17,22 @@ from products.warehouse_sources.sources.sdk import (SourceResponse,frozen,rest_api_resource,+ rest_api_resources,)HEADERS = {"Accept": "application/vnd.api+json", "Content-Type": "application/vnd.api+json"}+PAGINATOR: JSONResponseCursorPaginatorConfig = {+ "type": "cursor",+ "cursor_path": "meta.pagination.next-page",+ "cursor_param": "page[number]",+ "raise_on_repeated_cursor": True,+}@frozenclass ScalrResumeConfig:- next_page: str+ next_page: str | None = None+ fan_out: dict[str, Any] | None = Nonedef base_url(config: ScalrSourceConfig, team_id: int, api_version: str) -> str:@@ -73,6 +84,66 @@ def flatten_resource(row: dict[str, Any]) -> dict[str, Any]:return result+def flatten_state_version(row: dict[str, Any]) -> dict[str, Any]:+ # The base64 state file can hold provider credentials and secret resource attributes.+ result = flatten_resource(row)+ result.pop("state", None)+ outputs = result.get("outputs")+ if isinstance(outputs, list):+ result["outputs"] = [+ {**output, "value": None} if isinstance(output, dict) and output.get("sensitive") else output+ for output in outputs+ ]+ return result+++def included_plans(body: dict[str, Any]) -> list[dict[str, Any]]:+ if not isinstance(body.get("data"), list):+ raise ValueError("Scalr returned a run list without a data array.")+ return [flatten_resource(item) for item in body.get("included") or [] if item.get("type") == "plans"]+++def _list_params(endpoint: ScalrEndpoint, account_id: str) -> dict[str, Any]:+ params: dict[str, Any] = {"page[size]": 50, "page[number]": 1}+ if endpoint.account_filter:+ params["filter[account]"] = account_id+ if endpoint.sort:+ params["sort"] = endpoint.sort+ if endpoint.include:+ params["include"] = endpoint.include+ return params+++def _resource(+ name: str, endpoint: ScalrEndpoint, params: dict[str, Any], should_use_incremental_field: bool+) -> EndpointResource:+ endpoint_config: Endpoint = {"path": endpoint.path, "params": params}+ resource: EndpointResource = {+ "name": name,+ "table_format": "delta",+ "write_disposition": "merge" if should_use_incremental_field else "replace",+ "primary_key": list(endpoint.primary_keys),+ "columns": {"created_at": {"data_type": "timestamp"}, "updated_at": {"data_type": "timestamp"}},+ "endpoint": endpoint_config,+ }+ if endpoint.include:+ # The whole body goes to the map, because the rows sit in `included`, beside `data`.+ resource["data_map"] = included_plans+ else:+ endpoint_config.update(+ {"data_selector": "data", "data_selector_required": True, "data_selector_empty_ok": True}+ )+ resource["data_map"] = flatten_state_version if endpoint.per_workspace else flatten_resource+ if endpoint.per_workspace:+ endpoint_config["path"] = f"{endpoint.path}?filter[workspace]={{workspace_id}}"+ params["workspace_id"] = {"type": "resolve", "resource": "workspaces", "field": "id"}+ # Without an explicit paginator, a fan-out child reads only its first page.+ endpoint_config["paginator"] = PAGINATOR+ # A workspace deleted after the parent page was read returns 404.+ endpoint_config["response_actions"] = [{"status_code": 404, "action": "ignore"}]+ return resource++def scalr_source(config: ScalrSourceConfig,manager: ResumableSourceManager[ScalrResumeConfig],@@ -80,9 +151,7 @@ def scalr_source(api_version: str,) -> SourceResponse:endpoint = ENDPOINTS[inputs.schema_name]- params: dict[str, Any] = {"filter[account]": config.account_id, "page[size]": 50, "page[number]": 1}- if endpoint.sort:- params["sort"] = endpoint.sort+ params = _list_params(endpoint, config.account_id)if inputs.should_use_incremental_field:if inputs.schema_name != "workspaces" or inputs.incremental_field != "updated_at":raise ValueError("Scalr supports incremental sync only for the workspace updated_at field.")@@ -91,6 +160,24 @@ def scalr_source(value = watermark.isoformat() if isinstance(watermark, datetime) else str(watermark)params["filter[updated-at]"] = f"gte:{value}"+ resources: list[str | EndpointResource] = [+ _resource(inputs.schema_name, endpoint, params, inputs.should_use_incremental_field)+ ]+ if endpoint.per_workspace:+ resources.insert(+ 0,+ {+ "name": "workspaces",+ "endpoint": {+ "path": ENDPOINTS["workspaces"].path,+ "params": _list_params(ENDPOINTS["workspaces"], config.account_id),+ "data_selector": "data",+ "data_selector_required": True,+ "data_selector_empty_ok": True,+ },+ },+ )+rest_config: RESTAPIConfig = {"client": {"base_url": base_url(config, inputs.team_id, api_version),@@ -99,52 +186,54 @@ def scalr_source("allowed_hosts": [],"allow_redirects": False,"request_timeout": 30,- "paginator": {- "type": "cursor",- "cursor_path": "meta.pagination.next-page",- "cursor_param": "page[number]",- "raise_on_repeated_cursor": True,- },+ # A sampled response would keep the state file that flatten_state_version removes.+ "capture": not endpoint.per_workspace,+ "paginator": PAGINATOR,},- "resources": [- {- "name": inputs.schema_name,- "table_format": "delta",- "write_disposition": "merge" if inputs.should_use_incremental_field else "replace",- "primary_key": list(endpoint.primary_keys),- "data_map": flatten_resource,- "columns": {"created_at": {"data_type": "timestamp"}, "updated_at": {"data_type": "timestamp"}},- "endpoint": {- "path": endpoint.path,- "params": params,- "data_selector": "data",- "data_selector_required": True,- "data_selector_empty_ok": True,- },- }- ],+ "resources": resources,}+resume = manager.load_state() if manager.can_resume() else None- def save_checkpoint(state: dict[str, Any] | None) -> None:- if state and state.get("cursor") is not None:- manager.save_state(ScalrResumeConfig(next_page=str(state["cursor"])))-- resource = rest_api_resource(- rest_config,- inputs.team_id,- inputs.job_id,- None,- resume_hook=save_checkpoint,- initial_paginator_state={"cursor": resume.next_page} if resume else None,- )+ if endpoint.per_workspace:++ def save_fan_out(state: dict[str, Any] | None) -> None:+ if state:+ manager.save_state(ScalrResumeConfig(fan_out=state))++ resource = next(+ item+ for item in rest_api_resources(+ rest_config,+ inputs.team_id,+ inputs.job_id,+ None,+ resume_hook=save_fan_out,+ initial_paginator_state=resume.fan_out if resume else None,+ )+ if item.name == inputs.schema_name+ )+ else:++ def save_checkpoint(state: dict[str, Any] | None) -> None:+ if state and state.get("cursor") is not None:+ manager.save_state(ScalrResumeConfig(next_page=str(state["cursor"])))++ resource = rest_api_resource(+ rest_config,+ inputs.team_id,+ inputs.job_id,+ None,+ resume_hook=save_checkpoint,+ initial_paginator_state={"cursor": resume.next_page} if resume and resume.next_page else None,+ )return SourceResponse(name=inputs.schema_name,items=lambda: resource,primary_keys=list(endpoint.primary_keys),column_hints=resource.column_hints,- partition_keys=[endpoint.partition_key],- partition_mode="datetime",- partition_format="month",- sort_mode="asc" if endpoint.sort else "desc",+ partition_keys=[endpoint.partition_key] if endpoint.partition_key else None,+ partition_mode="datetime" if endpoint.partition_key else None,+ partition_format="month" if endpoint.partition_key else None,+ sort_mode=None if endpoint.per_workspace else "asc" if endpoint.sort else "desc",)diff --git a/products/warehouse_sources/sources/scalr/settings.py b/products/warehouse_sources/sources/scalr/settings.pyindex ec9ff073a83c..877a3c9c930b 100644--- a/products/warehouse_sources/sources/scalr/settings.py+++ b/products/warehouse_sources/sources/scalr/settings.py@@ -6,7 +6,10 @@ class ScalrEndpoint:path: strsort: str | None = Noneprimary_keys: tuple[str, ...] = ("id",)- partition_key: str = "created_at"+ partition_key: str | None = "created_at"+ account_filter: bool = True+ include: str | None = None+ per_workspace: bool = FalseENDPOINTS = {@@ -14,6 +17,11 @@ ENDPOINTS = {"workspaces": ScalrEndpoint(path="workspaces", sort="updated-at"),# A creation filter would miss later run status changes, so runs require full refresh."runs": ScalrEndpoint(path="runs"),+ "users": ScalrEndpoint(path="users", sort="created-at", account_filter=False),+ # Plans have only a by-ID route, so read them as included resources of the run list.+ "plans": ScalrEndpoint(path="runs", include="plan", partition_key=None),+ # The route requires a workspace filter, so it fans out over workspaces.+ "state_versions": ScalrEndpoint(path="state-versions", sort="created-at", account_filter=False, per_workspace=True),}INCREMENTAL_FIELDS: dict[str, list[IncrementalField]] = {diff --git a/products/warehouse_sources/sources/scalr/source.py b/products/warehouse_sources/sources/scalr/source.pyindex fb81812aff4f..bbab3da5a35e 100644--- a/products/warehouse_sources/sources/scalr/source.py+++ b/products/warehouse_sources/sources/scalr/source.py@@ -104,7 +104,7 @@ class ScalrSource(ResumableSource[ScalrSourceConfig, ScalrResumeConfig]):category=DataWarehouseSourceCategory.ENGINEERING___MONITORING,label="Scalr",caption="Create a service account token in Scalr under account scope > IAM > Service accounts. "- "Give the account read access to environments, workspaces, and runs.",+ "Give the account read access to the environments, workspaces, runs, users, and state versions you want to sync.",iconPath="/static/services/scalr.png",releaseStatus=ReleaseStatus.BETA,fields=cast(diff --git a/products/warehouse_sources/sources/scalr/tests/test_scalr.py b/products/warehouse_sources/sources/scalr/tests/test_scalr.pyindex a526496985bb..607d949e442b 100644--- a/products/warehouse_sources/sources/scalr/tests/test_scalr.py+++ b/products/warehouse_sources/sources/scalr/tests/test_scalr.py@@ -11,7 +11,13 @@ from products.warehouse_sources.sources.scalr._config import ScalrSourceConfigfrom products.warehouse_sources.sources.scalr.scalr import ScalrResumeConfigfrom products.warehouse_sources.sources.scalr.source import ScalrSourcefrom products.warehouse_sources.sources.sdk import ResumableSourceManager, SourceInputs-from products.warehouse_sources.sources.sdk.testing import ScriptedResponse, SourceDriver, always, scripted_network+from products.warehouse_sources.sources.sdk.testing import (+ RecordedRequest,+ ScriptedResponse,+ SourceDriver,+ always,+ scripted_network,+)MIXINS = "products.warehouse_sources.backend.temporal.data_imports.sources.common.mixins"@@ -138,9 +144,10 @@ def test_validation_uses_one_small_page(config: ScalrSourceConfig) -> None:assert dict(network.requests_log[0].query) == {"filter[account]": ("acc-example",), "page[size]": ("1",)}[email protected]("endpoint", ["runs", "plans"])@pytest.mark.parametrize("body", [{"errors": [{"title": "Unexpected result"}]}, {"data": {"id": "wrong-shape"}}])-def test_malformed_response_fails(config: ScalrSourceConfig, body: dict[str, Any]) -> None:- result = SourceDriver(ScalrSource(), config).run("runs", [response(body)])+def test_malformed_response_fails(config: ScalrSourceConfig, endpoint: str, body: dict[str, Any]) -> None:+ result = SourceDriver(ScalrSource(), config).run(endpoint, [response(body)])assert isinstance(result.raised, ValueError)assert len(result.requests) == 1@@ -192,3 +199,127 @@ def test_unsupported_incremental_field_fails(config: ScalrSourceConfig, endpointassert isinstance(result.raised, ValueError)assert "only for the workspace updated_at" in str(result.raised)assert result.requests == []+++def test_plans_come_from_included_runs_on_every_page(config: ScalrSourceConfig) -> None:+ def runs_page(run_id: str, plan_id: str, next_page: int | None) -> ScriptedResponse:+ return response(+ {+ "data": [{"id": run_id, "type": "runs", "attributes": {}}],+ "included": [+ {"id": plan_id, "type": "plans", "attributes": {"resource-additions": 2, "status": "finished"}},+ {"id": "ws-1", "type": "workspaces", "attributes": {}},+ ],+ "meta": {"pagination": {"next-page": next_page}},+ }+ )++ result = SourceDriver(ScalrSource(), config).run(+ "plans",+ [runs_page("run-1", "plan-1", 2), runs_page("run-2", "plan-2", 3), response({"data": [], "meta": {}})],+ )++ assert result.raised is None+ assert [(row["id"], row["resource_additions"]) for row in result.rows] == [("plan-1", 2), ("plan-2", 2)]+ assert [request.path for request in result.requests] == ["/api/iacp/v3/runs"] * 3+ assert all(request.param("include") == "plan" for request in result.requests)+++def test_state_versions_fan_out_over_workspaces_without_state_secrets(config: ScalrSourceConfig) -> None:+ state_versions = {+ "ws-1": [+ response(+ {+ "data": [+ {+ "id": "sv-1",+ "type": "state-versions",+ "attributes": {+ "created-at": "2025-01-01T00:00:00Z",+ "serial": 1,+ "state": "c2VjcmV0",+ "outputs": [+ {"name": "password", "sensitive": True, "value": "secret"},+ {"name": "url", "sensitive": False, "value": "https://example.com"},+ ],+ },+ }+ ],+ "meta": {"pagination": {"next-page": 2}},+ }+ ),+ page("state-versions", "sv-2", None),+ ],+ "ws-2": [response({"errors": [{"status": "404"}]}, 404)],+ }++ def respond(request: RecordedRequest) -> ScriptedResponse:+ if request.path.endswith("/workspaces"):+ return response(+ {+ "data": [{"id": ws, "type": "workspaces", "attributes": {}} for ws in state_versions],+ "meta": {"pagination": {"next-page": None}},+ }+ )+ workspace = request.param("filter[workspace]")+ assert workspace is not None+ return state_versions[workspace].pop(0)++ result = SourceDriver(ScalrSource(), config).run("state_versions", respond)++ assert result.raised is None+ assert [row["id"] for row in result.rows] == ["sv-1", "sv-2"]+ assert "state" not in result.rows[0]+ assert result.rows[0]["outputs"] == [+ {"name": "password", "sensitive": True, "value": None},+ {"name": "url", "sensitive": False, "value": "https://example.com"},+ ]+ assert [(request.path, request.param("filter[workspace]")) for request in result.requests] == [+ ("/api/iacp/v3/workspaces", None),+ ("/api/iacp/v3/state-versions", "ws-1"),+ ("/api/iacp/v3/state-versions", "ws-1"),+ ("/api/iacp/v3/state-versions", "ws-2"),+ ]+ assert result.requests[2].param("page[number]") == "2"+ assert all(options.get("capture") is False for options in result.session_options)+++def test_state_versions_resume_skips_finished_workspaces(config: ScalrSourceConfig) -> None:+ def respond_with(state_versions: dict[str, list[ScriptedResponse]]) -> Any:+ def respond(request: RecordedRequest) -> ScriptedResponse:+ if request.path.endswith("/workspaces"):+ return response(+ {+ "data": [{"id": ws, "type": "workspaces", "attributes": {}} for ws in ("ws-1", "ws-2")],+ "meta": {"pagination": {"next-page": None}},+ }+ )+ workspace = request.param("filter[workspace]")+ assert workspace is not None+ return state_versions[workspace].pop(0)++ return respond++ driver = SourceDriver(ScalrSource(), config)+ first = driver.run(+ "state_versions",+ respond_with(+ {+ "ws-1": [page("state-versions", "sv-1", None)],+ "ws-2": [response({"errors": [{"status": "401"}]}, 401)],+ }+ ),+ )++ assert isinstance(first.raised, HTTPError)+ assert first.final_state is not None++ resumed = driver.run(+ "state_versions",+ respond_with({"ws-2": [page("state-versions", "sv-2", None)]}),+ resume_state=first.final_state,+ )++ assert resumed.raised is None+ assert [row["id"] for row in resumed.rows] == ["sv-2"]+ assert [request.param("filter[workspace]") for request in resumed.requests] == [None, "ws-2"]