Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

feat(warehouse): add opsgenie timeline, on-call, rotation and team member tables (#123999)

Tom Owers5d3df1c35d7b
diff --git a/posthog/test/repo_invariants/dataclass_frozen_baseline.txt b/posthog/test/repo_invariants/dataclass_frozen_baseline.txt
index a9b80ad025aa..ee44caedfa94 100644
--- a/posthog/test/repo_invariants/dataclass_frozen_baseline.txt
+++ b/posthog/test/repo_invariants/dataclass_frozen_baseline.txt
@@ -821,8 +821,6 @@
1 products/warehouse_sources/backend/temporal/data_imports/sources/openweather/settings.py
1 products/warehouse_sources/backend/temporal/data_imports/sources/opinion_stage/opinion_stage.py
1 products/warehouse_sources/backend/temporal/data_imports/sources/opinion_stage/settings.py
-1 products/warehouse_sources/backend/temporal/data_imports/sources/opsgenie/opsgenie.py
-1 products/warehouse_sources/backend/temporal/data_imports/sources/opsgenie/settings.py
1 products/warehouse_sources/backend/temporal/data_imports/sources/opuswatch/opuswatch.py
1 products/warehouse_sources/backend/temporal/data_imports/sources/orb/orb.py
1 products/warehouse_sources/backend/temporal/data_imports/sources/orb/settings.py
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md
index 5e49c3a67399..b2773427cc13 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md
@@ -8892,15 +8892,15 @@ Note: Parsed official resource references for schedules, occurrences, receipts,
## Opsgenie — gaps
-Today (8): `alerts`, `escalations`, `incidents`, `integrations`, `schedules`, `services`, `teams`, `users`
+Today (12): `alerts`, `escalations`, `incident_timeline_entries`, `incidents`, `integrations`, `schedule_on_calls`, `schedule_rotations`, `schedules`, `services`, `team_members`, `teams`, `users`
Diffed against: <https://docs.opsgenie.com/docs/api-overview>
- [ ] `GET /v2/alerts/{identifier}/logs` — per-alert state transition history (created, acked, escalated, closed) — the basis of MTTA/MTTR (high)
-- [ ] `Team member API (GET /v2/teams/{id}/members)` — membership table joining the users and teams already synced (high)
-- [ ] `Schedule rotation API (GET /v2/schedules/{id}/rotations)` — lookup resolving the rotation structure behind each synced schedule; without it schedules are opaque (high)
-- [ ] `Who is on call API (GET /v2/schedules/{id}/on-calls, /v2/schedules/on-calls)` — resolves schedule + rotation into the actual on-call person, the headline operational fact (high)
-- [ ] `Incident timeline API (GET /v1/incidents/{id}/timeline)` — state/transition history for the incidents table already synced (high)
+- [x] `Team member API (GET /v2/teams/{id}/members)` — membership table joining the users and teams already synced (high) — shipped as `team_members`; Opsgenie has no member list GET, so members come from `GET /v2/teams/{id}`
+- [x] `Schedule rotation API (GET /v2/schedules/{id}/rotations)` — lookup resolving the rotation structure behind each synced schedule; without it schedules are opaque (high)
+- [x] `Who is on call API (GET /v2/schedules/{id}/on-calls, /v2/schedules/on-calls)` — resolves schedule + rotation into the actual on-call person, the headline operational fact (high) — shipped as `schedule_on_calls` from the per-schedule path; the all-schedules list is deprecated
+- [x] `Incident timeline API (GET /v1/incidents/{id}/timeline)` — state/transition history for the incidents table already synced (high) — shipped as `incident_timeline_entries` from `GET /v2/incident-timelines/{id}/entries`
- [ ] `GET /v2/alerts/{identifier}/notes` — responder commentary attached to alerts, needed for incident review analysis (medium)
- [ ] `GET /v2/alerts/{identifier}/recipients` — who was notified per alert and in what state — notification-reach analysis (medium)
- [ ] `Schedule override API (GET /v2/schedules/{id}/overrides)` — explains gaps between rotation and actual on-call coverage (medium)
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/opsgenie/canonical_descriptions.py b/products/warehouse_sources/backend/temporal/data_imports/sources/opsgenie/canonical_descriptions.py
index aa9b9365eb90..776188b25906 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/opsgenie/canonical_descriptions.py
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/opsgenie/canonical_descriptions.py
@@ -126,4 +126,54 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {
"teamId": "Id of the team the integration is assigned to, if any.",
},
},
+ "incident_timeline_entries": {
+ "description": "An entry on an incident's timeline, such as a status change, a responder action, or a custom note.",
+ "docs_url": "https://docs.opsgenie.com/docs/incident-timeline-api",
+ "columns": {
+ "incident_id": "Id of the incident the entry belongs to.",
+ "id": "Unique identifier of the timeline entry within its incident.",
+ "group": "Group of the entry (e.g. incident or custom).",
+ "type": "Type of the entry (e.g. CustomUserEntry).",
+ "eventTime": "Time the event on the timeline happened.",
+ "hidden": "Whether the entry is hidden on the timeline.",
+ "actor": "Name and type of the user, team, or API key that created the entry.",
+ "description": "Text of the entry and its content type.",
+ "lastEdit": "Time and actor of the last edit to the entry.",
+ },
+ },
+ "schedule_on_calls": {
+ "description": "The users on call for each schedule at the time of the sync, with teams and escalations resolved to users.",
+ "docs_url": "https://docs.opsgenie.com/docs/who-is-on-call-api",
+ "columns": {
+ "schedule_id": "Id of the schedule.",
+ "_parent": "Id, name, and enabled state of the schedule.",
+ "onCallRecipients": "Usernames of the users on call for the schedule at the time of the sync.",
+ },
+ },
+ "schedule_rotations": {
+ "description": "A rotation inside an on-call schedule, defining the participants and how often the shift hands over.",
+ "docs_url": "https://docs.opsgenie.com/docs/schedule-rotation-api",
+ "columns": {
+ "schedule_id": "Id of the schedule the rotation belongs to.",
+ "id": "Unique identifier of the rotation.",
+ "name": "Name of the rotation.",
+ "startDate": "Time the rotation starts.",
+ "endDate": "Time the rotation ends, if set.",
+ "type": "Hand-over interval of the rotation: daily, weekly, or hourly.",
+ "length": "Number of intervals in one shift.",
+ "participants": "Users, teams, escalations, or none slots that take turns in the rotation.",
+ "timeRestriction": "Days and hours the rotation is active, if restricted.",
+ },
+ },
+ "team_members": {
+ "description": "A user's membership of a team, with the user's role in that team.",
+ "docs_url": "https://docs.opsgenie.com/docs/team-api",
+ "columns": {
+ "team_id": "Id of the team.",
+ "user_id": "Id of the user.",
+ "username": "Username (email address) of the user.",
+ "user": "Id and username of the user.",
+ "role": "Role of the user in the team (e.g. admin or user).",
+ },
+ },
}
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/opsgenie/opsgenie.py b/products/warehouse_sources/backend/temporal/data_imports/sources/opsgenie/opsgenie.py
index 971dbdbfc9d5..9dbebfd23774 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/opsgenie/opsgenie.py
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/opsgenie/opsgenie.py
@@ -1,6 +1,6 @@
import dataclasses
from datetime import UTC, date, datetime
-from typing import Any, Optional
+from typing import Any, Optional, cast
from urllib.parse import urlencode
import requests
@@ -11,21 +11,30 @@ from products.warehouse_sources.backend.temporal.data_imports.sources.common.res
RESTAPIConfig,
rest_api_resource,
)
+from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.fanout import (
+ build_dependent_resource,
+)
from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.paginators import (
BasePaginator,
+ JSONResponseCursorPaginator,
SinglePagePaginator,
)
+from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.resource import Resource
+from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.typing import ClientConfig
from products.warehouse_sources.backend.temporal.data_imports.sources.common.resumable import ResumableSourceManager
from products.warehouse_sources.backend.temporal.data_imports.sources.common.typings import SourceResponse
-from products.warehouse_sources.backend.temporal.data_imports.sources.opsgenie.settings import OPSGENIE_ENDPOINTS
+from products.warehouse_sources.backend.temporal.data_imports.sources.opsgenie.settings import (
+ DEFAULT_PAGE_SIZE,
+ OPSGENIE_ENDPOINTS,
+ OpsgenieEndpointConfig,
+)
OPSGENIE_BASE_URLS = {
"us": "https://api.opsgenie.com",
"eu": "https://api.eu.opsgenie.com",
}
-# Opsgenie's max page size is 100; the default is 20.
-PAGE_SIZE = 100
+PAGE_SIZE = DEFAULT_PAGE_SIZE
# The alert/incident search endpoints reject requests where `offset + limit` exceeds
# 20,000. Instead of truncating there, we re-slice the search into a new createdAt
@@ -36,10 +45,13 @@ MAX_SEARCH_RESULTS = 20_000
REQUEST_TIMEOUT_SECONDS = 60
[email protected](frozen=True)
class OpsgenieResumeConfig:
- offset: int
+ offset: int = 0
window_start_ms: Optional[int] = None
+ # Fan-out tables checkpoint the shape `build_dependent_resource` emits: which parents finished
+ # and where the current one stopped.
+ fanout_state: Optional[dict[str, Any]] = None
def _get_base_url(region: str) -> str:
@@ -165,6 +177,14 @@ class OpsgeniePaginator(BasePaginator):
self._has_next_page = True
+def _flatten_team_member(row: dict[str, Any]) -> dict[str, Any]:
+ user = row.get("user") or {}
+ return {**row, "user_id": user.get("id"), "username": user.get("username")}
+
+
+_CHILD_ROW_MAPS = {"team_members": _flatten_team_member}
+
+
def validate_credentials(api_key: str, region: str, endpoint: Optional[str] = None) -> tuple[bool, int, str | None]:
"""Probe Opsgenie with a cheap single-row request.
@@ -172,6 +192,9 @@ def validate_credentials(api_key: str, region: str, endpoint: Optional[str] = No
The caller decides how to treat 403 (valid key, missing access for the probed endpoint).
"""
config = OPSGENIE_ENDPOINTS.get(endpoint) if endpoint else None
+ if config is not None and config.fanout is not None:
+ # A fan-out path needs a parent id, so probe the parent listing it reads first.
+ config = OPSGENIE_ENDPOINTS[config.fanout.parent_name]
path = config.path if config else "/v2/users"
params = {"limit": 1} if config is None or config.paginated else {}
url = f"{_get_base_url(region)}{path}"
@@ -211,6 +234,20 @@ def opsgenie_source(
should_use_incremental_field: bool = False,
) -> SourceResponse:
config = OPSGENIE_ENDPOINTS[endpoint]
+ client_config: ClientConfig = {
+ "base_url": _get_base_url(region),
+ # Framework auth so the key is redacted from logs and error messages; the GenieKey
+ # scheme is Opsgenie's own bearer-style header.
+ "auth": {
+ "type": "api_key",
+ "api_key": f"GenieKey {api_key}",
+ "name": "Authorization",
+ "location": "header",
+ },
+ }
+
+ if config.fanout is not None:
+ return _fanout_source(endpoint, config, client_config, team_id, job_id, resumable_source_manager)
params: dict[str, Any] = {}
paginator: BasePaginator
@@ -233,18 +270,7 @@ def opsgenie_source(
paginator = SinglePagePaginator()
rest_config: RESTAPIConfig = {
- "client": {
- "base_url": _get_base_url(region),
- # Framework auth so the key is redacted from logs and error messages; the GenieKey
- # scheme is Opsgenie's own bearer-style header.
- "auth": {
- "type": "api_key",
- "api_key": f"GenieKey {api_key}",
- "name": "Authorization",
- "location": "header",
- },
- "paginator": paginator,
- },
+ "client": {**client_config, "paginator": paginator},
"resource_defaults": {},
"resources": [
{
@@ -286,7 +312,7 @@ def opsgenie_source(
return SourceResponse(
name=endpoint,
items=lambda: resource,
- primary_keys=[config.primary_key],
+ primary_keys=config.primary_keys,
# Search-window endpoints request createdAt ascending; full-refresh endpoints
# replace wholesale, so ascending is correct everywhere.
sort_mode="asc",
@@ -297,3 +323,67 @@ def opsgenie_source(
partition_keys=[config.partition_key] if config.partition_key else None,
column_hints=resource.column_hints,
)
+
+
+def _fanout_source(
+ endpoint: str,
+ config: OpsgenieEndpointConfig,
+ client_config: ClientConfig,
+ team_id: int,
+ job_id: str,
+ resumable_source_manager: ResumableSourceManager[OpsgenieResumeConfig],
+) -> SourceResponse:
+ assert config.fanout is not None
+ parent_config = OPSGENIE_ENDPOINTS[config.fanout.parent_name]
+ parent_paginator: BasePaginator = (
+ OpsgeniePaginator(limit=parent_config.page_size, supports_search_window=parent_config.supports_search_window)
+ if parent_config.paginated
+ else SinglePagePaginator()
+ )
+ child_paginator: BasePaginator = (
+ JSONResponseCursorPaginator(cursor_path=config.next_cursor_path, cursor_param="offset")
+ if config.next_cursor_path
+ else SinglePagePaginator()
+ )
+
+ # The shared fan-out checkpoint lists every finished parent, so its size grows with each parent.
+ # Only the small, unpaginated parents (schedules, teams) are resumable; the incident timeline
+ # starts again from the first incident after an interruption.
+ resumable = not parent_config.paginated
+ resume = resumable_source_manager.load_state() if resumable and resumable_source_manager.can_resume() else None
+
+ def save_fanout_checkpoint(state: Optional[dict[str, Any]]) -> None:
+ if state:
+ resumable_source_manager.save_state(OpsgenieResumeConfig(fanout_state=state))
+
+ resource = cast(
+ Resource,
+ build_dependent_resource(
+ endpoint_configs=OPSGENIE_ENDPOINTS,
+ child_endpoint=endpoint,
+ fanout=config.fanout,
+ client_config=client_config,
+ path_format_values={},
+ team_id=team_id,
+ job_id=job_id,
+ db_incremental_field_last_value=None,
+ # Schedules and teams return their full collection and take no page size.
+ page_size_param="limit" if parent_config.paginated else None,
+ parent_endpoint_extra={"paginator": parent_paginator, "data_selector": parent_config.data_selector},
+ child_endpoint_extra={"paginator": child_paginator, "data_selector": config.data_selector},
+ resume_hook=save_fanout_checkpoint if resumable else None,
+ initial_paginator_state=resume.fanout_state if resume else None,
+ ),
+ )
+ row_map = _CHILD_ROW_MAPS.get(endpoint)
+ if row_map is not None:
+ resource = resource.add_map(row_map)
+
+ return SourceResponse(
+ name=endpoint,
+ items=lambda: resource,
+ primary_keys=config.primary_keys,
+ partition_count=1,
+ partition_size=1,
+ column_hints=resource.column_hints,
+ )
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/opsgenie/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/opsgenie/settings.py
index b36686dd4490..8c4efdc2a807 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/opsgenie/settings.py
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/opsgenie/settings.py
@@ -1,13 +1,20 @@
from dataclasses import dataclass, field
from typing import Optional
+from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.fanout import (
+ DependentEndpointConfig,
+)
from products.warehouse_sources.backend.types import IncrementalField, IncrementalFieldType
+# Opsgenie's max page size is 100; the default is 20.
+DEFAULT_PAGE_SIZE = 100
-@dataclass
+
+@dataclass(frozen=True)
class OpsgenieEndpointConfig:
+ name: str
path: str # Path under the API base URL, including the API version, e.g. "/v2/alerts"
- primary_key: str = "id"
+ primary_keys: list[str] = field(default_factory=lambda: ["id"])
partition_key: Optional[str] = None # Stable datetime field used to partition (never a mutable field)
incremental_fields: list[IncrementalField] = field(default_factory=list)
# True for endpoints that accept `limit`/`offset` pagination params. Endpoints left
@@ -21,6 +28,15 @@ class OpsgenieEndpointConfig:
# hard-cap `offset + limit` at 20,000 per query, so backfills re-slice into new
# createdAt windows when they hit the cap.
supports_search_window: bool = False
+ page_size: int = DEFAULT_PAGE_SIZE
+ default_incremental_field: Optional[str] = None
+ # Set for resources only listed per parent row (e.g. a schedule's rotations); the parent is
+ # re-listed each sync.
+ fanout: Optional[DependentEndpointConfig] = None
+ # JSON path of the rows in the response body. Every Opsgenie collection sits under `data`.
+ data_selector: str = "data"
+ # JSON path of the next-page cursor, for fan-out children that page with a cursor, not an offset.
+ next_cursor_path: Optional[str] = None
_CREATED_AT_INCREMENTAL: list[IncrementalField] = [
@@ -35,6 +51,7 @@ _CREATED_AT_INCREMENTAL: list[IncrementalField] = [
OPSGENIE_ENDPOINTS: dict[str, OpsgenieEndpointConfig] = {
"alerts": OpsgenieEndpointConfig(
+ name="alerts",
path="/v2/alerts",
partition_key="createdAt",
incremental_fields=_CREATED_AT_INCREMENTAL,
@@ -46,6 +63,7 @@ OPSGENIE_ENDPOINTS: dict[str, OpsgenieEndpointConfig] = {
supports_search_window=True,
),
"incidents": OpsgenieEndpointConfig(
+ name="incidents",
path="/v1/incidents",
partition_key="createdAt",
incremental_fields=_CREATED_AT_INCREMENTAL,
@@ -53,25 +71,90 @@ OPSGENIE_ENDPOINTS: dict[str, OpsgenieEndpointConfig] = {
supports_search_window=True,
),
"users": OpsgenieEndpointConfig(
+ name="users",
path="/v2/users",
paginated=True,
),
"teams": OpsgenieEndpointConfig(
+ name="teams",
path="/v2/teams",
),
"schedules": OpsgenieEndpointConfig(
+ name="schedules",
path="/v2/schedules",
),
"escalations": OpsgenieEndpointConfig(
+ name="escalations",
path="/v2/escalations",
),
"services": OpsgenieEndpointConfig(
+ name="services",
path="/v1/services",
paginated=True,
),
"integrations": OpsgenieEndpointConfig(
+ name="integrations",
path="/v2/integrations",
),
+ "incident_timeline_entries": OpsgenieEndpointConfig(
+ name="incident_timeline_entries",
+ path="/v2/incident-timelines/{incident_id}/entries",
+ primary_keys=["incident_id", "id"],
+ # The timeline endpoint caps `limit` at 20 and pages with an opaque `nextOffset` cursor.
+ page_size=20,
+ next_cursor_path="data.nextOffset",
+ data_selector="data.entries",
+ fanout=DependentEndpointConfig(
+ parent_name="incidents",
+ resolve_param="incident_id",
+ resolve_field="id",
+ include_from_parent=["id"],
+ parent_field_renames={"id": "incident_id"},
+ parent_params={"sort": "createdAt", "order": "asc"},
+ child_params={"order": "asc"},
+ ),
+ ),
+ "schedule_on_calls": OpsgenieEndpointConfig(
+ name="schedule_on_calls",
+ path="/v2/schedules/{schedule_id}/on-calls",
+ primary_keys=["schedule_id"],
+ # One object per schedule. `flat=true` resolves teams and escalations down to the users
+ # on call right now. `/v2/schedules/on-calls` lists every schedule at once but is deprecated.
+ fanout=DependentEndpointConfig(
+ parent_name="schedules",
+ resolve_param="schedule_id",
+ resolve_field="id",
+ include_from_parent=["id"],
+ parent_field_renames={"id": "schedule_id"},
+ child_params={"flat": "true"},
+ ),
+ ),
+ "schedule_rotations": OpsgenieEndpointConfig(
+ name="schedule_rotations",
+ path="/v2/schedules/{schedule_id}/rotations",
+ primary_keys=["schedule_id", "id"],
+ fanout=DependentEndpointConfig(
+ parent_name="schedules",
+ resolve_param="schedule_id",
+ resolve_field="id",
+ include_from_parent=["id"],
+ parent_field_renames={"id": "schedule_id"},
+ ),
+ ),
+ # Opsgenie has no member listing endpoint; the members come embedded in the single-team read.
+ "team_members": OpsgenieEndpointConfig(
+ name="team_members",
+ path="/v2/teams/{team_id}",
+ primary_keys=["team_id", "user_id"],
+ data_selector="data.members",
+ fanout=DependentEndpointConfig(
+ parent_name="teams",
+ resolve_param="team_id",
+ resolve_field="id",
+ include_from_parent=["id"],
+ parent_field_renames={"id": "team_id"},
+ ),
+ ),
}
ENDPOINTS = tuple(OPSGENIE_ENDPOINTS.keys())
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/opsgenie/tests/test_opsgenie.py b/products/warehouse_sources/backend/temporal/data_imports/sources/opsgenie/tests/test_opsgenie.py
index 8892babdd815..bf9b7d25e2a7 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/opsgenie/tests/test_opsgenie.py
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/opsgenie/tests/test_opsgenie.py
@@ -117,6 +117,77 @@ class TestPagination:
assert len(result.requests) == 1
+class TestFanout:
+ def test_incident_timeline_walks_each_incidents_cursor(self) -> None:
+ result = _driver().run(
+ "incident_timeline_entries",
+ [
+ _page([{"id": "inc-1"}]),
+ ScriptedResponse(json={"data": {"entries": [{"id": "e1"}], "nextOffset": "e1"}}),
+ ScriptedResponse(json={"data": {"entries": [{"id": "e2"}]}}),
+ ],
+ )
+
+ assert result.raised is None
+ assert result.paths == [
+ "/v1/incidents",
+ "/v2/incident-timelines/inc-1/entries",
+ "/v2/incident-timelines/inc-1/entries",
+ ]
+ assert result.requests[0].param("sort") == "createdAt"
+ assert result.params("offset") == ["0", None, "e1"]
+ assert result.params("limit") == [str(PAGE_SIZE), "20", "20"]
+ assert [(row["incident_id"], row["id"]) for row in result.rows] == [("inc-1", "e1"), ("inc-1", "e2")]
+ assert result.saved_states == []
+
+ @pytest.mark.parametrize(
+ "endpoint,parent_path,child_body,expected",
+ [
+ (
+ "team_members",
+ "/v2/teams",
+ {
+ "data": {
+ "id": "p1",
+ "members": [{"user": {"id": "u1", "username": "[email protected]"}, "role": "admin"}],
+ }
+ },
+ {"team_id": "p1", "user_id": "u1", "username": "[email protected]", "role": "admin"},
+ ),
+ (
+ "schedule_on_calls",
+ "/v2/schedules",
+ {"data": {"_parent": {"id": "p1"}, "onCallRecipients": ["[email protected]"]}},
+ {"schedule_id": "p1", "onCallRecipients": ["[email protected]"]},
+ ),
+ ],
+ )
+ def test_single_object_child_becomes_keyed_rows(
+ self, endpoint: str, parent_path: str, child_body: dict[str, Any], expected: dict[str, Any]
+ ) -> None:
+ result = _driver().run(endpoint, [_page([{"id": "p1"}]), ScriptedResponse(json=child_body)])
+
+ assert result.raised is None
+ assert result.paths[0] == parent_path
+ assert result.requests[0].param("limit") is None
+ assert len(result.rows) == 1
+ assert expected.items() <= result.rows[0].items()
+
+ def test_resume_skips_completed_parents(self) -> None:
+ resume = OpsgenieResumeConfig(
+ fanout_state={"completed": ["/v2/schedules/s1/rotations"], "current": None, "child_state": None}
+ )
+ result = _driver().run(
+ "schedule_rotations",
+ [_page([{"id": "s1"}, {"id": "s2"}]), _page([{"id": "r2"}])],
+ resume_state=resume,
+ )
+
+ assert result.raised is None
+ assert result.paths == ["/v2/schedules", "/v2/schedules/s2/rotations"]
+ assert [(row["schedule_id"], row["id"]) for row in result.rows] == [("s2", "r2")]
+
+
class TestValidateCredentials:
@pytest.mark.parametrize(
"status_code,expected_ok,expected_status",
@@ -146,5 +217,5 @@ class TestOpsgenieSourceResponse:
assert result.raised is None
assert response is not None
assert response.name == endpoint
- assert response.primary_keys == [OPSGENIE_ENDPOINTS[endpoint].primary_key]
+ assert response.primary_keys == OPSGENIE_ENDPOINTS[endpoint].primary_keys
assert callable(response.items)
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/opsgenie/tests/test_opsgenie_source.py b/products/warehouse_sources/backend/temporal/data_imports/sources/opsgenie/tests/test_opsgenie_source.py
index c2e4a1478478..9ffde63908af 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/opsgenie/tests/test_opsgenie_source.py
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/opsgenie/tests/test_opsgenie_source.py
@@ -38,9 +38,16 @@ class TestOpsgenieSource:
with scripted_network([ScriptedResponse(status=403)]):
assert self.source.validate_credentials(self.config, team_id=1, schema_name=None) == (True, None)
- def test_validate_credentials_rejects_403_for_specific_schema(self) -> None:
+ @pytest.mark.parametrize(
+ "schema_name,probed_path",
+ [
+ ("integrations", "/v2/integrations"),
+ ("team_members", "/v2/teams"),
+ ],
+ )
+ def test_validate_credentials_rejects_403_for_specific_schema(self, schema_name: str, probed_path: str) -> None:
with scripted_network([ScriptedResponse(status=403)]) as network:
- ok, error = self.source.validate_credentials(self.config, team_id=1, schema_name="integrations")
+ ok, error = self.source.validate_credentials(self.config, team_id=1, schema_name=schema_name)
assert ok is False
assert error == "Your Opsgenie API key does not have access to this resource"
- assert network.requests_log[0].path == "/v2/integrations"
+ assert network.requests_log[0].path == probed_path