:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(warehouse): add octopus deploy lifecycles, interruptions and users tables (#123159)
Tom Owers428282a19e86
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.mdindex f5fd51a4e1f5..6a02b417203a 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md@@ -8554,10 +8554,10 @@ Today (11): `channels`, `deployments`, `environments`, `events`, `machines`, `prDiffed against: <https://demo.octopus.app/api>-- [ ] `users (/api/users)` — Lookup that resolves the user ids already carried on events, deployments, tasks and interruptions (high)+- [x] `users (/api/users)` — Lookup that resolves the user ids already carried on events, deployments, tasks and interruptions (high)- [ ] `teams + teammembership (/api/teams, /api/teammembership)` — Team definitions and the user→team junction — the only way to aggregate deployment activity by team (high)-- [ ] `lifecycles (/api/{space}/lifecycles)` — Lookup resolving the LifecycleId on projects/releases into ordered phases and environments, needed to judge whether a deployment progressed normally (high)-- [ ] `interruptions (/api/{space}/interruptions)` — Manual intervention and approval records with responsible-team and resolution timestamps — the wait time inside deployment lead time (high)+- [x] `lifecycles (/api/{space}/lifecycles)` — Lookup resolving the LifecycleId on projects/releases into ordered phases and environments, needed to judge whether a deployment progressed normally (high)+- [x] `interruptions (/api/{space}/interruptions)` — Manual intervention and approval records with responsible-team and resolution timestamps — the wait time inside deployment lead time (high)- [ ] `insights/reports (/api/{space}/insights/reports)` — Octopus's own DORA-style reporting objects (deployment frequency, lead time, failure rate) — the vendor's headline metric (high)- [ ] `runbookRuns (/api/{space}/runbookRuns)` — Operational run history that sits alongside deployments; without it half of what Octopus executes is invisible (medium)- [ ] `runbooks (/api/{space}/runbooks)` — Lookup resolving the RunbookId on runbook runs and project triggers (medium)@@ -8567,7 +8567,7 @@ Diffed against: <https://demo.octopus.app/api>- [ ] `workers + workerpools (/api/{space}/workers, /api/{space}/workerpools)` — We sync machines (deployment targets) but not the worker fleet that actually executes steps, so execution capacity analysis is impossible (medium)- [ ] `deploymentprocesses (/api/{space}/deploymentprocesses)` — Step-level definition per project, needed to break deployment duration down by step (low)-Note: Diffed against the live self-describing API root on the public demo instance, which enumerates every resource link (space-scoped links are templated as /api/Spaces-1/...). Excluded as config/plumbing: subscriptions (webhooks), gitcredentials, certificates, signingkeys, accounts, feeds, proxies, smtp/telemetry/maintenance/performance configuration, migrations and import/export. Event lookup tables (events/categories, events/groups, events/agents, events/documenttypes) exist and would enrich the events table, but are low value on their own.+Note: Diffed against the live self-describing API root on the public demo instance, which enumerates every resource link (space-scoped links are templated as /api/Spaces-1/...). Excluded as config/plumbing: subscriptions (webhooks), gitcredentials, certificates, signingkeys, accounts, feeds, proxies, smtp/telemetry/maintenance/performance configuration, migrations and import/export. Event lookup tables (events/categories, events/groups, events/agents, events/documenttypes) exist and would enrich the events table, but are low value on their own. `insights/reports` lists only Insights report definitions (scope filters and time zone, Enterprise-only); the DORA metrics sit behind a per-report `/metrics` call that returns pre-aggregated series for a chosen time range, which the synced `deployments` and `tasks` tables already let you compute, so it is not synced.## Okendo — gapsdiff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/canonical_descriptions.py b/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/canonical_descriptions.pyindex 7fa122eb5fc1..5b9aa0052d17 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/canonical_descriptions.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/canonical_descriptions.py@@ -142,6 +142,53 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {"ProjectId": "Identifier of the project the task relates to, when applicable.",},},+ "lifecycles": {+ "description": "Lifecycles define the ordered phases and environments a release is promoted through.",+ "docs_url": "https://octopus.com/docs/releases/lifecycles",+ "columns": {+ "Id": "Unique identifier for the lifecycle; projects and channels reference it as LifecycleId.",+ "SpaceId": "Identifier of the space the lifecycle belongs to.",+ "Name": "Display name of the lifecycle.",+ "Slug": "URL-friendly identifier for the lifecycle.",+ "Description": "Description of the lifecycle.",+ "Phases": "Ordered phases, each with its automatic and optional deployment environments and promotion rules.",+ "ReleaseRetentionPolicy": "How long releases are kept for this lifecycle.",+ "TentacleRetentionPolicy": "How long extracted packages and files are kept on deployment targets.",+ },+ },+ "users": {+ "description": "Users of the Octopus Deploy instance, including service accounts.",+ "docs_url": "https://octopus.com/docs/security/users-and-teams",+ "columns": {+ "Id": "Unique identifier for the user; referenced as UserId on events and ResponsibleUserId on interruptions.",+ "Username": "Login name of the user.",+ "DisplayName": "Display name of the user.",+ "EmailAddress": "Email address of the user.",+ "IsActive": "Whether the user account is active.",+ "IsService": "Whether the user is a service account.",+ "ServiceAccountType": "Type of service account, when IsService is true.",+ "Identities": "External identity provider identities linked to the user.",+ "Created": "When the user was created.",+ },+ },+ "interruptions": {+ "description": "Interruptions are manual interventions and guided failure prompts that pause a task until a user responds.",+ "docs_url": "https://octopus.com/docs/projects/built-in-step-templates/manual-intervention-and-approvals",+ "columns": {+ "Id": "Unique identifier for the interruption.",+ "SpaceId": "Identifier of the space the interruption belongs to.",+ "Title": "Title of the interruption.",+ "Type": "Kind of interruption, such as a manual intervention or a guided failure.",+ "Created": "When the interruption was raised.",+ "IsPending": "Whether the interruption still waits for a response.",+ "TaskId": "Identifier of the server task the interruption paused; join to tasks.",+ "CorrelationId": "Identifier of the step or action that raised the interruption.",+ "ResponsibleUserId": "Identifier of the user who took responsibility for the interruption.",+ "ResponsibleTeamIds": "Identifiers of the teams allowed to respond to the interruption.",+ "RelatedDocumentIds": "Identifiers of the documents (deployment, project, environment) the interruption relates to.",+ "Form": "Form presented to the responder, with its submitted values.",+ },+ },"events": {"description": "Audit events record every significant action on the instance (deployments queued, releases created, settings changed).","docs_url": "https://octopus.com/docs/security/users-and-teams/auditing",diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/settings.pyindex d4bf1a220a11..75a7801e2ff5 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/settings.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/settings.py@@ -61,6 +61,22 @@ OCTOPUS_DEPLOY_ENDPOINTS: dict[str, OctopusDeployEndpointConfig] = {name="machines",path="/machines",),+ "lifecycles": OctopusDeployEndpointConfig(+ name="lifecycles",+ path="/lifecycles",+ ),+ "users": OctopusDeployEndpointConfig(+ name="users",+ path="/api/users",+ space_scoped=False,+ ),+ # Interruptions have no server-side date filter, and IsPending / ResponsibleUserId change as+ # they are handled, so full refresh only. Created is set once.+ "interruptions": OctopusDeployEndpointConfig(+ name="interruptions",+ path="/interruptions",+ partition_key="Created",+ ),# Releases and deployments expose no server-side date filter (only project/environment# filters), so they are full-refresh only. Their creation timestamps are immutable and make# stable partition keys.diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/tests/test_octopus_deploy.py b/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/tests/test_octopus_deploy.pyindex 9be2c7b69ecc..8cb5ed139817 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/tests/test_octopus_deploy.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/tests/test_octopus_deploy.py@@ -127,16 +127,18 @@ class TestOctopusDeploySourceResponse:"endpoint, primary_keys, partition_key",[("spaces", ["Id"], None),+ ("users", ["Id"], None),("projects", ["SpaceId", "Id"], None),("releases", ["SpaceId", "Id"], "Assembled"),("deployments", ["SpaceId", "Id"], "Created"),("tasks", ["SpaceId", "Id"], "QueueTime"),("events", ["SpaceId", "Id"], "Occurred"),+ ("interruptions", ["SpaceId", "Id"], "Created"),],)def test_response_shape(self, endpoint, primary_keys, partition_key):script = [_page([], has_next=False)]- if endpoint != "spaces":+ if endpoint not in ("spaces", "users"):script = [_page([{"Id": "Spaces-1"}], has_next=False), *script]result = _driver().run(endpoint, script)assert result.raised is None@@ -204,13 +206,14 @@ class TestGetRows:# Spaces-1 was already completed before the crash — not re-fetched.assert "/api/Spaces-1/projects" not in result.paths- def test_instance_level_endpoint_skips_space_fan_out(self):- responses = [_page([{"Id": "Spaces-1"}, {"Id": "Spaces-2"}], has_next=False)]- result = _driver().run("spaces", responses)+ @pytest.mark.parametrize("endpoint, path", [("spaces", "/api/spaces"), ("users", "/api/users")])+ def test_instance_level_endpoint_skips_space_fan_out(self, endpoint, path):+ responses = [_page([{"Id": "A-1"}, {"Id": "A-2"}], has_next=False)]+ result = _driver().run(endpoint, responses)assert result.raised is None- assert result.paths == ["/api/spaces"]- assert [row["Id"] for row in result.rows] == ["Spaces-1", "Spaces-2"]+ assert result.paths == [path]+ assert result.rows == [{"Id": "A-1"}, {"Id": "A-2"}]def test_incremental_filter_applied_to_every_page(self):responses = [