Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

feat(warehouse): add octopus deploy lifecycles, interruptions and users tables (#123159)

Tom Owers428282a19e86
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md
index f5fd51a4e1f5..6a02b417203a 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md
@@ -8554,10 +8554,10 @@ Today (11): `channels`, `deployments`, `environments`, `events`, `machines`, `pr
Diffed against: <https://demo.octopus.app/api>
-- [ ] `users (/api/users)` — Lookup that resolves the user ids already carried on events, deployments, tasks and interruptions (high)
+- [x] `users (/api/users)` — Lookup that resolves the user ids already carried on events, deployments, tasks and interruptions (high)
- [ ] `teams + teammembership (/api/teams, /api/teammembership)` — Team definitions and the user→team junction — the only way to aggregate deployment activity by team (high)
-- [ ] `lifecycles (/api/{space}/lifecycles)` — Lookup resolving the LifecycleId on projects/releases into ordered phases and environments, needed to judge whether a deployment progressed normally (high)
-- [ ] `interruptions (/api/{space}/interruptions)` — Manual intervention and approval records with responsible-team and resolution timestamps — the wait time inside deployment lead time (high)
+- [x] `lifecycles (/api/{space}/lifecycles)` — Lookup resolving the LifecycleId on projects/releases into ordered phases and environments, needed to judge whether a deployment progressed normally (high)
+- [x] `interruptions (/api/{space}/interruptions)` — Manual intervention and approval records with responsible-team and resolution timestamps — the wait time inside deployment lead time (high)
- [ ] `insights/reports (/api/{space}/insights/reports)` — Octopus's own DORA-style reporting objects (deployment frequency, lead time, failure rate) — the vendor's headline metric (high)
- [ ] `runbookRuns (/api/{space}/runbookRuns)` — Operational run history that sits alongside deployments; without it half of what Octopus executes is invisible (medium)
- [ ] `runbooks (/api/{space}/runbooks)` — Lookup resolving the RunbookId on runbook runs and project triggers (medium)
@@ -8567,7 +8567,7 @@ Diffed against: <https://demo.octopus.app/api>
- [ ] `workers + workerpools (/api/{space}/workers, /api/{space}/workerpools)` — We sync machines (deployment targets) but not the worker fleet that actually executes steps, so execution capacity analysis is impossible (medium)
- [ ] `deploymentprocesses (/api/{space}/deploymentprocesses)` — Step-level definition per project, needed to break deployment duration down by step (low)
-Note: Diffed against the live self-describing API root on the public demo instance, which enumerates every resource link (space-scoped links are templated as /api/Spaces-1/...). Excluded as config/plumbing: subscriptions (webhooks), gitcredentials, certificates, signingkeys, accounts, feeds, proxies, smtp/telemetry/maintenance/performance configuration, migrations and import/export. Event lookup tables (events/categories, events/groups, events/agents, events/documenttypes) exist and would enrich the events table, but are low value on their own.
+Note: Diffed against the live self-describing API root on the public demo instance, which enumerates every resource link (space-scoped links are templated as /api/Spaces-1/...). Excluded as config/plumbing: subscriptions (webhooks), gitcredentials, certificates, signingkeys, accounts, feeds, proxies, smtp/telemetry/maintenance/performance configuration, migrations and import/export. Event lookup tables (events/categories, events/groups, events/agents, events/documenttypes) exist and would enrich the events table, but are low value on their own. `insights/reports` lists only Insights report definitions (scope filters and time zone, Enterprise-only); the DORA metrics sit behind a per-report `/metrics` call that returns pre-aggregated series for a chosen time range, which the synced `deployments` and `tasks` tables already let you compute, so it is not synced.
## Okendo — gaps
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/canonical_descriptions.py b/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/canonical_descriptions.py
index 7fa122eb5fc1..5b9aa0052d17 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/canonical_descriptions.py
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/canonical_descriptions.py
@@ -142,6 +142,53 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {
"ProjectId": "Identifier of the project the task relates to, when applicable.",
},
},
+ "lifecycles": {
+ "description": "Lifecycles define the ordered phases and environments a release is promoted through.",
+ "docs_url": "https://octopus.com/docs/releases/lifecycles",
+ "columns": {
+ "Id": "Unique identifier for the lifecycle; projects and channels reference it as LifecycleId.",
+ "SpaceId": "Identifier of the space the lifecycle belongs to.",
+ "Name": "Display name of the lifecycle.",
+ "Slug": "URL-friendly identifier for the lifecycle.",
+ "Description": "Description of the lifecycle.",
+ "Phases": "Ordered phases, each with its automatic and optional deployment environments and promotion rules.",
+ "ReleaseRetentionPolicy": "How long releases are kept for this lifecycle.",
+ "TentacleRetentionPolicy": "How long extracted packages and files are kept on deployment targets.",
+ },
+ },
+ "users": {
+ "description": "Users of the Octopus Deploy instance, including service accounts.",
+ "docs_url": "https://octopus.com/docs/security/users-and-teams",
+ "columns": {
+ "Id": "Unique identifier for the user; referenced as UserId on events and ResponsibleUserId on interruptions.",
+ "Username": "Login name of the user.",
+ "DisplayName": "Display name of the user.",
+ "EmailAddress": "Email address of the user.",
+ "IsActive": "Whether the user account is active.",
+ "IsService": "Whether the user is a service account.",
+ "ServiceAccountType": "Type of service account, when IsService is true.",
+ "Identities": "External identity provider identities linked to the user.",
+ "Created": "When the user was created.",
+ },
+ },
+ "interruptions": {
+ "description": "Interruptions are manual interventions and guided failure prompts that pause a task until a user responds.",
+ "docs_url": "https://octopus.com/docs/projects/built-in-step-templates/manual-intervention-and-approvals",
+ "columns": {
+ "Id": "Unique identifier for the interruption.",
+ "SpaceId": "Identifier of the space the interruption belongs to.",
+ "Title": "Title of the interruption.",
+ "Type": "Kind of interruption, such as a manual intervention or a guided failure.",
+ "Created": "When the interruption was raised.",
+ "IsPending": "Whether the interruption still waits for a response.",
+ "TaskId": "Identifier of the server task the interruption paused; join to tasks.",
+ "CorrelationId": "Identifier of the step or action that raised the interruption.",
+ "ResponsibleUserId": "Identifier of the user who took responsibility for the interruption.",
+ "ResponsibleTeamIds": "Identifiers of the teams allowed to respond to the interruption.",
+ "RelatedDocumentIds": "Identifiers of the documents (deployment, project, environment) the interruption relates to.",
+ "Form": "Form presented to the responder, with its submitted values.",
+ },
+ },
"events": {
"description": "Audit events record every significant action on the instance (deployments queued, releases created, settings changed).",
"docs_url": "https://octopus.com/docs/security/users-and-teams/auditing",
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/settings.py
index d4bf1a220a11..75a7801e2ff5 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/settings.py
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/settings.py
@@ -61,6 +61,22 @@ OCTOPUS_DEPLOY_ENDPOINTS: dict[str, OctopusDeployEndpointConfig] = {
name="machines",
path="/machines",
),
+ "lifecycles": OctopusDeployEndpointConfig(
+ name="lifecycles",
+ path="/lifecycles",
+ ),
+ "users": OctopusDeployEndpointConfig(
+ name="users",
+ path="/api/users",
+ space_scoped=False,
+ ),
+ # Interruptions have no server-side date filter, and IsPending / ResponsibleUserId change as
+ # they are handled, so full refresh only. Created is set once.
+ "interruptions": OctopusDeployEndpointConfig(
+ name="interruptions",
+ path="/interruptions",
+ partition_key="Created",
+ ),
# Releases and deployments expose no server-side date filter (only project/environment
# filters), so they are full-refresh only. Their creation timestamps are immutable and make
# stable partition keys.
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/tests/test_octopus_deploy.py b/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/tests/test_octopus_deploy.py
index 9be2c7b69ecc..8cb5ed139817 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/tests/test_octopus_deploy.py
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/octopus_deploy/tests/test_octopus_deploy.py
@@ -127,16 +127,18 @@ class TestOctopusDeploySourceResponse:
"endpoint, primary_keys, partition_key",
[
("spaces", ["Id"], None),
+ ("users", ["Id"], None),
("projects", ["SpaceId", "Id"], None),
("releases", ["SpaceId", "Id"], "Assembled"),
("deployments", ["SpaceId", "Id"], "Created"),
("tasks", ["SpaceId", "Id"], "QueueTime"),
("events", ["SpaceId", "Id"], "Occurred"),
+ ("interruptions", ["SpaceId", "Id"], "Created"),
],
)
def test_response_shape(self, endpoint, primary_keys, partition_key):
script = [_page([], has_next=False)]
- if endpoint != "spaces":
+ if endpoint not in ("spaces", "users"):
script = [_page([{"Id": "Spaces-1"}], has_next=False), *script]
result = _driver().run(endpoint, script)
assert result.raised is None
@@ -204,13 +206,14 @@ class TestGetRows:
# Spaces-1 was already completed before the crash — not re-fetched.
assert "/api/Spaces-1/projects" not in result.paths
- def test_instance_level_endpoint_skips_space_fan_out(self):
- responses = [_page([{"Id": "Spaces-1"}, {"Id": "Spaces-2"}], has_next=False)]
- result = _driver().run("spaces", responses)
+ @pytest.mark.parametrize("endpoint, path", [("spaces", "/api/spaces"), ("users", "/api/users")])
+ def test_instance_level_endpoint_skips_space_fan_out(self, endpoint, path):
+ responses = [_page([{"Id": "A-1"}, {"Id": "A-2"}], has_next=False)]
+ result = _driver().run(endpoint, responses)
assert result.raised is None
- assert result.paths == ["/api/spaces"]
- assert [row["Id"] for row in result.rows] == ["Spaces-1", "Spaces-2"]
+ assert result.paths == [path]
+ assert result.rows == [{"Id": "A-1"}, {"Id": "A-2"}]
def test_incremental_filter_applied_to_every_page(self):
responses = [