:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(warehouse-sources): add develocity build tests and failures tables (#119247)
Tom Owers21b5fedd41c1
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.mdindex bac7f0928ccd..61b96721af18 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md@@ -3718,15 +3718,15 @@ Note: `Groups` covers all three group operations. `/v1/mgmt/group/all` returns e## Develocity — **thin**-Today (3): `builds`, `gradle_builds`, `maven_builds`+Today (5): `build_failures`, `build_tests`, `builds`, `gradle_builds`, `maven_builds`Diffed against: <https://docs.develocity.ai/downloads/develocity-api-specification/develocity-api-specification-2026.3.0.yaml>-- [ ] `Build tests (GET /api/tests/build/{id})` — analyze individual test outcomes, durations, and retries beyond aggregate build test performance (high)-- [ ] `Failure groups (GET /api/failures/groups)` — track recurring build and test failures, affected builds, and first and last occurrences (high)-- [ ] `Test cases (GET /api/tests/cases)` — measure failure and flakiness rates for individual tests across builds (medium)+- [x] `Build tests (GET /api/tests/build/{id})` — analyze individual test outcomes, durations, and retries beyond aggregate build test performance (high). Added as `build_tests` (fan-out over `builds`).+- ~~`Failure groups (GET /api/failures/groups)` — track recurring build and test failures, affected builds, and first and last occurrences (high)~~ — skipped: groups have no identifier, their counts depend on the `buildStartTime` window of the query, and the response has no pagination past 1000 groups. `build_failures` carries the per-build failure rows.+- ~~`Test cases (GET /api/tests/cases)` — measure failure and flakiness rates for individual tests across builds (medium)~~ — skipped: it takes one exact container name per call (so a fan-out over every container) and returns only outcome counts for the query window. `build_tests` carries the per-build outcomes these counts come from.- [ ] `Test containers (GET /api/tests/containers)` — compare test suite outcomes and locate containers with persistent failures or flaky tests (medium)-- [ ] `Build failures (GET /api/failures/builds/{id})` — attach failure messages and affected tasks or tests to imported builds (medium)+- [x] `Build failures (GET /api/failures/builds/{id})` — attach failure messages and affected tasks or tests to imported builds (medium). Added as `build_failures` (fan-out over `builds`).- [ ] `Gradle dependencies (GET /api/builds/{id}/gradle-dependencies)` — analyze dependency versions and origins across Gradle builds (medium)- [ ] `Maven dependencies (GET /api/builds/{id}/maven-dependencies)` — analyze dependency versions and origins across Maven builds (medium)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/develocity/canonical_descriptions.py b/products/warehouse_sources/backend/temporal/data_imports/sources/develocity/canonical_descriptions.pyindex ae01b3b715ec..e1ba1d17d52a 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/develocity/canonical_descriptions.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/develocity/canonical_descriptions.py@@ -3,7 +3,7 @@ from products.warehouse_sources.backend.temporal.data_imports.sources.common.can)from products.warehouse_sources.backend.temporal.data_imports.sources.develocity.settings import API_DOCS_URL-CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {+BUILDS_DESCRIPTIONS: CanonicalDescriptions = {name: {"description": description,"docs_url": API_DOCS_URL,@@ -28,3 +28,30 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {"maven_builds": "Maven Build Scans with build attributes, cache performance, and test performance.",}.items()}++CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {+ **BUILDS_DESCRIPTIONS,+ "build_tests": {+ "description": "Test executions of each build, one row per build, with every test work unit, container, and test case.",+ "docs_url": API_DOCS_URL,+ "columns": {+ "buildId": "The Build Scan identifier of the build that ran the tests.",+ "availableAt": "Time when Develocity finished receiving and processing the build, in milliseconds since the Unix epoch.",+ "summary": "Total and serial test duration, and outcome counts for test cases and test containers.",+ "workUnits": "Test work units of the build, each with its outcome, duration, and nested tests. Every test lists its executions, so retries appear as several executions.",+ },+ },+ "build_failures": {+ "description": "Build and test failures of each build, one row per build. Develocity marks the Failures API as beta.",+ "docs_url": API_DOCS_URL,+ "columns": {+ "buildId": "The Build Scan identifier of the failed or successful build.",+ "availableAt": "Time when Develocity finished receiving and processing the build, in milliseconds since the Unix epoch.",+ "buildToolType": "The tool that ran the build.",+ "gradle": "Build failures with task paths, and test failures, of a Gradle build. Null for other build tools.",+ "maven": "Build failures with goal executions, and test failures, of a Maven build. Null for other build tools.",+ "npm": "Build failures with process names, and test failures, of an npm build. Null for other build tools.",+ "docker": "Build failures with build steps of a Docker build. Null for other build tools.",+ },+ },+}diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/develocity/develocity.py b/products/warehouse_sources/backend/temporal/data_imports/sources/develocity/develocity.pyindex 27b960f408e2..2b9325a4309d 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/develocity/develocity.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/develocity/develocity.py@@ -10,25 +10,46 @@ from products.warehouse_sources.backend.temporal.data_imports.sources.common.resrest_api_resource,)from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.auth import BearerTokenAuth+from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.fanout import (+ DependentEndpointConfig,+ build_dependent_resource,+)from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.paginators import (SinglePagePaginator,)from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.rest_client import RESTClient-from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.typing import EndpointResource+from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.typing import (+ ClientConfig,+ Endpoint,+ EndpointResource,+ JSONResponseCursorPaginatorConfig,+)from products.warehouse_sources.backend.temporal.data_imports.sources.common.resumable import ResumableSourceManagerfrom products.warehouse_sources.backend.temporal.data_imports.sources.common.schema import schema_for_resourcefrom products.warehouse_sources.backend.temporal.data_imports.sources.common.typings import SourceInputs, SourceResponsefrom products.warehouse_sources.backend.temporal.data_imports.sources.develocity.settings import (+ AVAILABLE_AT_FIELDS,+ BUILD_ID_FIELD,ENDPOINTS,INVALID_KEY,INVALID_URL,MISSING_PERMISSION,+ MODEL_NOT_APPLICABLE,PAGE_SIZE,- PRIMARY_KEYS,+ BuildChildEndpoint,+ BuildEndpoint,)from products.warehouse_sources.backend.temporal.data_imports.sources.generated_configs.develocity import (DevelocitySourceConfig,)+from products.warehouse_sources.backend.types import IncrementalField++BUILDS_PAGINATOR: JSONResponseCursorPaginatorConfig = {+ "type": "cursor",+ "cursor_path": "$[-1].id",+ "cursor_param": "fromBuild",+ "raise_on_repeated_cursor": True,+}@frozen@@ -36,6 +57,15 @@ class DevelocityResumeConfig:cursor: str+@frozen+class FanoutEndpoint:+ name: str+ path: str+ incremental_fields: list[IncrementalField]+ default_incremental_field: str | None = "availableAt"+ page_size: int = PAGE_SIZE++def validated_api_url(instance_url: str, team_id: int) -> str:from products.warehouse_sources.backend.temporal.data_imports.sources.common.mixins import ( # noqa: PLC0415 -- avoids loading Django models during source discoveryValidateDatabaseHostMixin,@@ -65,16 +95,21 @@ def validated_api_url(instance_url: str, team_id: int) -> str:return f"https://{parts.netloc}/api/"-def get_resource(name: str, incremental: bool, last_value: int | str | None) -> EndpointResource:- endpoint = schema_for_resource(ENDPOINTS, name)+def builds_params(incremental: bool, last_value: int | str | None) -> dict[str, Any]:# The API excludes the starting instant. Re-read one millisecond to retain builds with equal timestamps.start = max(0, int(last_value) - 1) if incremental and last_value is not None else 0- params: dict[str, Any] = {+ return {"fromInstant": start,"reverse": "false","maxBuilds": PAGE_SIZE,"maxWaitSecs": 1,}+++def get_resource(+ endpoint: BuildEndpoint, name: str, incremental: bool, last_value: int | str | None+) -> EndpointResource:+ params = builds_params(incremental, last_value)if endpoint.build_tool:params["query"] = f"buildTool:{endpoint.build_tool}"params["models"] = list(endpoint.models)@@ -86,16 +121,51 @@ def get_resource(name: str, incremental: bool, last_value: int | str | None) ->"params": params,"data_selector": "$","data_selector_required": True,- "paginator": {- "type": "cursor",- "cursor_path": "$[-1].id",- "cursor_param": "fromBuild",- "raise_on_repeated_cursor": True,- },+ "paginator": BUILDS_PAGINATOR,},}+def build_child_resource(+ endpoint: BuildChildEndpoint, name: str, client_config: ClientConfig, inputs: SourceInputs+) -> Any:+ incremental = inputs.should_use_incremental_field+ fanout = DependentEndpointConfig(+ parent_name="builds",+ resolve_param="id",+ resolve_field="id",+ include_from_parent=["id", "availableAt"],+ parent_field_renames={"id": BUILD_ID_FIELD, "availableAt": "availableAt"},+ # The child endpoints have no time filter, so the builds listing bounds an incremental sync.+ parent_params=builds_params(incremental, inputs.db_incremental_field_last_value),+ child_params=dict(endpoint.params),+ child_response_actions=[+ {"status_code": 400, "json_field": "type", "json_values": [MODEL_NOT_APPLICABLE], "action": "ignore"}+ ],+ )+ parent_extra: Endpoint = {"data_selector": "$", "data_selector_required": True, "paginator": BUILDS_PAGINATOR}+ child_extra: Endpoint = {"data_selector": "$", "data_selector_required": True, "paginator": "single_page"}+ return build_dependent_resource(+ endpoint_configs={+ "builds": FanoutEndpoint(name="builds", path="builds", incremental_fields=AVAILABLE_AT_FIELDS),+ name: FanoutEndpoint(name=name, path=endpoint.path, incremental_fields=AVAILABLE_AT_FIELDS),+ },+ child_endpoint=name,+ fanout=fanout,+ client_config=client_config,+ path_format_values={},+ team_id=inputs.team_id,+ job_id=inputs.job_id,+ db_incremental_field_last_value=inputs.db_incremental_field_last_value,+ should_use_incremental_field=incremental,+ incremental_field=inputs.incremental_field,+ incremental_config_factory=lambda _field: None,+ page_size_param=None,+ parent_endpoint_extra=parent_extra,+ child_endpoint_extra=child_extra,+ )++def validate_credentials(config: DevelocitySourceConfig, team_id: int, schema_name: str | None = None) -> tuple[bool, str | None]:@@ -140,19 +210,29 @@ def develocity_source(inputs: SourceInputs,manager: ResumableSourceManager[DevelocityResumeConfig],) -> SourceResponse:+ endpoint = schema_for_resource(ENDPOINTS, inputs.schema_name)+ client_config: ClientConfig = {+ "base_url": validated_api_url(config.instance_url, inputs.team_id),+ "auth": {"type": "bearer", "token": config.access_key},+ "allowed_hosts": [],+ "allow_redirects": False,+ "request_timeout": 60,+ }+ if isinstance(endpoint, BuildChildEndpoint):+ # No build cursor is saved, so the pipeline must checkpoint the availableAt watermark instead.+ child = build_child_resource(endpoint, inputs.schema_name, client_config, inputs)+ return SourceResponse(+ name=inputs.schema_name,+ items=lambda: child,+ primary_keys=[BUILD_ID_FIELD],+ sort_mode="asc",+ supports_resume=False,+ )+resource_config = get_resource(- inputs.schema_name, inputs.should_use_incremental_field, inputs.db_incremental_field_last_value+ endpoint, inputs.schema_name, inputs.should_use_incremental_field, inputs.db_incremental_field_last_value)- rest_config: RESTAPIConfig = {- "client": {- "base_url": validated_api_url(config.instance_url, inputs.team_id),- "auth": {"type": "bearer", "token": config.access_key},- "allowed_hosts": [],- "allow_redirects": False,- "request_timeout": 60,- },- "resources": [resource_config],- }+ rest_config: RESTAPIConfig = {"client": client_config, "resources": [resource_config]}saved = manager.load_state() if manager.can_resume() else Nonedef save_checkpoint(state: dict[str, Any] | None) -> None:@@ -170,6 +250,6 @@ def develocity_source(return SourceResponse(name=inputs.schema_name,items=lambda: resource,- primary_keys=PRIMARY_KEYS,+ primary_keys=["id"],sort_mode="asc",)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/develocity/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/develocity/settings.pyindex 468a4a50d918..9c65011339b8 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/develocity/settings.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/develocity/settings.py@@ -1,3 +1,5 @@+from typing import Any+from posthog.dataclasses import frozenfrom products.warehouse_sources.backend.types import IncrementalField, IncrementalFieldType@@ -12,7 +14,15 @@ class BuildEndpoint:models: tuple[str, ...] = ()-ENDPOINTS: dict[str, BuildEndpoint] = {+@frozen+class BuildChildEndpoint:+ """A per-build endpoint, fetched once for each build the `builds` listing returns."""++ path: str+ params: dict[str, Any]+++ENDPOINTS: dict[str, BuildEndpoint | BuildChildEndpoint] = {"builds": BuildEndpoint(),"gradle_builds": BuildEndpoint(build_tool="gradle",@@ -22,20 +32,25 @@ ENDPOINTS: dict[str, BuildEndpoint] = {build_tool="maven",models=("maven-attributes", "maven-build-cache-performance", "maven-test-performance"),),+ "build_tests": BuildChildEndpoint(path="tests/build/{id}", params={}),+ # Failures is a beta API. maxFailures caps the build and test failure lists separately.+ "build_failures": BuildChildEndpoint(path="failures/builds/{id}", params={"maxFailures": 1000}),}-PRIMARY_KEYS = ["id"]-INCREMENTAL_FIELDS: dict[str, list[IncrementalField]] = {- name: [- {- "label": "availableAt",- "type": IncrementalFieldType.Integer,- "field": "availableAt",- "field_type": IncrementalFieldType.Integer,- }- ]- for name in ENDPOINTS-}+AVAILABLE_AT_FIELDS: list[IncrementalField] = [+ {+ "label": "availableAt",+ "type": IncrementalFieldType.Integer,+ "field": "availableAt",+ "field_type": IncrementalFieldType.Integer,+ }+]+INCREMENTAL_FIELDS: dict[str, list[IncrementalField]] = dict.fromkeys(ENDPOINTS, AVAILABLE_AT_FIELDS)++# A child response carries no build identifier, so the parent build's id becomes the key.+BUILD_ID_FIELD = "buildId"+# Develocity answers this problem type when a build's tool does not produce the requested data.+MODEL_NOT_APPLICABLE = "urn:gradle:enterprise:api:problems:build-model-not-applicable"INVALID_URL = "Enter an HTTPS Develocity instance URL without a path, query, or embedded credentials."INVALID_KEY = "Your Develocity access key is invalid or expired. Generate a new key in My settings > Access keys."diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/develocity/tests/test_develocity.py b/products/warehouse_sources/backend/temporal/data_imports/sources/develocity/tests/test_develocity.pyindex 5657777a0c70..9f6cdb86ee9d 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/develocity/tests/test_develocity.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/develocity/tests/test_develocity.py@@ -18,6 +18,7 @@ from products.warehouse_sources.backend.temporal.data_imports.sources.common.resfrom products.warehouse_sources.backend.temporal.data_imports.sources.common.resumable import ResumableSourceManagerfrom products.warehouse_sources.backend.temporal.data_imports.sources.common.schema import UnknownResourceErrorfrom products.warehouse_sources.backend.temporal.data_imports.sources.common.testing import (+ RecordedRequest,ScriptedResponse,SourceDriver,always,@@ -144,6 +145,75 @@ class TestDevelocity(TestCase):assert isinstance(result.raised, ValueError) and "pagination is not advancing" in str(result.raised)assert len(result.requests) == (1 if resume else 2)+ @parameterized.expand(+ [+ ("build_tests", "/api/tests/build/", {}, False, None, "0"),+ ("build_failures", "/api/failures/builds/", {"maxFailures": "1000"}, False, None, "0"),+ ("build_tests", "/api/tests/build/", {}, True, 5000, "4999"),+ ("build_failures", "/api/failures/builds/", {"maxFailures": "1000"}, True, 5000, "4999"),+ ]+ )+ def test_child_tables_fan_out_over_builds(+ self,+ name: str,+ child_path: str,+ child_params: dict[str, str],+ incremental: bool,+ last_value: int | None,+ from_instant: str,+ ) -> None:+ builds_pages = {None: [build("build-one", 6000)], "build-one": [build("build-two", 7000)], "build-two": []}++ def answer(request: RecordedRequest) -> ScriptedResponse:+ if request.path == "/api/builds":+ return ScriptedResponse(json=builds_pages[request.param("fromBuild")])+ return ScriptedResponse(json={"buildToolType": "gradle", "path": request.path})++ result = self.driver.run(+ name,+ answer,+ should_use_incremental_field=incremental,+ db_incremental_field_last_value=last_value,+ incremental_field="availableAt",+ )+ assert result.raised is None+ assert result.rows == [+ {"buildToolType": "gradle", "path": f"{child_path}build-one", "buildId": "build-one", "availableAt": 6000},+ {"buildToolType": "gradle", "path": f"{child_path}build-two", "buildId": "build-two", "availableAt": 7000},+ ]+ builds_requests = [request for request in result.requests if request.path == "/api/builds"]+ assert [request.param("fromBuild") for request in builds_requests] == [None, "build-one", "build-two"]+ assert {request.param("fromInstant") for request in builds_requests} == {from_instant}+ for request in result.requests:+ if request.path != "/api/builds":+ assert {key: values[0] for key, values in request.query.items()} == child_params+ assert result.response is not None+ assert result.response.primary_keys == ["buildId"] and result.response.sort_mode == "asc"+ assert result.response.supports_resume is False+ assert result.saved_states == []++ @parameterized.expand(+ [+ ("urn:gradle:enterprise:api:problems:build-model-not-applicable", None),+ ("urn:gradle:enterprise:api:problems:validation", HTTPError),+ ]+ )+ def test_child_bad_request(self, problem_type: str, raised: type[BaseException] | None) -> None:+ def answer(request: RecordedRequest) -> ScriptedResponse:+ if request.path == "/api/builds":+ return ScriptedResponse(+ json=[build("docker-build", 1000)] if request.param("fromBuild") is None else []+ )+ if request.path.endswith("/docker-build"):+ return ScriptedResponse(json={"type": problem_type, "status": 400}, status=400)+ return ScriptedResponse(json={"buildToolType": "docker"})++ result = self.driver.run("build_failures", answer)+ if raised is None:+ assert result.raised is None and result.rows == []+ else:+ assert isinstance(result.raised, raised)+@parameterized.expand([(200, None, True, None),