Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

feat(phase-11): OpenAPI breaking-change gate (#436)

Beenu Arorae5b839e9655c
diff --git a/.github/workflows/openapi-breaking.yml b/.github/workflows/openapi-breaking.yml
new file mode 100644
index 000000000..18728d539
--- /dev/null
+++ b/.github/workflows/openapi-breaking.yml
@@ -0,0 +1,59 @@
+name: OpenAPI breaking-change gate
+
+# Phase 11 (world-class program) — check-openapi.yml proves the spec matches the
+# code (drift); this adds the missing breaking-change semantics. On a PR that
+# touches docs/openapi.yaml, diff the PR's spec against the base branch's spec
+# and fail on any change that would break an existing generated SDK client
+# (removed endpoint/schema/field, type change, tightened request, dropped enum
+# value). Ship a breaking change deliberately by bumping the version and adding
+# a CHANGELOG BREAKING note (then a maintainer re-runs with --allow-breaking).
+
+on:
+ pull_request:
+ paths:
+ - "docs/openapi.yaml"
+ - "scripts/openapi_diff.py"
+ - "tests/test_openapi_diff.py"
+ - ".github/workflows/openapi-breaking.yml"
+
+concurrency:
+ group: openapi-breaking-${{ github.ref }}
+ cancel-in-progress: true
+
+permissions:
+ contents: read
+
+jobs:
+ breaking-change:
+ name: No breaking OpenAPI changes vs base
+ runs-on: ubuntu-latest
+ timeout-minutes: 10
+ steps:
+ - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
+ with:
+ fetch-depth: 0
+
+ - uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
+ with:
+ python-version: "3.12"
+
+ - name: Install deps
+ run: pip install --quiet pyyaml pytest
+
+ - name: Unit-test the detector
+ run: python -m pytest tests/test_openapi_diff.py -v --tb=short
+
+ - name: Extract base-branch spec
+ run: |
+ set -euo pipefail
+ BASE="${{ github.event.pull_request.base.sha }}"
+ if git cat-file -e "${BASE}:docs/openapi.yaml" 2>/dev/null; then
+ git show "${BASE}:docs/openapi.yaml" > /tmp/base-openapi.yaml
+ echo "extracted base spec from ${BASE}"
+ else
+ echo "no base spec (new file) — nothing to compare"
+ cp docs/openapi.yaml /tmp/base-openapi.yaml
+ fi
+
+ - name: Diff for breaking changes
+ run: python scripts/openapi_diff.py --old /tmp/base-openapi.yaml --new docs/openapi.yaml
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 4b1e889d1..2c445bccd 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
+- **Phase 11 — OpenAPI breaking-change gate.** `check-openapi.yml` proved the spec matches the code (drift) but had **no breaking-change semantics** — a PR could delete an endpoint, remove a response field, tighten a request body, or drop an enum value with every check green while every generated SDK client silently broke (the reality-audit `NO GATE` row). New `scripts/openapi_diff.py` (pyyaml-only) classifies the changes between two specs as breaking vs non-breaking *from an existing client's perspective*: removed path/operation/schema/property, changed property type signature, optional→required, a new required field on a request-shaped schema, a removed enum value, or a new required parameter. New `openapi-breaking.yml` diffs the PR's `docs/openapi.yaml` against the base branch and fails on any breaking change; a deliberate breaking release ships with a version bump + a CHANGELOG BREAKING note and `--allow-breaking`. 15 detector tests (`tests/test_openapi_diff.py`) prove every breaking class is caught **and** that safe additive changes (new path, new optional field, new enum value, new response field) are *not* flagged — a breaking-change gate that cries wolf gets disabled. Claim-to-gate matrix: "OpenAPI stability for 3 SDKs + MCP" NO GATE → GATED, and the **ratchet ceiling lowered 2 → 1** (15 GATED / 11 PARTIAL / 1 NO GATE — only the wet-eval live-agent scoreboard tables remain, closing in Phase 4c with a budgeted live run). Per-language SDK generated-client contract-drift is tracked as 11b.
- **Phase 10 — connector runtime-contract conformance suite + published matrix.** The reality audit left the "live Test connection" click-and-connect claim with **NO gate at all**, and the schema/vault claim only partially gated. New `scripts/connector_conformance.py` + `services/connectors/tests/test_conformance.py` gate the runtime contract across **all 69 connectors**: every connector must implement `test_connection` as an async coroutine (the contract behind the "live Test connection" button), implement `fetch_alerts` as an async coroutine, declare only valid `Capability` verbs, and **mark every secret-shaped field `type="secret"`** — a field named `api_key`/`token`/`password` rendered as a plain `string` would be stored outside the vault, the exact leak this check prevents. The published `docs/connectors/conformance-matrix.md` (69/69 conform) is drift-gated by `--check`, so a new connector cannot land without conforming and the matrix can't diverge from the registry. Claim-to-gate matrix: "Connectors: live Test connection" NO GATE → PARTIAL, and the **ratchet ceiling lowered 3 → 2** (14 GATED / 11 PARTIAL / 2 NO GATE). Detection-content lifecycle is already gated by Phase 4a (DAC candidate-rule) + 4b (truth table). Live-vendor sandbox smoke + rate-limit/checkpoint durability are tracked as 10b in `docs/audit/PROGRESS.md`.
- **Phase 9a — autonomy-safety policy + honest rollback contract + scorecard.** The reality audit found three holes behind "L0-L4 automation maturity gates every action": dry-run was opt-in (a mis-configured caller executes for real), ~15 executor `rollback()`s silently `return True` with no reverse vendor call (the platform claimed to reverse containment it never did), and there was no post-action verification. New `services/actions/app/services/autonomy_safety.py` closes them at the policy layer: `decide()` makes **dry-run the safe default** — anything not explicitly permitted to auto-execute is previewed (`DRY_RUN`), never silently executed; CRITICAL blast never auto-executes; HIGH only at L4 with a whitelist entry AND the `AISOC_ALLOW_HIGH_BLAST_AUTO` break-glass flag. `rollback_capability()` with a **pinned `REVERSIBLE_ACTIONS={block_ip}`** set makes the rollback claim honest and bounded — a caller learns "this cannot be auto-reversed" instead of a silent `True`, and the set can't grow without a conscious edit + real reverse implementation. Every unattended containment (`AUTO`, blast ≥ MEDIUM) sets `requires_verification`, and `AutonomyScorecard` counts executions that were never verified — and executions of irreversible actions — as **visible gaps** rather than assumed-away. 16 tests (`services/actions/tests/test_autonomy_safety.py`, in the actions coverage matrix). Enforcement-wiring of `decide()` into the live `/dispatch` + `submit_action` router, rewriting the silent-`return True` executor rollbacks, real vendor verifiers, and a durable approval-SLA timer table (replacing the in-memory `approval_timeout.py`) are tracked as 9b in `docs/audit/PROGRESS.md`.
- **Phase 8 — LLMOps: prompt registry, model pins, response cache, structured-output validation.** A coherent `services/agents/app/llm/` LLMOps layer, all dependency-light and gated. (1) **Prompt registry** (`prompt_registry.py`) — every production prompt is a named, versioned, sha256-hashed artifact; the committed `prompts.lock.json` pins version→hash, and `scripts/check_prompt_lock.py --check` (wired into the CI lint job) fails when a prompt's text changes without a version bump + lock regeneration. This makes the `AGENTS.md` "prompt change ⇒ re-grade the eval harness" rule *enforceable* — you can no longer edit a prompt silently. (2) **Model pins + provider fallback** (`model_pins.py`) — logical roles pinned to concrete models with ordered fallback chains that ALWAYS terminate in the deterministic tier (env-overridable primaries, but the deterministic floor is gate-enforced), replacing the scattered `os.getenv("...", "gpt-4o-mini")` defaults. (3) **Content-addressed response cache** (`response_cache.py`) — keyed on sha256(model+prompt+input) with field separation and LRU eviction; safe under the determinism contract (a hit is byte-identical). (4) **Fail-closed structured-output validation** (`structured_output.py`) — strips code fences/prose, parses JSON, validates against a caller schema, and on ANY failure returns a deterministic fallback rather than propagating a half-parsed object into an autonomy decision. 15 tests (`services/agents/tests/test_llm_ops.py`, appended to the CI agents gate). Migration of the inline agent prompts to `registry.get()` is tracked as 8b. Doc: `apps/docs/docs/concepts/llmops.md`.
diff --git a/ROADMAP.md b/ROADMAP.md
index 46e500a10..cfd4240bc 100644
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -33,7 +33,7 @@ A proof-first, security-first program to make every README claim gate-backed, cl
- [x] Phase 8 — LLMOps (version+hash-pinned prompt registry with a CI drift gate; model pins + deterministic-terminated provider-fallback chains; content-addressed response cache; fail-closed structured-output validation)
- [x] Phase 9 — Autonomy safety (dry-run-by-default policy, honest bounded rollback-capability contract replacing silent `return True`, mandatory post-action verification for unattended containment, break-glass HIGH-blast flag, autonomy scorecard — all gated at the policy layer; live-router wiring + durable approval-SLA timer table tracked as 9b)
- [x] Phase 10 — Connector + content quality (runtime-contract conformance suite across all 69 connectors — gates the "live Test connection" capability that had NO gate + secret-field-marking; published `conformance-matrix.md` with a drift gate; detection lifecycle already gated by Phase 4a DAC + Phase 4b truth table. Live-vendor sandbox smoke + rate-limit/checkpoint durability tracked as 10b)
-- [ ] Phase 11 — API / SDK / release engineering (oasdiff, SDK contract drift, versioning, release discipline)
+- [x] Phase 11 — API / SDK / release engineering (pure-Python OpenAPI breaking-change detector `scripts/openapi_diff.py` + `openapi-breaking.yml` gate: PR spec vs base fails on removed endpoint/schema/field, type change, tightened request, or dropped enum value — closes the OpenAPI NO GATE row; per-language SDK generated-client contract-drift tracked as 11b)
- [ ] Phase 12 — Observability + governance (SLOs, single trace, GOVERNANCE.md, DCO, committers, neutral home, trust docs)
## v4.0 — Shipped
diff --git a/docs/audit/CLAIM_TO_GATE_MATRIX.md b/docs/audit/CLAIM_TO_GATE_MATRIX.md
index d713765ca..93fc3cfa0 100644
--- a/docs/audit/CLAIM_TO_GATE_MATRIX.md
+++ b/docs/audit/CLAIM_TO_GATE_MATRIX.md
@@ -24,7 +24,7 @@ Statuses: `GATED` (a CI job fails when the claim stops being true) · `PARTIAL`
| Hunt-as-Code + `/hunt` | README L172 | `ci.yml :: p1-eval` (`hunt_corpus`) | GATED | - |
| Weekly benchmark scoreboard runs live against `main` | README L173 | `wet-eval.yml` (weekly) | NO GATE (no-ops without secret; live-agent tables are placeholders) | Phase 4 Tier 1 |
| MCP server exposes 13 tools | README L179 | `ci.yml :: mcp` | GATED | - |
-| Plugin SDK Python/TS/Go | README L79, L193 | `ci.yml :: sdk-*` | PARTIAL (build/test gated; contract-drift vs `docs/openapi.yaml` ungated) | Phase 11 |
+| Plugin SDK Python/TS/Go | README L79, L193 | `ci.yml :: sdk-*` | PARTIAL (build/test gated; OpenAPI breaking-change now gated via `openapi-breaking.yml`, so a spec change that would break the generated SDKs is caught; per-language generated-client contract-drift is 11b) | Phase 11b |
| Prompt-injection resistance | (implied by agent claims) | `ci.yml :: python-test` (agents) runs `test_prompt_sanitizer.py` + `test_prompt_envelope.py` | PARTIAL (unit-level nonce envelope + guard gated; 150-payload adversarial eval + tool-call provenance in Phase 4 Tier 2) | Phase 4 |
| Cross-tenant isolation (Postgres) | (implied by multi-tenant) | `cross-tenant-rbac.yml` (nightly, 3 endpoints) + `ci.yml` | PARTIAL (Postgres only, compiled-SQL not live DB) | Phase 1.3 |
| Cross-tenant isolation (Qdrant/Neo4j/Redis/ClickHouse/Kafka) | (implied by multi-tenant) | `isolation.yml` (offline: read paths construct a tenant scope) + `isolation-live.yml` (live A-vs-B replay: Neo4j property filter, Redis keyspace namespacing, ClickHouse via production `lake_sql.rewrite_for_tenant`, Kafka per-tenant envelope filter) | GATED | - |
@@ -34,13 +34,13 @@ Statuses: `GATED` (a CI job fails when the claim stops being true) · `PARTIAL`
| Container image / IaC / secret scanning (Trivy/checkov/tfsec/gitleaks/Semgrep) | (implied by security) | `security.yml` (claim-matrix ratchet HARD; gitleaks/Semgrep/Trivy/checkov/tfsec observe) | PARTIAL (matrix ratchet enforced; secret + code/IaC/container scanners report-and-ratchet, allowlist at `.security/allowlist.yml` / `.gitleaksignore`; GitHub push-protection is the always-on hard secret gate) | Phase 2 |
| Signed / attested release artifacts | (implied by "run next to crown jewels") | `release.yml` + `publish-images.yml` (cosign keyless signatures, CycloneDX SBOM attestation, BuildKit SLSA provenance + SPDX SBOM per image; SHA-pinned actions), `build-extensions.yml` (extension cosign) | GATED | - |
| Insecure defaults hard-fail in production | (implied by self-host) | `ci.yml` api tests (`test_security_defaults.py::test_enforce_*`; `enforce_secure_defaults` raises at boot in production) | GATED | - |
-| OpenAPI stability for 3 SDKs + MCP | (implied by SDKs) | `check-openapi.yml` (drift only) | NO GATE (no breaking-change semantics) | Phase 11 |
+| OpenAPI stability for 3 SDKs + MCP | (implied by SDKs) | `check-openapi.yml` (drift) + `openapi-breaking.yml` (`scripts/openapi_diff.py` — PR spec vs base fails on removed endpoint/schema/field, type change, tightened request, or dropped enum value) | GATED | - |
## Summary
-- GATED: 14
+- GATED: 15
- PARTIAL: 11
-- NO GATE: 2 (Phase 2 moved insecure-defaults-hard-fail to GATED and secret/IaC/container scanning to PARTIAL; the Phase 2 continuation moved signed/attested releases to GATED; Phase 3.4 moved cross-store isolation to GATED via the live-container replay; Phase 4 moved the DAC candidate-rule gate and the imported-count honesty gate to GATED; Phase 10 moved connector "live Test connection" NO GATE → PARTIAL via the conformance contract; remaining NO GATE — wet-eval live tables (Phase 4c) and OpenAPI breaking-change semantics (Phase 11) — close next)
+- NO GATE: 1 (Phase 2 moved insecure-defaults-hard-fail to GATED and secret/IaC/container scanning to PARTIAL; the Phase 2 continuation moved signed/attested releases to GATED; Phase 3.4 moved cross-store isolation to GATED; Phase 4 moved the DAC candidate-rule gate and imported-count honesty gate to GATED; Phase 10 moved connector "live Test connection" off NO GATE; Phase 11 moved OpenAPI breaking-change semantics to GATED. The **last** NO GATE row — wet-eval live-agent scoreboard tables — closes in Phase 4c, which needs a budgeted live-agent run)
The ratchet is enforced by `scripts/check_claim_gate_matrix.py` (wired into `security.yml`): the NO GATE count may only decrease.
diff --git a/scripts/check_claim_gate_matrix.py b/scripts/check_claim_gate_matrix.py
index a8fbdbeb9..1a10aaf65 100644
--- a/scripts/check_claim_gate_matrix.py
+++ b/scripts/check_claim_gate_matrix.py
@@ -27,9 +27,10 @@ MATRIX = ROOT / "docs" / "audit" / "CLAIM_TO_GATE_MATRIX.md"
# Ratchet baseline: the number of NO GATE rows allowed. Lower this as phases
# close gaps; never raise it. Phase 1 closed injection / non-Postgres isolation
# / no-exfiltration; Phase 2 closed insecure-defaults + secret/IaC scanning;
-# Phase 10 moved connector "live Test connection" off NO GATE. Remaining NO GATE
-# rows (wet-eval live tables, OpenAPI breaking-change) close in Phases 4c / 11.
-MAX_NO_GATE = 2
+# Phase 10 moved connector "live Test connection" off NO GATE; Phase 11 moved
+# OpenAPI breaking-change semantics to GATED. The last NO GATE row (wet-eval
+# live-agent scoreboard tables) closes in Phase 4c (needs a budgeted live run).
+MAX_NO_GATE = 1
def _parse_status_rows(text: str) -> list[str]:
diff --git a/scripts/openapi_diff.py b/scripts/openapi_diff.py
new file mode 100644
index 000000000..92b2eba66
--- /dev/null
+++ b/scripts/openapi_diff.py
@@ -0,0 +1,192 @@
+#!/usr/bin/env python3
+"""OpenAPI breaking-change detector (Phase 11 — API/SDK/release engineering).
+
+`check-openapi.yml` proves the committed spec matches the code (drift), but has
+**no breaking-change semantics** — the reality audit's `NO GATE` row. A PR could
+delete an endpoint, remove a response field, tighten a request, or drop an enum
+value and every check stays green while every generated SDK client silently
+breaks.
+
+This is a dependency-light (pyyaml only) diff that classifies the changes
+between two OpenAPI specs as **breaking** or non-breaking, from the perspective
+of an existing generated client. Breaking classes:
+
+* a path or operation (method) was removed,
+* a component schema was removed,
+* a property was removed from a schema (a response consumer relying on it breaks),
+* a property's type signature changed,
+* a property went optional → required, or a new required property was added to a
+ *request*-shaped schema (an existing caller's payload is now rejected),
+* an enum value was removed (a client with exhaustive handling breaks),
+* a new required parameter was added to an existing operation.
+
+`--old`/`--new` take spec files; exit 1 if any breaking change is found (unless
+`--allow-breaking`, which the release flow uses when it deliberately ships a
+major bump). The CI job (`openapi-breaking.yml`) diffs the PR spec against the
+base branch's spec, so a breaking change blocks the PR until it's intentional.
+"""
+
+from __future__ import annotations
+
+import argparse
+import sys
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any
+
+import yaml
+
+# Schema-name suffixes that indicate a request body (tightening these breaks
+# callers). Response-shaped schemas don't break a caller when a new optional
+# field appears, so a *new* required field only counts as breaking here.
+_REQUEST_SUFFIXES = ("Request", "Create", "Update", "In", "Input", "Payload", "Body")
+
+
+@dataclass(frozen=True)
+class Change:
+ breaking: bool
+ kind: str
+ detail: str
+
+
+def _type_signature(prop: dict[str, Any]) -> str:
+ """A comparable signature for a property's type. $ref, type, array item."""
+ if not isinstance(prop, dict):
+ return "any"
+ if "$ref" in prop:
+ return f"ref:{prop['$ref']}"
+ if "anyOf" in prop or "oneOf" in prop:
+ variants = prop.get("anyOf") or prop.get("oneOf") or []
+ return "union:" + ",".join(sorted(_type_signature(v) for v in variants))
+ t = prop.get("type")
+ if t == "array":
+ return f"array<{_type_signature(prop.get('items', {}))}>"
+ if t is not None:
+ return str(t)
+ return "any"
+
+
+def _schemas(spec: dict[str, Any]) -> dict[str, Any]:
+ return (spec.get("components") or {}).get("schemas") or {}
+
+
+def _op_params(op: dict[str, Any]) -> dict[str, bool]:
+ """name → required, for an operation's parameters."""
+ out: dict[str, bool] = {}
+ for p in op.get("parameters") or []:
+ if isinstance(p, dict) and "name" in p:
+ out[str(p["name"])] = bool(p.get("required", False))
+ return out
+
+
+def _is_request_schema(name: str) -> bool:
+ return any(name.endswith(sfx) for sfx in _REQUEST_SUFFIXES)
+
+
+def diff(old: dict[str, Any], new: dict[str, Any]) -> list[Change]:
+ changes: list[Change] = []
+
+ # ── paths + operations ──────────────────────────────────────────────────
+ old_paths = old.get("paths") or {}
+ new_paths = new.get("paths") or {}
+ _METHODS = {"get", "put", "post", "delete", "patch", "head", "options", "trace"}
+ for path, old_item in old_paths.items():
+ if path not in new_paths:
+ changes.append(Change(True, "path_removed", f"path '{path}' was removed"))
+ continue
+ new_item = new_paths[path]
+ for method in _METHODS:
+ if method in (old_item or {}):
+ if method not in (new_item or {}):
+ changes.append(Change(True, "operation_removed", f"{method.upper()} {path} was removed"))
+ continue
+ old_params = _op_params(old_item[method])
+ new_params = _op_params(new_item[method])
+ for name, required in new_params.items():
+ if required and name not in old_params:
+ changes.append(Change(True, "required_param_added", f"{method.upper()} {path}: new required parameter '{name}'"))
+ for name in old_params:
+ if not old_params[name] and new_params.get(name) is True:
+ changes.append(Change(True, "param_now_required", f"{method.upper()} {path}: parameter '{name}' became required"))
+
+ # ── component schemas ──────────────────────────────────────────────────
+ old_schemas = _schemas(old)
+ new_schemas = _schemas(new)
+ for name, old_schema in old_schemas.items():
+ if name not in new_schemas:
+ changes.append(Change(True, "schema_removed", f"schema '{name}' was removed"))
+ continue
+ new_schema = new_schemas[name]
+ old_props = old_schema.get("properties") or {}
+ new_props = new_schema.get("properties") or {}
+ old_required = set(old_schema.get("required") or [])
+ new_required = set(new_schema.get("required") or [])
+
+ for prop_name, old_prop in old_props.items():
+ if prop_name not in new_props:
+ changes.append(Change(True, "property_removed", f"{name}.{prop_name} was removed"))
+ continue
+ old_sig = _type_signature(old_prop)
+ new_sig = _type_signature(new_props[prop_name])
+ if old_sig != new_sig:
+ changes.append(Change(True, "property_type_changed", f"{name}.{prop_name}: {old_sig} → {new_sig}"))
+ # optional → required
+ if prop_name in new_required and prop_name not in old_required:
+ changes.append(Change(True, "property_now_required", f"{name}.{prop_name} became required"))
+
+ # new required property on a request-shaped schema tightens callers
+ if _is_request_schema(name):
+ for prop_name in new_required - old_required:
+ if prop_name not in old_props:
+ changes.append(Change(True, "required_property_added", f"{name}.{prop_name} is a new required request field"))
+
+ # enum narrowing (per property)
+ for prop_name, old_prop in old_props.items():
+ if prop_name not in new_props:
+ continue
+ old_enum = set(old_prop.get("enum") or []) if isinstance(old_prop, dict) else set()
+ new_enum = set(new_props[prop_name].get("enum") or []) if isinstance(new_props[prop_name], dict) else set()
+ if old_enum and (removed := old_enum - new_enum):
+ changes.append(Change(True, "enum_value_removed", f"{name}.{prop_name}: enum values removed {sorted(map(str, removed))}"))
+
+ return changes
+
+
+def load_spec(path: Path) -> dict[str, Any]:
+ return yaml.safe_load(path.read_text(encoding="utf-8")) or {}
+
+
+def main() -> int:
+ parser = argparse.ArgumentParser(description=__doc__)
+ parser.add_argument("--old", required=True, type=Path, help="baseline spec (e.g. base branch / last release)")
+ parser.add_argument("--new", required=True, type=Path, help="candidate spec (e.g. this PR)")
+ parser.add_argument("--allow-breaking", action="store_true", help="permit breaking changes (deliberate major bump)")
+ args = parser.parse_args()
+
+ old = load_spec(args.old)
+ new = load_spec(args.new)
+ changes = diff(old, new)
+ breaking = [c for c in changes if c.breaking]
+
+ if not breaking:
+ print(f"OK: no breaking OpenAPI changes ({len(changes)} total change(s))")
+ return 0
+
+ print(f"Found {len(breaking)} BREAKING OpenAPI change(s):", file=sys.stderr)
+ for c in breaking:
+ print(f" - [{c.kind}] {c.detail}", file=sys.stderr)
+
+ if args.allow_breaking:
+ print("--allow-breaking set: not failing (deliberate breaking release).", file=sys.stderr)
+ return 0
+ print(
+ "\nBreaking API changes break every generated SDK client. Either avoid the "
+ "break, or ship it deliberately with a version bump + CHANGELOG BREAKING note "
+ "and re-run with --allow-breaking.",
+ file=sys.stderr,
+ )
+ return 1
+
+
+if __name__ == "__main__":
+ raise SystemExit(main())
diff --git a/tests/test_openapi_diff.py b/tests/test_openapi_diff.py
new file mode 100644
index 000000000..db274c236
--- /dev/null
+++ b/tests/test_openapi_diff.py
@@ -0,0 +1,128 @@
+"""Phase 11 — OpenAPI breaking-change detector tests.
+
+Proves the detector flags every breaking class from an existing client's
+perspective and, crucially, does NOT flag safe additive changes (a breaking-
+change gate that cries wolf on every additive PR gets disabled).
+"""
+
+from __future__ import annotations
+
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "scripts"))
+
+import openapi_diff as od # noqa: E402
+
+
+def _spec(paths=None, schemas=None):
+ return {
+ "openapi": "3.1.0",
+ "paths": paths or {},
+ "components": {"schemas": schemas or {}},
+ }
+
+
+def _kinds(changes):
+ return {c.kind for c in changes if c.breaking}
+
+
+# ── Breaking classes ─────────────────────────────────────────────────────────
+
+
+def test_removed_path_is_breaking():
+ old = _spec(paths={"/a": {"get": {}}})
+ new = _spec(paths={})
+ assert "path_removed" in _kinds(od.diff(old, new))
+
+
+def test_removed_operation_is_breaking():
+ old = _spec(paths={"/a": {"get": {}, "post": {}}})
+ new = _spec(paths={"/a": {"get": {}}})
+ assert "operation_removed" in _kinds(od.diff(old, new))
+
+
+def test_removed_schema_is_breaking():
+ old = _spec(schemas={"User": {"properties": {"id": {"type": "string"}}}})
+ new = _spec(schemas={})
+ assert "schema_removed" in _kinds(od.diff(old, new))
+
+
+def test_removed_property_is_breaking():
+ old = _spec(schemas={"User": {"properties": {"id": {"type": "string"}, "email": {"type": "string"}}}})
+ new = _spec(schemas={"User": {"properties": {"id": {"type": "string"}}}})
+ assert "property_removed" in _kinds(od.diff(old, new))
+
+
+def test_property_type_change_is_breaking():
+ old = _spec(schemas={"User": {"properties": {"id": {"type": "string"}}}})
+ new = _spec(schemas={"User": {"properties": {"id": {"type": "integer"}}}})
+ assert "property_type_changed" in _kinds(od.diff(old, new))
+
+
+def test_optional_to_required_is_breaking():
+ old = _spec(schemas={"User": {"properties": {"name": {"type": "string"}}, "required": []}})
+ new = _spec(schemas={"User": {"properties": {"name": {"type": "string"}}, "required": ["name"]}})
+ assert "property_now_required" in _kinds(od.diff(old, new))
+
+
+def test_new_required_field_on_request_schema_is_breaking():
+ old = _spec(schemas={"LoginRequest": {"properties": {"email": {"type": "string"}}, "required": ["email"]}})
+ new = _spec(
+ schemas={"LoginRequest": {"properties": {"email": {"type": "string"}, "otp": {"type": "string"}}, "required": ["email", "otp"]}}
+ )
+ assert "required_property_added" in _kinds(od.diff(old, new))
+
+
+def test_enum_value_removal_is_breaking():
+ old = _spec(schemas={"Sev": {"properties": {"level": {"enum": ["low", "high", "critical"]}}}})
+ new = _spec(schemas={"Sev": {"properties": {"level": {"enum": ["low", "high"]}}}})
+ assert "enum_value_removed" in _kinds(od.diff(old, new))
+
+
+def test_new_required_parameter_is_breaking():
+ old = _spec(paths={"/a": {"get": {"parameters": []}}})
+ new = _spec(paths={"/a": {"get": {"parameters": [{"name": "tenant", "required": True}]}}})
+ assert "required_param_added" in _kinds(od.diff(old, new))
+
+
+# ── Safe additive changes must NOT be flagged ────────────────────────────────
+
+
+def test_added_path_is_not_breaking():
+ old = _spec(paths={"/a": {"get": {}}})
+ new = _spec(paths={"/a": {"get": {}}, "/b": {"get": {}}})
+ assert _kinds(od.diff(old, new)) == set()
+
+
+def test_added_optional_property_is_not_breaking():
+ old = _spec(schemas={"User": {"properties": {"id": {"type": "string"}}, "required": ["id"]}})
+ new = _spec(schemas={"User": {"properties": {"id": {"type": "string"}, "nickname": {"type": "string"}}, "required": ["id"]}})
+ assert _kinds(od.diff(old, new)) == set()
+
+
+def test_new_required_field_on_response_schema_is_not_breaking():
+ # A response gaining a field doesn't break a consumer; only request-shaped
+ # schemas tighten callers.
+ old = _spec(schemas={"UserResponse": {"properties": {"id": {"type": "string"}}, "required": ["id"]}})
+ new = _spec(
+ schemas={"UserResponse": {"properties": {"id": {"type": "string"}, "created": {"type": "string"}}, "required": ["id", "created"]}}
+ )
+ assert "required_property_added" not in _kinds(od.diff(old, new))
+
+
+def test_added_enum_value_is_not_breaking():
+ old = _spec(schemas={"Sev": {"properties": {"level": {"enum": ["low", "high"]}}}})
+ new = _spec(schemas={"Sev": {"properties": {"level": {"enum": ["low", "high", "critical"]}}}})
+ assert _kinds(od.diff(old, new)) == set()
+
+
+def test_ref_type_signature_change_is_breaking():
+ old = _spec(schemas={"Case": {"properties": {"owner": {"$ref": "#/components/schemas/User"}}}})
+ new = _spec(schemas={"Case": {"properties": {"owner": {"$ref": "#/components/schemas/Actor"}}}})
+ assert "property_type_changed" in _kinds(od.diff(old, new))
+
+
+def test_identical_specs_have_no_changes():
+ s = _spec(paths={"/a": {"get": {}}}, schemas={"User": {"properties": {"id": {"type": "string"}}}})
+ assert od.diff(s, s) == []