Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
Generate the connector-type union, and classify a gate by what it does (#816)
Beenu Arora034e374f761c
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex 7e484d152..71f81e070 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -288,6 +288,31 @@ jobs:# commit the result.- name: Verify connector count is in sync (Phase 1.1)run: python scripts/generate_connector_count.py --check+ # The count comes from the registry; so, now, does the console's+ # `ConnectorType` union. It used to be written by hand, which is how ten+ # members came to name nothing the platform ingests — `ibm_qradar`+ # reached no profile and no connector, so strict mode rejected those+ # events and lenient mode minted a vendor named after the spelling, a+ # second alert source for a QRadar deployment `qradar` already fed.+ #+ # The self-test runs first and on the same tree: it hand-edits the+ # generated file, deletes it, puts the union back in connector.ts and+ # flattens a canonical fold, and requires each to be caught.+ - name: Connector type union — self-test+ run: python3 scripts/generate_connector_types.py --self-test+ - name: Verify the ConnectorType union matches the registry+ run: python3 scripts/generate_connector_types.py --check+ # The properties that make generation safe to rely on, asserted rather+ # than assumed: regeneration is idempotent, reversing the registry+ # changes no byte (nothing is positional, so no lock file is needed),+ # and identity is read off the class — `jira_connector.py` declares+ # `jira` and `tenable.py` declares `tenable_io`, so a filename slug+ # would misname both, which is how `generate_connector_docs.py` came to+ # emit duplicate pages while reporting 100% coverage.+ - name: Connector type union — generation properties+ run: |+ python3 -m pip install --quiet pytest+ python3 -m pytest tests/test_connector_types_generated.py -q# The count says how many connectors exist; this says whether the name# each one is registered under still reaches a normalization profile.# The connectors service declares `crowdstrike` and `okta` while the@@ -331,6 +356,14 @@ jobs:# without anyone noticing.- name: Verify the funnel response contract is documentedrun: python scripts/check_funnel_contract.py+ # Phase 2.1 shipped this gate and left it unwired, with "wire this into+ # CI itself once the matrix has stabilised" in its own docstring. It+ # stayed unwired, so the audit it automates — a service that ships tests+ # no job collects — went back to being something a human had to notice.+ # Seven services were in that state before Phase 2.1, 971 tests between+ # them. The matrix has stabilised: all 13 tested services are gated.+ - name: Every tested Python service is gated by a CI job+ run: python3 scripts/list_python_services_with_tests.py --check# A support bundle is the most concentrated pile of configuration a# deployment produces, and its whole purpose is to leave the building.# Redaction is allow-list rather than deny-list, so a setting added@@ -571,12 +604,19 @@ jobs:# reach. The inverse direction too — a workflow step naming a# scripts/ path that no longer exists cannot do what it says.#+ # Which scripts it resolves is decided structurally — a declared+ # verdict flag, an exit status derived from findings the script+ # accumulates, or a workflow job whose output another job branches on.+ # It used to be decided by filename, which hid nineteen gates, fifteen+ # of them run by a workflow with `--check`.+ ## Self-test first, on the same tree, so a resolver that quietly starts# returning "reached" for everything fails here rather than printing OK.run: |- python -m pip install --quiet pyyaml+ python -m pip install --quiet pyyaml pytestpython3 scripts/check_gate_coverage.py --self-testpython3 scripts/check_gate_coverage.py --list+ python3 -m pytest tests/test_gate_coverage_classifier.py -q- name: Gate store migration runners and their call sites# Neo4j, ClickHouse and Qdrant hold schema that code depends on, anddiff --git a/CHANGELOG.md b/CHANGELOG.mdindex 5cf59fa17..336a368ec 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -77,6 +77,55 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0### Added+- **The console's `ConnectorType` union is generated from the connector+ registry.** Its members were corrected in the previous release; the+ mechanism that let them drift was not. Hand-maintenance is how ten of them+ came to name nothing the platform could ingest, and `ibm_qradar` is what+ that costs: no connector declared it and no profile was keyed on it, so+ strict mode rejected those events while lenient mode minted a vendor called+ "ibm_qradar" — a second alert source for the QRadar deployment `qradar`+ already fed.++ `scripts/generate_connector_types.py` derives the union from the three+ places that decide what the normalizer resolves: the `_CONNECTOR_CLASSES`+ tuple in `services/connectors/app/connectors/__init__.py` (84 ids), the+ `connectorProfiles` keys in the ingest normalizer (5 more that no connector+ declares under that spelling, including the legacy `splunk_enterprise`), and+ the `connectorTypeCanonical` fold sources (5 alternate spellings the console+ still emits). 94 members, written to+ `packages/types/src/generated/connector-types.ts` as a `CONNECTOR_TYPES`+ tuple the union is taken from, alongside a machine-readable+ `connector-types.json` recording each member's origin. `connector.ts`+ re-exports rather than redeclares, and `--check` fails if it goes back to+ declaring its own — a generated file can be perfectly current and completely+ ignored while a hand-written union three directories away is what TypeScript+ resolves.++ That set is, by construction, the `resolvable` set+ `scripts/check_connector_profiles.py` already computed, so the two gates now+ share one definition instead of holding two. The fold map is emitted with+ the union as `CONNECTOR_TYPE_CANONICAL`, so the console can resolve a stored+ spelling to one product name the same way the normalizer does.++ Nothing is positional, so no lock file is needed and the+ `generate_detections.py` renumbering trap does not apply: a member *is* its+ `connector_id` and the output is a sorted set. A test reverses the registry+ and asserts the output does not move a byte. Identity is read off the class,+ the way `_build_registry()` resolves it — `jira_connector.py` declares+ `jira` and `tenable.py` declares `tenable_io`, so a filename slug would+ misname both, which is the defect `generate_connector_docs.py` shipped when+ it generated duplicate pages for six connectors while reporting 100%+ coverage.++ The `palo_alto_cortex` fold was the one entry naming a vendor rather than a+ product, and Palo Alto ships two the platform ingests. It stays on+ `cortex_xdr`: the spelling entered the union in the initial-release commit+ before either connector existed, it appears nowhere else in the tree and+ never has (no console code, no saved instance, no seed, no catalog entry, no+ fixture), the console's own catalog names "Cortex XDR" under EDR and lists+ no XSIAM, and XSIAM is reachable under `cortex_xsiam` for any deployment+ that means it. The evidence is recorded beside the map in `normalizer.go`.+- **`scripts/check_gate_coverage.py` — the gate on the gates.** The mostexpensive recurring defect in this repository is a mechanism that exists, istested, and has no caller on the path that needs it. A gate is the worst case@@ -339,6 +388,46 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0### Changed+- **`check_gate_coverage.py` decides what a check is from what a script does,+ not what it is called.** It classified by filename — `check_*`,+ `validate_*`, `_conformance.py`, plus a hand-kept list of five exceptions+ for the ones whose names did not announce a verdict. That is the same defect+ the script exists to catch, one level up: a gate named something unexpected+ was simply not inventoried, and an uninventoried gate is indistinguishable+ from one that does not exist.++ Nineteen were in that state and every one is a CI gate. Fifteen are run by a+ workflow with `--check` — `generate_connector_count.py`,+ `generate_connector_docs.py`, `generate_detections.py`,+ `generate_slo_alerts.py`, the four `export_*` scripts, `build_marketplace.py`,+ `build_quarantine_index.py`, `curate_detections.py`, `project_stats.py`,+ `storage_cost_model.py` and two more. Deleting any of those steps would have+ left the script reporting full coverage over a smaller tree.++ Classification is now three structural signals, all read from the tree: a+ declared verdict flag (`--check`, `--verify`, `--strict`, `--fail-*`,+ `--max-*`, `--self-test`); an exit status derived from findings the script+ accumulates; or a workflow job whose output another job branches on, which+ is how `wet_eval_check.py` gates — it always exits 0 by design and publishes+ its verdict as a JSON status file. The first two are intrinsic, so a gate+ nothing calls is still inventoried and the "unreachable check" direction+ does not become vacuous.++ Polarity is the distinction that keeps it from over-reporting: `if+ offenders: return 1` is a finding, `if not specs: return 1` is a generator+ aborting on an empty read. A script that only talks to a running service is+ excluded for the same reason — its non-zero exit is an operational error,+ not a verdict on the tree.++ The classifier is checked in reverse too: a script CI runs with a verdict+ flag that the classifier does not inventory now fails as+ `classifier-blind-spot`, so the classifier cannot silently narrow. The+ inventory goes 42 → 59, and the first run of the new one found a real+ orphan — `list_python_services_with_tests.py`, whose own docstring says+ "wire this into CI itself once the matrix has stabilised" and which had+ stayed unwired. It is wired now, and passes: all 13 tested Python services+ are gated.+- **A hosted deployment's hostname no longer appears in self-hosted docs as thereader's own URL.** Seven files under `apps/docs/` pointed at the managedinstance: two `curl` examples told a self-hoster to push their SIEM data todiff --git a/CONTRIBUTING.md b/CONTRIBUTING.mdindex 881d6d2da..0a9c9c6d8 100644--- a/CONTRIBUTING.md+++ b/CONTRIBUTING.md@@ -328,6 +328,17 @@ Add your class to the `_CONNECTOR_CLASSES` tuple and `__all__` inThe registry powers `/connectors/schemas` and the wizard's catalog grid —no other wiring required.+Then regenerate the two things derived from that tuple, and commit the result:++```bash+python3 scripts/generate_connector_count.py # the "N connectors" claims+python3 scripts/generate_connector_types.py # the console's ConnectorType union+```++Both have a `--check` mode wired into CI, so forgetting is a red build rather+than a quiet drift. The union used to be hand-written and ten of its members+named nothing the platform could ingest.+### 3. Add a marketplace plugin manifestDrop a `plugins/<connector-id>/plugin.yaml` mirroring your `schema()`. Seediff --git a/apps/docs/docs/compliance/evidence-pack.md b/apps/docs/docs/compliance/evidence-pack.mdindex 97ec095ef..a44654b5b 100644--- a/apps/docs/docs/compliance/evidence-pack.md+++ b/apps/docs/docs/compliance/evidence-pack.md@@ -33,7 +33,7 @@ and says so.| Prompt-injection resistance | 32-payload adversarial corpus with a recall floor and a false-positive ceiling | `services/agents/tests/adversarial/` || Claim-to-gate traceability | Every product claim mapped to the CI job that fails when it stops being true | `docs/audit/CLAIM_TO_GATE_MATRIX.md` |-The last row is the one worth reading first. It is the honest index: 113 rows+The last row is the one worth reading first. It is the honest index: 115 rows`GATED`, 9 `PARTIAL` with the specific gap named, and a ratchet that refusesto let a row sit at `NO GATE`.diff --git a/apps/docs/docs/connectors/hello-connector.md b/apps/docs/docs/connectors/hello-connector.mdindex 574ed7850..3b25597d4 100644--- a/apps/docs/docs/connectors/hello-connector.md+++ b/apps/docs/docs/connectors/hello-connector.md@@ -265,11 +265,12 @@ When you're ready to ship a real connector, here is the exact checklist:1. **Move the file.** From `services/connectors/app/connectors/_examples/<your_connector>.py` to `services/connectors/app/connectors/<your_connector>.py`.2. **Register it.** Add the class to `_CONNECTOR_CLASSES` in `services/connectors/app/connectors/__init__.py`. Keep the tuple alphabetised by `connector_id` to keep diffs predictable. Add the class name to the `__all__` list at the bottom of the same file.-3. **Add a marketplace manifest.** Create `plugins/<connector-id>/plugin.yaml` mirroring your `schema()`. The fields and secret flags must match. Use [`plugins/auth0/plugin.yaml`](https://github.com/beenuar/AiSOC/blob/main/plugins/auth0/plugin.yaml) as a template.-4. **Sync the marketplace.** Run `pnpm marketplace:sync` from the repo root. This regenerates the static catalog under `apps/web/public/marketplace/` so the catalog grid in the UI picks up the new entry.-5. **Write a docs page.** Add `apps/docs/docs/connectors/<your-connector>.md` and wire it into `apps/docs/sidebars.ts` under the `Connectors` category. Use [`apps/docs/docs/connectors/cloudflare.md`](https://github.com/beenuar/AiSOC/blob/main/apps/docs/docs/connectors/cloudflare.md) as a template — `What you get` table → `Prerequisites` → `Setup walkthrough` → `Polling details` → `Severity heuristics` → `Troubleshooting`.-6. **Add real tests.** Use `respx` to mock the vendor API. At minimum, exercise `test_connection()` (success + auth failure + network failure), `fetch_alerts()` (happy path + empty + 5xx + auth expiry), and `normalize()` (every severity branch).-7. **Open a PR.** The code review will check that schema fields are wizard-renderable, that secrets are marked `secret=True`, that the category fits the existing taxonomy, that polling failures are non-terminal, and that the docs page exists.+3. **Regenerate what the registry derives.** Run `python3 scripts/generate_connector_count.py` and `python3 scripts/generate_connector_types.py`, then commit the result. The first keeps every "N connectors" claim honest; the second regenerates the console's `ConnectorType` union from your `connector_id`. Both have a `--check` mode in CI, so skipping this is a red build rather than a silent drift.+4. **Add a marketplace manifest.** Create `plugins/<connector-id>/plugin.yaml` mirroring your `schema()`. The fields and secret flags must match. Use [`plugins/auth0/plugin.yaml`](https://github.com/beenuar/AiSOC/blob/main/plugins/auth0/plugin.yaml) as a template.+5. **Sync the marketplace.** Run `pnpm marketplace:sync` from the repo root. This regenerates the static catalog under `apps/web/public/marketplace/` so the catalog grid in the UI picks up the new entry.+6. **Write a docs page.** Add `apps/docs/docs/connectors/<your-connector>.md` and wire it into `apps/docs/sidebars.ts` under the `Connectors` category. Use [`apps/docs/docs/connectors/cloudflare.md`](https://github.com/beenuar/AiSOC/blob/main/apps/docs/docs/connectors/cloudflare.md) as a template — `What you get` table → `Prerequisites` → `Setup walkthrough` → `Polling details` → `Severity heuristics` → `Troubleshooting`.+7. **Add real tests.** Use `respx` to mock the vendor API. At minimum, exercise `test_connection()` (success + auth failure + network failure), `fetch_alerts()` (happy path + empty + 5xx + auth expiry), and `normalize()` (every severity branch).+8. **Open a PR.** The code review will check that schema fields are wizard-renderable, that secrets are marked `secret=True`, that the category fits the existing taxonomy, that polling failures are non-terminal, and that the docs page exists.That's the full path. If any of those steps feels heavier than you expected, file a bug — the goal is for adding a connector to feel mechanical, not heroic.diff --git a/docs/audit/CLAIM_TO_GATE_MATRIX.md b/docs/audit/CLAIM_TO_GATE_MATRIX.mdindex 14dbcc7a5..80e5cac7e 100644--- a/docs/audit/CLAIM_TO_GATE_MATRIX.md+++ b/docs/audit/CLAIM_TO_GATE_MATRIX.md@@ -50,6 +50,7 @@ Statuses: `GATED` (a CI job fails when the claim stops being true) · `PARTIAL`| Unified Data Explorer (NL + SQL across the lake) replaces the SIEM context-switch | README (data explorer) | `ci.yml` web-test job (`ExploreView.test.tsx` — NL question → `nl-query/translate` → `lake/sql` → BI table; raw-SQL run; source pivots to identity/graph/intel; error surfaced) + web-lint type-check | GATED | - || Autonomy posture is copilot-by-default with a visible per-action/blast scorecard | README (autopilot/copilot) | `ci.yml` web-test job (`AutonomyScorecard.test.tsx` — posture stays Copilot unless a high/critical-blast action auto-executes; distribution + override counts; rendered badge) | GATED | - || Broad SIEM/NDR/edge coverage (QRadar, Exabeam, Securonix, Devo, Netskope, Windows/Sysmon, Zeek/Suricata, syslog/CEF) | README (connector count) | `ci.yml` connectors job (`test_schemas.py` + `test_conformance.py` gate every connector; `test_siem_expansion.py` + `test_ndr_edge_expansion.py` pin each new connector's severity-ladder mapping) + `generate_connector_count.py --check` (registry-derived count, README phrase) | GATED | - |+| The console's `connector_type` vocabulary names only connectors the platform can ingest | `packages/types/src/generated/connector-types.ts` (the exported `ConnectorType` union) | `ci.yml` python-lint job (`scripts/generate_connector_types.py --self-test` then `--check` — the union is generated from `_CONNECTOR_CLASSES`, the `connectorProfiles` keys and the `connectorTypeCanonical` folds, so it cannot be hand-edited into naming something nothing resolves; the self-test hand-edits the output, deletes it, puts a union back in `connector.ts` and flattens a fold, and requires each to be caught) + `check_connector_profiles.py` (every member resolved against the normalizer and the registry, in both directions) + `tests/test_connector_types_generated.py` (13 cases: regeneration is idempotent, reversing the registry changes no byte, identity is read off the class so `jira_connector.py` -> `jira` and `tenable.py` -> `tenable_io` are not misnamed, and a new connector shows up as drift) | GATED | - || AI/LLM-usage governance: audit connector + detections + tiered lake storage | README (AI usage / storage) | `ci.yml` connectors job (`test_llm_usage.py` — OpenAI/Anthropic audit connector emits the dotted `event_type` the `llm-*` rules match) + `validate-detections.yml` (8 `llm-*` native rules replay) + api job (`test_storage_tiering.py` — hot/cold TTL-move policy + DDL gate; verified on live ClickHouse) | GATED | - || Outcomes compound: repeat alerts auto-suppressed from prior dispositions (measured, not estimated) | README (closed loop) | `ci.yml` agents job (`test_outcome_memory.py` — every durable triage outcome is written back as a per-signature prior; a later alert matching a trusted prior benign/FP disposition is auto-resolved WITHOUT re-triage; human priors trusted immediately, AI priors require corroboration, a prior TP never auto-closes) + api funnel `repeat_alerts_suppressed`/`repeat_suppression_rate` measured from `aisoc_outcome_suppressions` | GATED | - || Institutional memory nudges fuse-time confidence (bounded, per-tenant) | README (compounding memory) | `ci.yml` fusion job (`test_memory_nudge.py` — distilled per-signature priors nudge new alerts of that signature ±0.10 in `ConfidenceScorer`; unknown signatures unchanged; `MemoryPriorProvider` distils per-tenant priors from disposition history so one tenant never nudges another) | GATED | - |@@ -130,10 +131,11 @@ Statuses: `GATED` (a CI job fails when the claim stops being true) · `PARTIAL`| A step a playbook engine cannot run never reports that it ran | concepts/playbooks.md (error handling) | `ci.yml` agents job (`test_playbook_engine_correctness.py::TestUnimplementedStepTypesFailClosed` — an `approval` gate the engine cannot honour halts the run instead of passing it, every handler-less `StepType` is asserted to fail rather than succeed, a missing handler is not retried, an explicit `on_failure: continue` is still honoured, and a dry run reports `would_fail` rather than a bare `dry_run: true`) | GATED | - || Every `ActionType` resolves a capability contract, so the confidence matrix is never skipped | concepts/live-actions.md (the one alias, and when it goes) | `ci.yml` actions job (`scripts/check_action_contract.py` directions 7 and 8 — every `ActionType` must resolve a contract by name or alias, and every alias must key a real `ActionType`, point at a real contract, not shadow a capability of the same name, and describe a bridge some adapter actually implements) + `test_approval_gate_wiring.py` (`notify_slack` at 10% confidence was approved for auto-execution and now requires an analyst; the alias is asserted unable to *lower* a requirement) | GATED | one alias entry (`notify_slack` -> `notify`), with its retirement condition recorded: it goes when `ActionType` does || Every check this matrix names is actually run by CI | this matrix (each row names a gate) | `ci.yml` (`scripts/check_gate_coverage.py` — resolves the whole graph rather than reading workflow names: workflows, `make` recipes they invoke, the pytest suites they collect and the scripts those reach, so a check wired only through a test still counts. Both directions — a check no workflow reaches, and a workflow step naming a `scripts/` path that does not exist. `--self-test` runs first and asserts the resolver discriminates rather than returning "reached" for everything, and the corpus counts are printed so it cannot report on a tree it never opened) | GATED | the ratchet for a deliberately-unwired check is empty; it may shrink, never grow |+| What counts as a check is decided by what a script does, not what it is named | `scripts/check_gate_coverage.py` (the inventory it reports on) | `ci.yml` (three structural signals, all read from the tree: a declared verdict flag, an exit status derived from findings the script accumulates, and a workflow job whose output another job branches on. The first two are intrinsic, so an uncalled gate is still inventoried and the `SCRIPT -> WORKFLOW` direction is not vacuous. The classifier is also checked in reverse — a script CI runs with `--check` that the classifier does not inventory fails as `classifier-blind-spot`) + `tests/test_gate_coverage_classifier.py` (10 cases, including a gate planted under a name no convention would reveal, and a generator that aborts on an empty read asserted *not* to be a check) | GATED | - |## Summary-- GATED: 113+- GATED: 115- PARTIAL: 9- NO GATE: 0 (**every claim is backed by a failing test.** The last NO GATE — the weekly benchmark scoreboard running live against `main` — closed in Phase E1: `scripts/check_scoreboard.py` ties the published scoreboard to a deterministic per-PR live-agent MITRE-accuracy run, while the funded weekly `wet-eval.yml` appends the LLM-tier rows. The remaining PARTIAL rows are honest, named deferrals — each states the specific gap and what closes it — not unproven claims.)diff --git a/packages/types/src/connector.ts b/packages/types/src/connector.tsindex 73d55949b..84a713849 100644--- a/packages/types/src/connector.ts+++ b/packages/types/src/connector.ts@@ -2,45 +2,42 @@* Connector / Integration types*/+import type { ConnectorType } from "./generated/connector-types";+/*** The `connector_type` values the ingest normalizer resolves.*- * This union is the console's vocabulary, and it is a third name space beside- * the ids `services/connectors` declares and the keys `connectorProfiles` in- * `services/ingest/internal/normalizer/normalizer.go` uses. Ten members used to- * name nothing in either: `ibm_qradar` was not a profile key and no connector+ * This union was hand-written, and hand-maintenance is what let it drift: it+ * was a third name space beside the ids `services/connectors` declares and the+ * keys `connectorProfiles` in+ * `services/ingest/internal/normalizer/normalizer.go` uses, and ten members+ * named nothing in either. `ibm_qradar` was not a profile key and no connector* declared it, so strict mode rejected it and lenient mode minted a vendor* called "ibm_qradar" — a second alert source for the QRadar deployment* `qradar` already fed.*- * Six were the same product under a longer name and now fold onto the declared- * id through `connectorTypeCanonical`. Four named nothing the platform ingests- * and are gone: `vectra_ai` (no Vectra connector exists), `teams` (a ChatOps- * destination, not a source), and `custom_webhook` / `http_pull` / `kafka`- * (transports — the webhook path is the tenant inbox, which keys off a template- * id and never sets `connector_type`).+ * It is now generated from the registry that decides what the platform can+ * ingest. Five alternate spellings (`ibm_qradar`, `google_chronicle`,+ * `palo_alto_cortex`, `slack`, `syslog`) remain members and fold onto the+ * declared id through `connectorTypeCanonical`, which the generated module+ * mirrors as `CONNECTOR_TYPE_CANONICAL`. Five named nothing the platform+ * ingests and are gone: `vectra_ai` (no Vectra connector exists), `teams` (a+ * ChatOps destination, not a source), and `custom_webhook` / `http_pull` /+ * `kafka` (transports — the webhook path is the tenant inbox, which keys off a+ * template id and never sets `connector_type`).*- * `scripts/check_connector_profiles.py` gates every member of this union- * against the normalizer and the connector registry, in both directions.+ * Do not redeclare the union here. `scripts/generate_connector_types.py+ * --check` fails if this file declares its own, because a hand-written union+ * three directories from the generated one is what TypeScript would resolve.+ * `scripts/check_connector_profiles.py` gates every member against the+ * normalizer and the connector registry, in both directions.*/-export type ConnectorType =- | "crowdstrike_falcon"- | "microsoft_sentinel"- | "splunk_enterprise"- | "aws_security_hub"- | "okta_system_log"- | "sentinelone"- | "palo_alto_cortex"- | "google_chronicle"- | "ibm_qradar"- | "darktrace"- | "tenable_io"- | "qualys"- | "jira"- | "servicenow"- | "pagerduty"- | "slack"- | "syslog";+export type { ConnectorType } from "./generated/connector-types";+export {+ CONNECTOR_TYPES,+ CONNECTOR_TYPE_CANONICAL,+ canonicalConnectorType,+} from "./generated/connector-types";export type ConnectorCategory =| "edr"diff --git a/packages/types/src/generated/connector-types.json b/packages/types/src/generated/connector-types.jsonnew file mode 100644index 000000000..adbe337d8--- /dev/null+++ b/packages/types/src/generated/connector-types.json@@ -0,0 +1,213 @@+{+ "canonical": {+ "google_chronicle": "chronicle",+ "ibm_qradar": "qradar",+ "palo_alto_cortex": "cortex_xdr",+ "slack": "slack_audit",+ "syslog": "syslog_cef"+ },+ "counts": {+ "aliases": 2,+ "canonical-fold": 5,+ "profile": 5,+ "registry": 84,+ "total": 94+ },+ "generatedFrom": [+ "services/connectors/app/connectors/__init__.py",+ "services/ingest/internal/normalizer/normalizer.go"+ ],+ "members": [+ "abnormal_security",+ "ai_gateway",+ "ai_guardrail",+ "ai_runtime",+ "auditd",+ "auth0",+ "aws_cloudtrail",+ "aws_guardduty",+ "aws_security_hub",+ "aws_vpc_flow",+ "azure_activity",+ "azure_defender",+ "azure_entra",+ "box",+ "carbon_black",+ "chronicle",+ "cisco_umbrella",+ "cloudflare",+ "cloudflare_zt",+ "confluence_audit",+ "cortex_xdr",+ "cortex_xsiam",+ "crowdstrike",+ "crowdstrike_falcon",+ "darktrace",+ "datadog",+ "datadog_cloud_siem",+ "devo",+ "dropbox",+ "duo_security",+ "elastic",+ "email_inbox",+ "exabeam",+ "falco",+ "fleetdm",+ "gcp_cloud_audit",+ "gcp_scc",+ "github",+ "gitlab",+ "google_chronicle",+ "google_workspace",+ "greynoise",+ "ibm_qradar",+ "imperva",+ "jira",+ "jumpcloud",+ "kubernetes_audit",+ "lacework",+ "llm_usage",+ "m365_audit",+ "microsoft_sentinel",+ "mimecast",+ "netskope",+ "oci",+ "okta",+ "okta_system_log",+ "onepassword",+ "opsgenie",+ "orca",+ "osctrl",+ "pagerduty",+ "palo_alto_cortex",+ "prisma_cloud",+ "proofpoint",+ "qradar",+ "qualys",+ "rapid7_insightidr",+ "salesforce",+ "securonix",+ "sentinelone",+ "servicenow",+ "slack",+ "slack_audit",+ "snowflake",+ "snyk",+ "splunk",+ "splunk_enterprise",+ "sublime_security",+ "sumo_logic",+ "sysdig",+ "syslog",+ "syslog_cef",+ "tailscale",+ "tenable_io",+ "tines",+ "torq",+ "trellix_helix",+ "trend_vision_one",+ "vault",+ "wazuh",+ "windows_event",+ "wiz",+ "zeek_suricata",+ "zscaler"+ ],+ "origins": {+ "abnormal_security": "registry",+ "ai_gateway": "registry",+ "ai_guardrail": "profile",+ "ai_runtime": "profile",+ "auditd": "registry",+ "auth0": "registry",+ "aws_cloudtrail": "registry",+ "aws_guardduty": "registry",+ "aws_security_hub": "registry",+ "aws_vpc_flow": "registry",+ "azure_activity": "registry",+ "azure_defender": "registry",+ "azure_entra": "registry",+ "box": "registry",+ "carbon_black": "registry",+ "chronicle": "registry",+ "cisco_umbrella": "registry",+ "cloudflare": "registry",+ "cloudflare_zt": "registry",+ "confluence_audit": "registry",+ "cortex_xdr": "registry",+ "cortex_xsiam": "registry",+ "crowdstrike": "registry",+ "crowdstrike_falcon": "profile",+ "darktrace": "registry",+ "datadog": "registry",+ "datadog_cloud_siem": "registry",+ "devo": "registry",+ "dropbox": "registry",+ "duo_security": "registry",+ "elastic": "registry",+ "email_inbox": "registry",+ "exabeam": "registry",+ "falco": "registry",+ "fleetdm": "registry",+ "gcp_cloud_audit": "registry",+ "gcp_scc": "registry",+ "github": "registry",+ "gitlab": "registry",+ "google_chronicle": "canonical-fold",+ "google_workspace": "registry",+ "greynoise": "registry",+ "ibm_qradar": "canonical-fold",+ "imperva": "registry",+ "jira": "registry",+ "jumpcloud": "registry",+ "kubernetes_audit": "registry",+ "lacework": "registry",+ "llm_usage": "registry",+ "m365_audit": "registry",+ "microsoft_sentinel": "registry",+ "mimecast": "registry",+ "netskope": "registry",+ "oci": "registry",+ "okta": "registry",+ "okta_system_log": "profile",+ "onepassword": "registry",+ "opsgenie": "registry",+ "orca": "registry",+ "osctrl": "registry",+ "pagerduty": "registry",+ "palo_alto_cortex": "canonical-fold",+ "prisma_cloud": "registry",+ "proofpoint": "registry",+ "qradar": "registry",+ "qualys": "registry",+ "rapid7_insightidr": "registry",+ "salesforce": "registry",+ "securonix": "registry",+ "sentinelone": "registry",+ "servicenow": "registry",+ "slack": "canonical-fold",+ "slack_audit": "registry",+ "snowflake": "registry",+ "snyk": "registry",+ "splunk": "registry",+ "splunk_enterprise": "profile",+ "sublime_security": "registry",+ "sumo_logic": "registry",+ "sysdig": "registry",+ "syslog": "canonical-fold",+ "syslog_cef": "registry",+ "tailscale": "registry",+ "tenable_io": "registry",+ "tines": "registry",+ "torq": "registry",+ "trellix_helix": "registry",+ "trend_vision_one": "registry",+ "vault": "registry",+ "wazuh": "registry",+ "windows_event": "registry",+ "wiz": "registry",+ "zeek_suricata": "registry",+ "zscaler": "registry"+ },+ "regenerateWith": "python3 scripts/generate_connector_types.py"+}diff --git a/packages/types/src/generated/connector-types.ts b/packages/types/src/generated/connector-types.tsnew file mode 100644index 000000000..aff04ec89--- /dev/null+++ b/packages/types/src/generated/connector-types.ts@@ -0,0 +1,134 @@+// AUTO-GENERATED by scripts/generate_connector_types.py. Do not edit by hand.+// Run `python3 scripts/generate_connector_types.py` and commit the result.+//+// Every `connector_type` the ingest normalizer resolves, derived from the+// registry that decides what the platform can ingest:+//+// 84 connector ids declared in services/connectors (_CONNECTOR_CLASSES)+// 5 connectorProfiles keys no connector declares under that spelling+// 5 alternate spellings connectorTypeCanonical folds onto a declared id+// 94 total+//+// The union used to be written by hand, which is how ten members came to name+// nothing: `ibm_qradar` reached no profile and no connector, so strict mode+// rejected those events and lenient mode minted a vendor called "ibm_qradar" —+// a second alert source for the QRadar deployment `qradar` already fed.++/** Every `connector_type` value the ingest normalizer resolves. */+export const CONNECTOR_TYPES = [+ "abnormal_security", // registry+ "ai_gateway", // registry+ "ai_guardrail", // profile+ "ai_runtime", // profile+ "auditd", // registry+ "auth0", // registry+ "aws_cloudtrail", // registry+ "aws_guardduty", // registry+ "aws_security_hub", // registry+ "aws_vpc_flow", // registry+ "azure_activity", // registry+ "azure_defender", // registry+ "azure_entra", // registry+ "box", // registry+ "carbon_black", // registry+ "chronicle", // registry+ "cisco_umbrella", // registry+ "cloudflare", // registry+ "cloudflare_zt", // registry+ "confluence_audit", // registry+ "cortex_xdr", // registry+ "cortex_xsiam", // registry+ "crowdstrike", // registry+ "crowdstrike_falcon", // profile+ "darktrace", // registry+ "datadog", // registry+ "datadog_cloud_siem", // registry+ "devo", // registry+ "dropbox", // registry+ "duo_security", // registry+ "elastic", // registry+ "email_inbox", // registry+ "exabeam", // registry+ "falco", // registry+ "fleetdm", // registry+ "gcp_cloud_audit", // registry+ "gcp_scc", // registry+ "github", // registry+ "gitlab", // registry+ "google_chronicle", // canonical-fold+ "google_workspace", // registry+ "greynoise", // registry+ "ibm_qradar", // canonical-fold+ "imperva", // registry+ "jira", // registry+ "jumpcloud", // registry+ "kubernetes_audit", // registry+ "lacework", // registry+ "llm_usage", // registry+ "m365_audit", // registry+ "microsoft_sentinel", // registry+ "mimecast", // registry+ "netskope", // registry+ "oci", // registry+ "okta", // registry+ "okta_system_log", // profile+ "onepassword", // registry+ "opsgenie", // registry+ "orca", // registry+ "osctrl", // registry+ "pagerduty", // registry+ "palo_alto_cortex", // canonical-fold+ "prisma_cloud", // registry+ "proofpoint", // registry+ "qradar", // registry+ "qualys", // registry+ "rapid7_insightidr", // registry+ "salesforce", // registry+ "securonix", // registry+ "sentinelone", // registry+ "servicenow", // registry+ "slack", // canonical-fold+ "slack_audit", // registry+ "snowflake", // registry+ "snyk", // registry+ "splunk", // registry+ "splunk_enterprise", // profile+ "sublime_security", // registry+ "sumo_logic", // registry+ "sysdig", // registry+ "syslog", // canonical-fold+ "syslog_cef", // registry+ "tailscale", // registry+ "tenable_io", // registry+ "tines", // registry+ "torq", // registry+ "trellix_helix", // registry+ "trend_vision_one", // registry+ "vault", // registry+ "wazuh", // registry+ "windows_event", // registry+ "wiz", // registry+ "zeek_suricata", // registry+ "zscaler", // registry+] as const;++export type ConnectorType = (typeof CONNECTOR_TYPES)[number];++/**+ * Alternate spellings the normalizer folds onto the id services/connectors+ * declares, before anything keys off it. Mirrors `connectorTypeCanonical` in+ * services/ingest/internal/normalizer/normalizer.go so the console can resolve+ * a stored value to one product name instead of two.+ */+export const CONNECTOR_TYPE_CANONICAL: Readonly<Record<string, ConnectorType>> = {+ "google_chronicle": "chronicle",+ "ibm_qradar": "qradar",+ "palo_alto_cortex": "cortex_xdr",+ "slack": "slack_audit",+ "syslog": "syslog_cef",+};++/** Resolve an alternate spelling to the declared connector id. */+export function canonicalConnectorType(connectorType: string): string {+ return CONNECTOR_TYPE_CANONICAL[connectorType] ?? connectorType;+}diff --git a/scripts/check_connector_profiles.py b/scripts/check_connector_profiles.pyindex 4527d54d9..110c22558 100644--- a/scripts/check_connector_profiles.py+++ b/scripts/check_connector_profiles.py@@ -81,20 +81,26 @@ PUSH_ONLY_PROFILES: dict[str, str] = {# checked against those files rather than asserted here.LEGACY_PROFILE_KEYS = {"crowdstrike_falcon", "okta_system_log", "splunk_enterprise"}-# packages/types' ConnectorType union is a third name space, and it predates-# the connectors registry. Ten members named no connector the service declares-# and no profile: the console's older vocabulary (``google_chronicle`` for-# ``chronicle``, ``ibm_qradar`` for ``qradar``) plus four generic transports-# that were never connectors at all.+# packages/types' ConnectorType union used to be a third name space written by+# hand, and it predated the connectors registry. Ten members named no connector+# the service declares and no profile: the console's older vocabulary+# (``google_chronicle`` for ``chronicle``, ``ibm_qradar`` for ``qradar``) plus+# transports that were never connectors at all.#-# All ten are now closed. Six fold onto their declared connector through-# ``connectorTypeCanonical``; four denoted nothing the platform ingests and-# were removed from the union. This stays as an empty ratchet rather than being-# deleted, because the drift it recorded is the kind that comes back: it may-# shrink, never grow, and a new unresolvable union member fails the gate.+# All ten are closed. Five fold onto their declared connector through+# ``connectorTypeCanonical``; five denoted nothing the platform ingests and+# were removed. The union is now generated by+# ``scripts/generate_connector_types.py`` from the registry, the profile keys+# and the fold map, so drift of this shape cannot be reintroduced by hand —+# but the ratchet stays rather than being deleted, because it also catches a+# generator that starts emitting a name nothing resolves. It may shrink, never+# grow, and a new unresolvable union member fails the gate.KNOWN_UNION_ONLY_TYPES: set[str] = set()-TS_TYPES_REL = Path("packages/types/src/connector.ts")+# The generated module, not the hand-written ``connector.ts`` that re-exports+# it. Reading the consumer would mean this gate silently stopped inspecting+# anything the day the union moved.+TS_TYPES_REL = Path("packages/types/src/generated/connector-types.ts")# Identity destinations the generic path depends on. Losing any one of these# is the defect this gate exists for.@@ -271,11 +277,11 @@ def parse_doc_examples(root: Path) -> dict[str, list[str]]:# Gate# --------------------------------------------------------------------------def parse_union_types(path: Path) -> set[str]:- """Members of packages/types' ConnectorType union."""+ """Members of the generated CONNECTOR_TYPES tuple the union is built from."""src = path.read_text(encoding="utf-8")- match = re.search(r"export type ConnectorType\s*=(.*?);", src, re.DOTALL)+ match = re.search(r"export const CONNECTOR_TYPES = \[(.*?)\] as const;", src, re.DOTALL)if not match:- raise GateError(f"could not find the ConnectorType union in {path}")+ raise GateError(f"could not find the CONNECTOR_TYPES tuple in {path}")return set(re.findall(r'"([^"]+)"', match.group(1)))diff --git a/scripts/check_gate_coverage.py b/scripts/check_gate_coverage.pyindex 0468c2bdc..5e9adefd3 100644--- a/scripts/check_gate_coverage.py+++ b/scripts/check_gate_coverage.py@@ -51,9 +51,11 @@ opened is worse than no gate.from __future__ import annotationsimport argparse+import astimport jsonimport reimport sys+from collections import Counterfrom pathlib import Pathimport yaml@@ -65,23 +67,53 @@ MAKEFILE_REL = Path("Makefile")#: Directories whose pytest suites a workflow can name.TEST_ROOTS = ("tests", "services/*/tests", "packages/*/tests", "apps/*/tests")-#: A script is a "check" when its name says it decides something. Generators,-#: exporters and fixtures are out of scope: they have no verdict to report.-_CHECK_PREFIXES = ("check_", "validate_", "audit_", "lint_", "verify_")-_CHECK_SUFFIXES = ("_check.py", "_gates.py", "_conformance.py", "_audit.py")--#: Named individually because their filenames do not announce a verdict, but-#: they exit non-zero on a real finding and are relied on as gates.-_CHECK_EXTRA = {- "connector_conformance.py",- "detection_truth_table.py",- "openapi_diff.py",- "readme_gates.py",- "security_audit.py",-}--#: `sync_vendored_*.py --check` are drift gates; the sync half is a generator.-_CHECK_GLOBS = ("sync_vendored_*.py",)+# --------------------------------------------------------------------------+# What counts as a check+# --------------------------------------------------------------------------+# This used to be a filename test: `check_*`, `validate_*`, `_conformance.py`+# and a hand-kept list of five exceptions for the ones whose names did not+# announce a verdict. That is the same defect the script exists to catch, one+# level up — a gate named something unexpected was simply not inventoried, and+# an uninventoried gate is indistinguishable from one that does not exist.+# Nineteen were in that state, every one of them a CI gate: eleven+# `generate_*`/`export_*`/`build_*` scripts a workflow runs with `--check`,+# `project_stats.py`, `storage_cost_model.py`, `curate_detections.py` and the+# rest. Deleting any of their workflow steps would have left this script+# reporting full coverage.+#+# Classification is now structural: what a script *does*.+#+# verdict-flag it declares a CLI option whose only purpose is to turn+# the run into a verdict.+# findings-exit its exit status is derived from findings it accumulates+# — non-zero when the accumulator is non-empty, or zero+# when it is empty. The polarity matters: `if not specs:+# return 1` is a generator aborting on an empty read, not+# a gate reporting a finding.+# gates-a-workflow a workflow job that runs it publishes an output another+# job branches on. `wet_eval_check.py` is a preflight that+# always exits 0 by design and reports its verdict in a+# JSON status file; the job graph is where that shows.+#+# All three are read from the tree, never from the name. The first two are+# intrinsic, so a gate is inventoried whether or not anything calls it — which+# is what keeps the SCRIPT -> WORKFLOW direction below from being vacuous.++#: Flags that exist only so a caller can act on the result.+_VERDICT_FLAGS = ("--check", "--check-only", "--verify", "--strict", "--self-test")+_VERDICT_PREFIXES = ("--fail-", "--max-", "--require-", "--assert-")++#: A check reports on the repository. Scripts that talk only to a running+#: service (`inject_scenario.py` posts alerts, `generate_runbook.py` queries a+#: tracing backend) exit non-zero when the network call fails, which is an+#: operational error and not a finding about the tree.+_INSPECTS_TREE = re.compile(r"Path\(__file__\)|\.glob\(|\.rglob\(|\.iterdir\(|\.read_text\(|\.read_bytes\(|os\.walk\(")++#: Calls that grow a collection.+_MUTATORS = {"append", "add", "extend", "update"}++#: Annotations that say a name holds a collection of findings.+_COLLECTION_TYPES = {"list", "set", "dict", "tuple", "List", "Set", "Dict", "Counter", "defaultdict"}#: Checks that are deliberately not wired, with the reason. Shrink-only: a#: name here that turns out to be reachable fails the gate, so the list cannot@@ -96,19 +128,231 @@ class GateError(RuntimeError):# --------------------------------------------------------------------------# Inventory# ---------------------------------------------------------------------------def is_check(name: str) -> bool:- if name in _CHECK_EXTRA:- return True- if any(Path(name).match(g) for g in _CHECK_GLOBS):- return True- return name.startswith(_CHECK_PREFIXES) or name.endswith(_CHECK_SUFFIXES)+def _with_parents(tree: ast.Module) -> ast.Module:+ for node in ast.walk(tree):+ for child in ast.iter_child_nodes(node):+ child.parent = node # type: ignore[attr-defined]+ return tree+++def _inside_except(node: ast.AST) -> bool:+ parent = getattr(node, "parent", None)+ while parent is not None:+ if isinstance(parent, ast.ExceptHandler):+ return True+ parent = getattr(parent, "parent", None)+ return False+++def _verdict_flags(tree: ast.Module) -> list[str]:+ found = set()+ for node in ast.walk(tree):+ if isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute) and node.func.attr == "add_argument":+ for arg in node.args:+ if not (isinstance(arg, ast.Constant) and isinstance(arg.value, str) and arg.value.startswith("--")):+ continue+ if arg.value in _VERDICT_FLAGS or arg.value.startswith(_VERDICT_PREFIXES):+ found.add(arg.value)+ return sorted(found)+++def _is_collection_annotation(node: ast.AST | None) -> bool:+ if node is None:+ return False+ base = node.value if isinstance(node, ast.Subscript) else node+ return isinstance(base, ast.Name) and base.id in _COLLECTION_TYPES+++def _accumulators(tree: ast.Module) -> set[str]:+ """Names and attributes the script collects findings into.++ Attributes are included because a report object is the other common shape:+ `security_audit.py` decides on `report.high_critical` and+ `report.unscanned`, which are list fields it appends to.+ """+ assigned: set[str] = set()+ zeroed: set[str] = set()+ found: set[str] = set()+ for node in ast.walk(tree):+ targets: list[ast.AST] = []+ value: ast.AST | None = None+ annotation: ast.AST | None = None+ if isinstance(node, ast.Assign):+ targets, value = list(node.targets), node.value+ elif isinstance(node, ast.AnnAssign):+ targets, value, annotation = [node.target], node.value, node.annotation+ for target in targets:+ key = target.id if isinstance(target, ast.Name) else (target.attr if isinstance(target, ast.Attribute) else None)+ if key is None:+ continue+ if isinstance(target, ast.Name):+ assigned.add(key)+ if isinstance(value, ast.List | ast.Set | ast.Dict | ast.ListComp | ast.SetComp | ast.DictComp | ast.GeneratorExp):+ found.add(key)+ if isinstance(value, ast.Call) and isinstance(value.func, ast.Name) and value.func.id in {"list", "set", "dict", "sorted"}:+ found.add(key)+ if _is_collection_annotation(annotation):+ found.add(key)+ if isinstance(value, ast.Constant) and value.value == 0:+ zeroed.add(key)+ if isinstance(node, ast.FunctionDef | ast.AsyncFunctionDef) and _is_collection_annotation(node.returns):+ found.add(node.name)+ if isinstance(node, ast.AugAssign) and isinstance(node.target, ast.Name) and node.target.id in zeroed:+ found.add(node.target.id) # a counter of findings+ if isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute) and node.func.attr in _MUTATORS:+ container = node.func.value+ if isinstance(container, ast.Name):+ found.add(container.id)+ elif isinstance(container, ast.Attribute):+ found.add(container.attr)+ if isinstance(node, ast.For):+ if isinstance(node.iter, ast.Name) and node.iter.id in assigned:+ found.add(node.iter.id)+ elif isinstance(node.iter, ast.Attribute):+ found.add(node.iter.attr)+ if isinstance(node, ast.Call) and isinstance(node.func, ast.Name) and node.func.id == "len":+ for arg in node.args:+ if isinstance(arg, ast.Name) and arg.id in assigned:+ found.add(arg.id)+ elif isinstance(arg, ast.Attribute):+ found.add(arg.attr)+ return found+++def _referenced(node: ast.AST) -> set[str]:+ out: set[str] = set()+ for sub in ast.walk(node):+ if isinstance(sub, ast.Name):+ out.add(sub.id)+ elif isinstance(sub, ast.Attribute):+ out.add(sub.attr)+ return out+++def _negated(test: ast.AST, names: set[str]) -> bool:+ """Whether every accumulator in `test` appears under a `not`."""+ positive = set()+ for node in ast.walk(test):+ if isinstance(node, ast.UnaryOp) and isinstance(node.op, ast.Not):+ continue+ if isinstance(node, ast.Name) and node.id in names:+ positive.add(node.id)+ elif isinstance(node, ast.Attribute) and node.attr in names:+ positive.add(node.attr)+ negated = set()+ for node in ast.walk(test):+ if isinstance(node, ast.UnaryOp) and isinstance(node.op, ast.Not):+ negated |= _referenced(node.operand) & names+ return bool(negated) and not (positive - negated)+++def _exit_status(node: ast.AST) -> ast.AST | None:+ if isinstance(node, ast.Return):+ return node.value+ if isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute) and node.func.attr == "exit" and node.args:+ return node.args[0]+ if isinstance(node, ast.Raise) and isinstance(node.exc, ast.Call) and isinstance(node.exc.func, ast.Name):+ if node.exc.func.id == "SystemExit" and node.exc.args:+ return node.exc.args[0]+ return None+++def _is_int(node: ast.AST | None, *, zero: bool) -> bool:+ if not (isinstance(node, ast.Constant) and isinstance(node.value, int) and not isinstance(node.value, bool)):+ return False+ return (node.value == 0) if zero else (node.value != 0)+++def _exit_path(tree: ast.Module) -> list[ast.AST]:+ """Functions whose return value becomes the process exit status.++ Bounded transitive closure from whatever is passed to `sys.exit(...)`,+ because dispatchers are common: `security_audit.py`'s `main` returns+ `handlers[args.command](args)` and the verdict is three frames down. A+ module with no exit path at all is a library, not a check.+ """+ funcs = {n.name: n for n in ast.walk(tree) if isinstance(n, ast.FunctionDef | ast.AsyncFunctionDef)}+ entries: set[str] = set()+ for node in ast.walk(tree):+ args = None+ if isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute) and node.func.attr == "exit":+ args = node.args+ elif isinstance(node, ast.Raise) and isinstance(node.exc, ast.Call) and isinstance(node.exc.func, ast.Name):+ args = node.exc.args if node.exc.func.id == "SystemExit" else None+ for arg in args or []:+ entries |= {n.id for n in ast.walk(arg) if isinstance(n, ast.Name)} & set(funcs)+ if not entries and "main" in funcs:+ entries = {"main"}++ seen: set[str] = set()+ frontier, depth = entries, 0+ while frontier and depth < 4:+ seen |= frontier+ nxt: set[str] = set()+ for name in frontier:+ nxt |= ({n.id for n in ast.walk(funcs[name]) if isinstance(n, ast.Name)} & set(funcs)) - seen+ frontier, depth = nxt, depth + 1+ return [funcs[name] for name in sorted(seen)]+++def _findings_exit(tree: ast.Module) -> str | None:+ """The guard that turns accumulated findings into an exit status."""+ accumulators = _accumulators(tree)+ if not accumulators:+ return None+ for function in _exit_path(tree):+ for node in ast.walk(function):+ if isinstance(node, ast.IfExp) and (_is_int(node.body, zero=False) or _is_int(node.orelse, zero=False)):+ if _referenced(node.test) & accumulators:+ return ast.unparse(node)[:72]+ continue+ status = _exit_status(node)+ if status is None or _inside_except(node):+ continue+ nonzero = _is_int(status, zero=False)+ if not nonzero and not _is_int(status, zero=True):+ continue+ parent = getattr(node, "parent", None)+ while parent is not None:+ if isinstance(parent, ast.If) and (_referenced(parent.test) & accumulators):+ # Findings present -> fail. The inverse (`if not specs:+ # return 1`) is an abort on an empty read: the script+ # could not do its job, which is not a verdict on the tree.+ if nonzero != _negated(parent.test, accumulators):+ return ast.unparse(parent.test)[:72]+ break+ parent = getattr(parent, "parent", None)+ return None+++def classify_source(source: str) -> dict[str, str]:+ """Structural signals that make a script a check. Empty means it is not."""+ try:+ tree = _with_parents(ast.parse(source))+ except SyntaxError:+ return {}+ if not _INSPECTS_TREE.search(source):+ return {}+ signals: dict[str, str] = {}+ if flags := _verdict_flags(tree):+ signals["verdict-flag"] = " ".join(flags)+ if guard := _findings_exit(tree):+ signals["findings-exit"] = guard+ return signals-def collect_checks(root: Path) -> list[str]:+def collect_checks(root: Path, gating: dict[str, str] | None = None) -> dict[str, dict[str, str]]:+ """script name -> the structural signals that classify it as a check."""scripts = root / SCRIPTS_RELif not scripts.is_dir():raise GateError(f"no scripts directory at {scripts}")- found = sorted(p.name for p in scripts.glob("*.py") if is_check(p.name))+ found: dict[str, dict[str, str]] = {}+ for path in sorted(scripts.glob("*.py")):+ signals = classify_source(path.read_text(encoding="utf-8", errors="replace"))+ if gating and path.name in gating:+ signals["gates-a-workflow"] = gating[path.name]+ if signals:+ found[path.name] = signalsif not found:raise GateError(f"parsed zero check scripts from {scripts} — refusing to report a clean tree from an empty read")return found@@ -161,12 +405,38 @@ def _command_text(step: dict) -> str:return "\n".join(parts)+_NEEDS_OUTPUT = re.compile(r"needs\.([A-Za-z0-9_-]+)\.outputs")+_SCRIPT_IN_JOB = re.compile(r"scripts/([\w.-]+\.py)")+++def _gating_jobs(doc: dict, workflow: str) -> dict[str, str]:+ """script -> "workflow:job" for jobs whose output another job branches on.++ A workflow that merely runs a script is not evidence the script is a gate.+ A workflow that publishes a job output and makes a downstream job+ conditional on it has, structurally, asked the script for a verdict.+ """+ jobs = doc.get("jobs") or {}+ branched: set[str] = set()+ for job in jobs.values():+ if isinstance(job, dict):+ branched.update(_NEEDS_OUTPUT.findall(yaml.safe_dump(job, default_flow_style=False)))+ out: dict[str, str] = {}+ for job_id, job in jobs.items():+ if job_id not in branched or not isinstance(job, dict) or not job.get("outputs"):+ continue+ for name in _SCRIPT_IN_JOB.findall(yaml.safe_dump(job, default_flow_style=False)):+ out.setdefault(name, f"{workflow}:{job_id}")+ return out++class Surface:"""The text a workflow executes, with working directories resolved."""def __init__(self, root: Path):self.root = rootself.by_workflow: dict[str, list[tuple[str, str]]] = {}+ self.gating: dict[str, str] = {}wf_dir = root / WORKFLOWS_RELif not wf_dir.is_dir():raise GateError(f"no workflows directory at {wf_dir}")@@ -186,11 +456,28 @@ class Surface:for concrete in _expand_matrix(text, matrix):rows.append((concrete_wd.strip().strip("./"), concrete))self.by_workflow[path.name] = rows+ for name, origin in _gating_jobs(doc, path.name).items():+ self.gating.setdefault(name, origin)self.workflow_count = len(files)def texts(self) -> list[tuple[str, str, str]]:return [(wf, wd, text) for wf, rows in self.by_workflow.items() for wd, text in rows]+ def invoked_with_verdict_flag(self) -> dict[str, str]:+ """script -> the workflow that runs it with a verdict flag.++ The other direction on the classifier itself. If a workflow asks a+ script for a verdict and the classifier does not call it a check, the+ classifier has a blind spot — which is the defect this file is about.+ """+ flags = "|".join(re.escape(f) for f in (*_VERDICT_FLAGS, *_VERDICT_PREFIXES))+ pattern = re.compile(rf"scripts/([\w.-]+\.py)[^\n;&|]*?\s(?:{flags})")+ out: dict[str, str] = {}+ for workflow, _wd, text in self.texts():+ for name in pattern.findall(text):+ out.setdefault(name, workflow)+ return out+# --------------------------------------------------------------------------# Reachability@@ -339,10 +626,11 @@ def missing_script_paths(root: Path, surface: Surface) -> list[tuple[str, str]]:def evaluate(- checks: list[str],+ checks: dict[str, dict[str, str]],routes: dict[str, list[str]],dangling: list[tuple[str, str]],ratchet: dict[str, str],+ verdict_invocations: dict[str, str] | None = None,) -> list[tuple[str, str]]:failures: list[tuple[str, str]] = []@@ -352,9 +640,9 @@ def evaluate(failures.append(("check-unreachable",- f"{name} is a check and no workflow reaches it, directly or through make, "- "another script or a test suite CI runs. A gate that cannot fail is "- "indistinguishable from no gate, while its presence implies coverage.",+ f"{name} is a check ({', '.join(sorted(checks[name]))}) and no workflow reaches it, "+ "directly or through make, another script or a test suite CI runs. A gate that cannot "+ "fail is indistinguishable from no gate, while its presence implies coverage.",))if reached and name in ratchet:@@ -371,18 +659,33 @@ def evaluate(for wf, rel in dangling:failures.append(("workflow-path-missing", f"{wf} names {rel}, which does not exist — the step cannot do what it says"))+ # CLASSIFIER -> WORKFLOW. The classifier read in the other direction: a+ # script CI runs with `--check` is being asked for a verdict, so anything+ # the classifier leaves out is a blind spot in the classifier, not a+ # script that stopped being a gate.+ for name, workflow in sorted((verdict_invocations or {}).items()):+ if name not in checks:+ failures.append(+ (+ "classifier-blind-spot",+ f"{workflow} runs {name} with a verdict flag, but no structural signal classifies it as a check — "+ "it would not be inventoried, and losing its workflow step would go unnoticed",+ )+ )+return failuresdef load(root: Path) -> dict:surface = Surface(root)- checks = collect_checks(root)+ checks = collect_checks(root, surface.gating)routes, _, corpus = resolve(root, surface)return {"checks": checks,"routes": routes,"dangling": missing_script_paths(root, surface),"corpus": corpus,+ "verdict_invocations": surface.invoked_with_verdict_flag(),}@@ -405,7 +708,7 @@ def main(argv: list[str] | None = None) -> int:return 2checks, routes, corpus = data["checks"], data["routes"], data["corpus"]- failures = evaluate(checks, routes, data["dangling"], KNOWN_UNREACHED)+ failures = evaluate(checks, routes, data["dangling"], KNOWN_UNREACHED, data["verdict_invocations"])if args.json:print(@@ -413,9 +716,10 @@ def main(argv: list[str] | None = None) -> int:{"repo_root": str(root),"corpus": corpus,- "checks": {name: routes.get(name, []) for name in checks},+ "checks": {name: {"signals": checks[name], "routes": routes.get(name, [])} for name in checks},"dangling_workflow_paths": [{"workflow": w, "path": p} for w, p in data["dangling"]],"known_unreached": KNOWN_UNREACHED,+ "verdict_invocations": data["verdict_invocations"],"failures": [{"code": c, "detail": d} for c, d in failures],},indent=2,@@ -424,11 +728,15 @@ def main(argv: list[str] | None = None) -> int:)return 1 if failures else 0+ by_signal = Counter(signal for signals in checks.values() for signal in signals)+ intrinsic = sum(1 for s in checks.values() if set(s) - {"gates-a-workflow"})print(f"repo root {root}")print(f"workflows {WORKFLOWS_REL} ({corpus['workflows']} files)")print(f"scripts {SCRIPTS_REL} ({corpus['scripts']} python files, {len(checks)} of them checks)")print(f"make recipes {MAKEFILE_REL} ({corpus['make_recipes']} targets)")print(f"reachable nodes {corpus['reachable_nodes']} (workflows + make recipes + CI-collected tests + scripts they reach)")+ print("classified by " + ", ".join(f"{signal} {count}" for signal, count in sorted(by_signal.items())))+ print(f" {intrinsic} of {len(checks)} from an intrinsic signal, so they are inventoried with or without a caller")print(f"ratchet {len(KNOWN_UNREACHED)} check(s) deliberately unwired")print()@@ -437,7 +745,8 @@ def main(argv: list[str] | None = None) -> int:for name in checks:hits = routes.get(name, [])print(f" {name:{width}} {hits[0] if hits else '— NO WORKFLOW —'}")- for extra in hits[1:4]:+ print(f" {'':{width}} signals: {', '.join(sorted(checks[name]))}")+ for extra in hits[1:3]:print(f" {'':{width}} {extra}")print()@@ -462,24 +771,25 @@ def self_test(root: Path) -> int:print(f"self-test: cannot read the tree: {exc}", file=sys.stderr)return 2- clean = evaluate(base["checks"], base["routes"], base["dangling"], KNOWN_UNREACHED)+ clean = evaluate(base["checks"], base["routes"], base["dangling"], KNOWN_UNREACHED, base["verdict_invocations"])if clean:print("self-test: the unmodified tree already fails; fix that first", file=sys.stderr)for code, detail in clean:print(f" [{code}] {detail}", file=sys.stderr)return 1- def case(checks=None, routes=None, dangling=None, ratchet=None):+ def case(checks=None, routes=None, dangling=None, ratchet=None, invocations=None):return (checks if checks is not None else base["checks"],routes if routes is not None else base["routes"],dangling if dangling is not None else base["dangling"],ratchet if ratchet is not None else KNOWN_UNREACHED,+ invocations if invocations is not None else base["verdict_invocations"],)- orphan_checks = [*base["checks"], "check_brand_new_thing.py"]- orphan_routes = {**base["routes"], "check_brand_new_thing.py": []}-+ orphan = "check_brand_new_thing.py"+ orphan_checks = {**base["checks"], orphan: {"verdict-flag": "--check"}}+ orphan_routes = {**base["routes"], orphan: []}reached_name = next(n for n in base["checks"] if base["routes"].get(n))cases: list[tuple[str, str, tuple]] = [@@ -508,6 +818,11 @@ def self_test(root: Path) -> int:"ratchet-names-nothing",case(ratchet={"check_ghost.py": "example"}),),+ (+ "CLASSIFIER -> WORKFLOW: CI asks a script for a verdict the classifier does not inventory",+ "classifier-blind-spot",+ case(invocations={**base["verdict_invocations"], "some_unclassified_thing.py": "ci.yml"}),+ ),]print(f"self-test against {root}")@@ -520,6 +835,53 @@ def self_test(root: Path) -> int:print(f" {'PASS' if caught else 'FAIL'} {description}")print(f" expected [{expected}] got {sorted(codes) or 'nothing'}")+ # The classifier itself. The defect this replaced was a filename test, so+ # the case that matters is a gate whose name announces nothing: a+ # structural classifier must find it, and the old one could not.+ planted = '''\+"""A gate that no naming convention would reveal."""+import sys+from pathlib import Path++ROOT = Path(__file__).resolve().parent.parent++def main() -> int:+ offenders = [p.name for p in ROOT.glob("*.md") if "\\t" in p.read_text(errors="replace")]+ if offenders:+ print("tabs in markdown:", offenders)+ return 1+ return 0++sys.exit(main())+'''+ signals = classify_source(planted)+ print(f" {'PASS' if signals else 'FAIL'} CLASSIFIER: a gate planted under an unconventional name is still found")+ print(f" `frobnicate_widgets.py` matches no check_/validate_/audit_ prefix; signals {sorted(signals) or 'none'}")+ ok &= bool(signals)++ # And the inverse, because a classifier that said yes to everything would+ # pass the case above: a generator that aborts on an empty read is not a+ # gate, and the polarity of its guard is the only thing that says so.+ generator = '''\+"""Writes a document. Not a gate."""+import sys+from pathlib import Path++def main() -> int:+ specs = [p for p in Path("specs").glob("*.yaml")]+ if not specs:+ print("nothing to render")+ return 1+ Path("out.md").write_text("\\n".join(p.name for p in specs))+ return 0++sys.exit(main())+'''+ rejected = not classify_source(generator)+ print(f" {'PASS' if rejected else 'FAIL'} CLASSIFIER: a generator that aborts on an empty read is not a check")+ print(f" `if not specs: return 1` is an abort, not a finding; signals {sorted(classify_source(generator)) or 'none'}")+ ok &= rejected+# The reachability resolver itself, not just the rules on top of it: a# resolver that returned "reached" for everything would pass every case# above. This asserts it discriminates on the real tree.diff --git a/scripts/competitor_names.toml b/scripts/competitor_names.tomlindex ead4a2bf6..cb1dea6d7 100644--- a/scripts/competitor_names.toml+++ b/scripts/competitor_names.toml@@ -130,6 +130,11 @@ paths = ["marketplace/index.json", "apps/web/public/marketplace/index.json"]names = ["Torq", "Tines", "Wazuh"]reason = "Generated marketplace catalog; entries are the connector manifests above."+[[allow]]+paths = ["packages/types/src/generated/connector-types.ts", "packages/types/src/generated/connector-types.json"]+names = ["Torq", "Tines", "Wazuh"]+reason = "The console's connector_type vocabulary, generated from the connector registry by scripts/generate_connector_types.py. Each name here is a connector id the platform ingests under; removing one would make the type reject events from a connector that ships."+[[allow]]paths = ["apps/docs/sidebars.ts"]names = ["Torq", "Tines", "Wazuh"]diff --git a/scripts/generate_connector_types.py b/scripts/generate_connector_types.pynew file mode 100644index 000000000..370cc42da--- /dev/null+++ b/scripts/generate_connector_types.py@@ -0,0 +1,541 @@+#!/usr/bin/env python3+"""Generate the console's ``ConnectorType`` union from the connector registry.++Why this exists+---------------+``packages/types/src/connector.ts`` carried a hand-written union of+``connector_type`` values. Hand-maintenance is what let it drift: ten members+named nothing the platform could ingest, and ``ibm_qradar`` was the expensive+one — no connector declared it and no profile was keyed on it, so strict mode+rejected those events and lenient mode minted a vendor called "ibm_qradar", a+second alert source for the same QRadar deployment ``qradar`` already fed.++PR #811 corrected the members. It did not remove the mechanism, so this does:+the union is now derived from the registry that decides what the platform can+ingest, and ``--check`` fails when the committed file no longer matches.++What the union means+--------------------+"Every ``connector_type`` the ingest normalizer resolves", which is exactly+three sets, each read from its own source of truth:++ registry ``_CONNECTOR_CLASSES`` in+ ``services/connectors/app/connectors/__init__.py``, resolved+ to ``cls.connector_id`` the same way ``_build_registry()``+ does — through the class, never through the filename.+ profile ``connectorProfiles`` keys in+ ``services/ingest/internal/normalizer/normalizer.go``. Some+ are longer legacy spellings a declared id reaches through+ ``connectorTypeAliases`` (``crowdstrike`` ->+ ``crowdstrike_falcon``); ``splunk_enterprise`` is reachable+ only because the union carries it, which is why dropping it+ would fail ``check_connector_profiles.py``.+ canonical-fold ``connectorTypeCanonical`` sources — alternate spellings the+ normalizer folds onto a declared id before anything keys off+ it. Generation must preserve these, not flatten them away:+ the console still emits ``ibm_qradar`` and the fold is what+ makes it arrive as ``qradar``.++That set is, by construction, the ``resolvable`` set+``scripts/check_connector_profiles.py`` computes, so the two gates now agree+on one definition instead of two.++Nothing here is positional+--------------------------+``scripts/generate_detections.py`` was not idempotent because it assigned+``det-{category}-{idx:03d}`` by position, so inserting anywhere but the end+moved every later id; it needed a committed lock file. This generator assigns+no identity at all. A member *is* its ``connector_id``, taken from the class,+and the output is a sorted set — so reordering the registry cannot change a+byte of the output and no lock file is required. ``tests/`` proves both+properties rather than asserting them here.++Identity comes from the class, not the filename+-----------------------------------------------+``scripts/generate_connector_docs.py`` derived a connector's identity from its+filename and produced thin duplicate pages for the six connectors whose files+are spelled with underscores, while the coverage gate reported 100%. Two+connectors in this tree would break the same way: ``jira_connector.py``+declares ``jira`` and ``tenable.py`` declares ``tenable_io``. This generator+reads ``connector_id`` off the class, which is what ``CONNECTOR_REGISTRY`` is+keyed by, and refuses a class with an empty or duplicate id exactly as+``_build_registry()`` does.++Usage+-----+ python3 scripts/generate_connector_types.py # write outputs+ python3 scripts/generate_connector_types.py --check # fail on drift+ python3 scripts/generate_connector_types.py --json+ python3 scripts/generate_connector_types.py --self-test++``--repo-root`` overrides the tree under inspection. The resolved root, every+file read and every count are printed before the verdict, and an empty parse+is a hard error rather than a quiet zero: a generator that reports a clean+tree it never opened is worse than no generator.+"""++from __future__ import annotations++import argparse+import ast+import json+import re+import sys+from pathlib import Path++REGISTRY_REL = Path("services/connectors/app/connectors/__init__.py")+CONNECTORS_REL = Path("services/connectors/app/connectors")+NORMALIZER_REL = Path("services/ingest/internal/normalizer/normalizer.go")+TS_OUT_REL = Path("packages/types/src/generated/connector-types.ts")+JSON_OUT_REL = Path("packages/types/src/generated/connector-types.json")+CONSUMER_REL = Path("packages/types/src/connector.ts")++#: Where each union member comes from, in the order a member is attributed to+#: the first source that claims it.+ORIGIN_REGISTRY = "registry"+ORIGIN_PROFILE = "profile"+ORIGIN_FOLD = "canonical-fold"+++class GateError(RuntimeError):+ """An input could not be read. Never downgraded to a passing result."""+++# --------------------------------------------------------------------------+# Registry (Python, via AST — no runtime import of the connectors package)+# --------------------------------------------------------------------------+def _registered_class_names(tree: ast.Module) -> list[str]:+ for node in tree.body:+ value = None+ if isinstance(node, ast.AnnAssign) and getattr(node.target, "id", None) == "_CONNECTOR_CLASSES":+ value = node.value+ elif isinstance(node, ast.Assign) and any(getattr(t, "id", None) == "_CONNECTOR_CLASSES" for t in node.targets):+ value = node.value+ if isinstance(value, ast.Tuple):+ names = [e.id for e in value.elts if isinstance(e, ast.Name)]+ if len(names) != len(value.elts):+ raise GateError("_CONNECTOR_CLASSES contains an entry that is not a bare class name")+ return names+ raise GateError(f"could not find the _CONNECTOR_CLASSES tuple in {REGISTRY_REL}")+++def _import_modules(tree: ast.Module) -> dict[str, str]:+ """Imported class name -> the module in app.connectors it comes from."""+ out: dict[str, str] = {}+ for node in ast.walk(tree):+ if isinstance(node, ast.ImportFrom) and node.module and node.module.startswith("app.connectors."):+ for alias in node.names:+ out[alias.asname or alias.name] = node.module.split(".")[-1]+ return out+++def _class_attribute(tree: ast.Module, class_name: str, attribute: str) -> str | None:+ for cls in (n for n in ast.walk(tree) if isinstance(n, ast.ClassDef) and n.name == class_name):+ for stmt in cls.body:+ target = None+ if isinstance(stmt, ast.Assign) and len(stmt.targets) == 1:+ target = getattr(stmt.targets[0], "id", None)+ elif isinstance(stmt, ast.AnnAssign):+ target = getattr(stmt.target, "id", None)+ if target == attribute and isinstance(stmt.value, ast.Constant) and isinstance(stmt.value.value, str):+ return stmt.value.value+ return None+++def parse_registry(root: Path) -> dict[str, dict[str, str]]:+ """connector_id -> {class, module, name}, resolved as _build_registry() does."""+ registry_file = root / REGISTRY_REL+ connectors_dir = root / CONNECTORS_REL+ if not registry_file.exists():+ raise GateError(f"expected input does not exist: {registry_file}")++ init_tree = ast.parse(registry_file.read_text(encoding="utf-8"))+ modules = _import_modules(init_tree)+ out: dict[str, dict[str, str]] = {}+ for class_name in _registered_class_names(init_tree):+ module = modules.get(class_name)+ if module is None:+ raise GateError(f"{class_name} is in _CONNECTOR_CLASSES but is not imported from app.connectors.*")+ path = connectors_dir / f"{module}.py"+ if not path.exists():+ raise GateError(f"{class_name} is imported from app.connectors.{module}, which does not exist")+ tree = ast.parse(path.read_text(encoding="utf-8"))+ connector_id = _class_attribute(tree, class_name, "connector_id")+ # The same two refusals _build_registry() makes, for the same reasons.+ if not connector_id:+ raise GateError(f"connector class {class_name} has an empty connector_id; refusing to register")+ if connector_id in out:+ raise GateError(f"duplicate connector_id {connector_id!r} between {out[connector_id]['class']} and {class_name}")+ out[connector_id] = {+ "class": class_name,+ "module": f"{module}.py",+ "name": _class_attribute(tree, class_name, "connector_name") or "",+ }+ if not out:+ raise GateError("parsed zero connector ids — refusing to generate from an empty read")+ return out+++# --------------------------------------------------------------------------+# Normalizer (Go, via brace-balanced literals)+# --------------------------------------------------------------------------+def _go_block(src: str, header: str) -> str:+ start = src.find(header)+ if start == -1:+ raise GateError(f"could not find `{header}` in {NORMALIZER_REL}")+ depth, i = 0, src.index("{", start)+ for j in range(i, len(src)):+ if src[j] == "{":+ depth += 1+ elif src[j] == "}":+ depth -= 1+ if depth == 0:+ return src[i : j + 1]+ raise GateError(f"unbalanced braces after `{header}` in {NORMALIZER_REL}")+++def parse_normalizer(root: Path) -> tuple[set[str], dict[str, str], dict[str, str]]:+ """(profile keys, connectorTypeAliases, connectorTypeCanonical)."""+ path = root / NORMALIZER_REL+ if not path.exists():+ raise GateError(f"expected input does not exist: {path}")+ src = path.read_text(encoding="utf-8")++ profiles = set(re.findall(r'^\t"([^"]+)":\s*\{', _go_block(src, "var connectorProfiles = map[string]connectorProfile{"), re.MULTILINE))+ if not profiles:+ raise GateError("parsed zero connectorProfiles keys — refusing to generate from an empty read")++ def pairs(header: str) -> dict[str, str]:+ if header not in src:+ return {}+ return dict(re.findall(r'"([^"]+)":\s*"([^"]+)"', _go_block(src, header)))++ aliases = pairs("var connectorTypeAliases = map[string]string{")+ canonical = pairs("var connectorTypeCanonical = map[string]string{")+ return profiles, aliases, canonical+++# --------------------------------------------------------------------------+# Payload+# --------------------------------------------------------------------------+def build_payload(registry: dict[str, dict[str, str]], profiles: set[str], aliases: dict[str, str], canonical: dict[str, str]) -> dict:+ """The union and its provenance. A pure function of the three inputs."""+ origins: dict[str, str] = {}+ for connector_id in registry:+ origins[connector_id] = ORIGIN_REGISTRY+ for key in sorted(profiles):+ origins.setdefault(key, ORIGIN_PROFILE)+ for alternate in sorted(canonical):+ origins.setdefault(alternate, ORIGIN_FOLD)++ members = sorted(origins)+ return {+ "members": members,+ "origins": {m: origins[m] for m in members},+ "canonical": dict(sorted(canonical.items())),+ "counts": {+ "total": len(members),+ ORIGIN_REGISTRY: sum(1 for m in members if origins[m] == ORIGIN_REGISTRY),+ ORIGIN_PROFILE: sum(1 for m in members if origins[m] == ORIGIN_PROFILE),+ ORIGIN_FOLD: sum(1 for m in members if origins[m] == ORIGIN_FOLD),+ "aliases": len(aliases),+ },+ "generatedFrom": [str(REGISTRY_REL), str(NORMALIZER_REL)],+ "regenerateWith": "python3 scripts/generate_connector_types.py",+ }+++def render_json(payload: dict) -> str:+ return json.dumps(payload, indent=2, sort_keys=True) + "\n"+++def render_typescript(payload: dict) -> str:+ members = payload["members"]+ origins = payload["origins"]+ canonical = payload["canonical"]+ counts = payload["counts"]++ lines: list[str] = [+ "// AUTO-GENERATED by scripts/generate_connector_types.py. Do not edit by hand.",+ "// Run `python3 scripts/generate_connector_types.py` and commit the result.",+ "//",+ "// Every `connector_type` the ingest normalizer resolves, derived from the",+ "// registry that decides what the platform can ingest:",+ "//",+ f"// {counts[ORIGIN_REGISTRY]:>3} connector ids declared in services/connectors (_CONNECTOR_CLASSES)",+ f"// {counts[ORIGIN_PROFILE]:>3} connectorProfiles keys no connector declares under that spelling",+ f"// {counts[ORIGIN_FOLD]:>3} alternate spellings connectorTypeCanonical folds onto a declared id",+ f"// {counts['total']:>3} total",+ "//",+ "// The union used to be written by hand, which is how ten members came to name",+ "// nothing: `ibm_qradar` reached no profile and no connector, so strict mode",+ '// rejected those events and lenient mode minted a vendor called "ibm_qradar" —',+ "// a second alert source for the QRadar deployment `qradar` already fed.",+ "",+ "/** Every `connector_type` value the ingest normalizer resolves. */",+ "export const CONNECTOR_TYPES = [",+ ]+ width = max(len(m) for m in members) + 2+ for member in members:+ literal = f'"{member}",'+ lines.append(f" {literal:<{width + 1}} // {origins[member]}")+ lines += [+ "] as const;",+ "",+ "export type ConnectorType = (typeof CONNECTOR_TYPES)[number];",+ "",+ "/**",+ " * Alternate spellings the normalizer folds onto the id services/connectors",+ " * declares, before anything keys off it. Mirrors `connectorTypeCanonical` in",+ " * services/ingest/internal/normalizer/normalizer.go so the console can resolve",+ " * a stored value to one product name instead of two.",+ " */",+ "export const CONNECTOR_TYPE_CANONICAL: Readonly<Record<string, ConnectorType>> = {",+ ]+ for alternate, target in canonical.items():+ lines.append(f' "{alternate}": "{target}",')+ lines += [+ "};",+ "",+ "/** Resolve an alternate spelling to the declared connector id. */",+ "export function canonicalConnectorType(connectorType: string): string {",+ " return CONNECTOR_TYPE_CANONICAL[connectorType] ?? connectorType;",+ "}",+ "",+ ]+ return "\n".join(lines)+++# --------------------------------------------------------------------------+# Gate+# --------------------------------------------------------------------------+#: The consumer must re-export the generated union rather than redeclare it.+#: Without this rule the generated file can sit in the tree, current and+#: ignored, while a hand-written union three directories away is what+#: TypeScript actually resolves — the drift this change exists to end.+_REDECLARATION = re.compile(r"^\s*export\s+type\s+ConnectorType\s*=", re.MULTILINE)+_REEXPORT = re.compile(r"""export\s+type\s*\{[^}]*\bConnectorType\b[^}]*\}\s*from\s*["'][^"']*generated/connector-types["']""")+++def evaluate(payload: dict, on_disk_ts: str | None, on_disk_json: str | None, consumer: str | None) -> list[tuple[str, str]]:+ failures: list[tuple[str, str]] = []++ if on_disk_ts is None:+ failures.append(("output-missing", f"{TS_OUT_REL} does not exist; run the generator"))+ elif on_disk_ts != render_typescript(payload):+ failures.append(("output-drifted", f"{TS_OUT_REL} is not what generation produces; run the generator and commit the result"))++ if on_disk_json is None:+ failures.append(("output-missing", f"{JSON_OUT_REL} does not exist; run the generator"))+ elif on_disk_json != render_json(payload):+ failures.append(("output-drifted", f"{JSON_OUT_REL} is not what generation produces; run the generator and commit the result"))++ if consumer is None:+ failures.append(("consumer-missing", f"{CONSUMER_REL} does not exist"))+ else:+ if _REDECLARATION.search(consumer):+ failures.append(+ (+ "consumer-redeclares",+ f"{CONSUMER_REL} declares its own ConnectorType union; the generated one would be ignored "+ "and the hand-maintained list is the mechanism that drifted",+ )+ )+ if not _REEXPORT.search(consumer):+ failures.append(("consumer-not-reexporting", f"{CONSUMER_REL} does not re-export ConnectorType from {TS_OUT_REL.stem}"))++ # The folds are the part generation is most likely to flatten away, so+ # they are checked against the rendered output rather than the payload.+ rendered = on_disk_ts or ""+ for alternate, target in payload["canonical"].items():+ if alternate not in payload["members"]:+ failures.append(+ (+ "fold-source-dropped",+ f"{alternate!r} folds onto {target!r} but is not a union member; the console's spelling would not type-check",+ )+ )+ if target not in payload["members"]:+ failures.append(("fold-target-dropped", f"{alternate!r} folds onto {target!r}, which is not a union member"))+ if on_disk_ts is not None and f'"{alternate}": "{target}"' not in rendered:+ failures.append(("fold-not-emitted", f"the {alternate!r} -> {target!r} fold is missing from {TS_OUT_REL}"))+ return failures+++def load(root: Path) -> dict:+ registry = parse_registry(root)+ profiles, aliases, canonical = parse_normalizer(root)+ return {"registry": registry, "profiles": profiles, "aliases": aliases, "canonical": canonical}+++def _read(path: Path) -> str | None:+ return path.read_text(encoding="utf-8") if path.exists() else None+++def main(argv: list[str] | None = None) -> int:+ parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)+ parser.add_argument("--repo-root", type=Path, default=Path(__file__).resolve().parent.parent)+ parser.add_argument("--check", action="store_true", help="fail (exit 1) on drift instead of rewriting the outputs")+ parser.add_argument("--json", action="store_true")+ parser.add_argument("--self-test", action="store_true", help="prove the gate detects injected drift in each direction")+ args = parser.parse_args(argv)++ root = args.repo_root.resolve()+ if args.self_test:+ return self_test(root)++ try:+ inputs = load(root)+ except GateError as exc:+ print(f"generate_connector_types: FAILED to read the tree: {exc}", file=sys.stderr)+ return 2++ payload = build_payload(**inputs)+ ts_path, json_path = root / TS_OUT_REL, root / JSON_OUT_REL++ if not args.check:+ ts_path.parent.mkdir(parents=True, exist_ok=True)+ ts_path.write_text(render_typescript(payload), encoding="utf-8")+ json_path.write_text(render_json(payload), encoding="utf-8")++ failures = evaluate(payload, _read(ts_path), _read(json_path), _read(root / CONSUMER_REL))++ if args.json:+ print(+ json.dumps(+ {"repo_root": str(root), **payload, "failures": [{"code": c, "detail": d} for c, d in failures]}, indent=2, sort_keys=True+ )+ )+ return 1 if failures else 0++ counts = payload["counts"]+ print(f"repo root {root}")+ print(f"registry {REGISTRY_REL} ({len(inputs['registry'])} declared connector ids)")+ print(+ f"normalizer {NORMALIZER_REL} ({len(inputs['profiles'])} profiles, "+ f"{counts['aliases']} type aliases, {len(inputs['canonical'])} canonical folds)"+ )+ print(f"outputs {TS_OUT_REL}")+ print(f" {JSON_OUT_REL}")+ print(f"consumer {CONSUMER_REL}")+ print()+ print(+ f"union {counts['total']} members = {counts[ORIGIN_REGISTRY]} registry "+ f"+ {counts[ORIGIN_PROFILE]} profile + {counts[ORIGIN_FOLD]} canonical-fold"+ )+ print(f"folds preserved {', '.join(f'{a} -> {t}' for a, t in payload['canonical'].items())}")+ print()++ if failures:+ print(f"FAIL — {len(failures)} finding(s):")+ for code, detail in failures:+ print(f" [{code}] {detail}")+ return 1+ print("OK — the committed union is what the registry, the profiles and the canonical")+ print(" folds generate, and the console re-exports it instead of redeclaring it.")+ return 0+++# --------------------------------------------------------------------------+# Self-test+# --------------------------------------------------------------------------+def self_test(root: Path) -> int:+ """Inject drift in each direction and require --check to catch each one."""+ try:+ inputs = load(root)+ except GateError as exc:+ print(f"self-test: cannot read the tree: {exc}", file=sys.stderr)+ return 2++ payload = build_payload(**inputs)+ good_ts, good_json = render_typescript(payload), render_json(payload)+ consumer = _read(root / CONSUMER_REL) or ""++ clean = evaluate(payload, good_ts, good_json, consumer)+ if clean:+ print("self-test: the unmodified tree already fails; fix that first", file=sys.stderr)+ for code, detail in clean:+ print(f" [{code}] {detail}", file=sys.stderr)+ return 1++ def drop_member(name: str) -> dict:+ reduced = dict(payload.items())+ reduced["members"] = [m for m in payload["members"] if m != name]+ return reduced++ hand_edited_ts = good_ts.replace(' "qualys",', "", 1)+ a_connector, a_fold = next(iter(inputs["registry"])), next(iter(inputs["canonical"]))++ cases: list[tuple[str, str, tuple]] = [+ (+ "a hand edit removes a union member from the generated TypeScript",+ "output-drifted",+ (payload, hand_edited_ts, good_json, consumer),+ ),+ (+ "a hand edit changes the generated JSON",+ "output-drifted",+ (payload, good_ts, good_json.replace('"total"', '"totl"', 1), consumer),+ ),+ (+ "the generated TypeScript is deleted",+ "output-missing",+ (payload, None, good_json, consumer),+ ),+ (+ "the console goes back to declaring its own union",+ "consumer-redeclares",+ (payload, good_ts, good_json, consumer + '\nexport type ConnectorType = "a" | "b";\n'),+ ),+ (+ "the console stops re-exporting the generated union",+ "consumer-not-reexporting",+ (payload, good_ts, good_json, _REEXPORT.sub("", consumer)),+ ),+ (+ "a canonical fold is flattened out of the union",+ "fold-source-dropped",+ (drop_member(a_fold), good_ts, good_json, consumer),+ ),+ (+ "a fold's target leaves the registry",+ "fold-target-dropped",+ (drop_member(inputs["canonical"][a_fold]), good_ts, good_json, consumer),+ ),+ (+ "the fold map is dropped from the rendered TypeScript",+ "fold-not-emitted",+ (payload, re.sub(r'^\s+"[a-z_]+": "[a-z_]+",$', "", good_ts, flags=re.MULTILINE), good_json, consumer),+ ),+ ]++ print(f"self-test against {root}")+ print(f"clean tree: {payload['counts']['total']}-member union, 0 failures (the baseline every case below perturbs)")+ print(f" a declared id is {a_connector!r}; a folded spelling is {a_fold!r}\n")+ ok = True+ for description, expected, call in cases:+ codes = {code for code, _ in evaluate(*call)}+ caught = expected in codes+ ok &= caught+ print(f" {'PASS' if caught else 'FAIL'} {description}")+ print(f" expected [{expected}] got {sorted(codes) or 'nothing'}")++ # The registry reader itself: identity must come from the class, because+ # two connectors are declared under an id their filename does not spell.+ by_filename = {info["module"][:-3] for info in inputs["registry"].values()}+ mismatched = sorted(cid for cid, info in inputs["registry"].items() if info["module"][:-3] != cid)+ filename_would_break = bool(mismatched) and set(inputs["registry"]) != by_filename+ print(f" {'PASS' if filename_would_break else 'FAIL'} IDENTITY: connector_id is read from the class, not the filename")+ print(f" {len(mismatched)} connector(s) would be misnamed by a filename slug: {', '.join(mismatched) or 'none'}")+ ok &= filename_would_break++ print()+ if not ok:+ print("self-test FAILED: the gate did not catch drift it claims to catch")+ return 1+ print(f"self-test OK: {len(cases)} injected defects, each caught by its own code")+ return 0+++if __name__ == "__main__":+ sys.exit(main())diff --git a/services/ingest/internal/normalizer/normalizer.go b/services/ingest/internal/normalizer/normalizer.goindex e1df9628f..c550be40f 100644--- a/services/ingest/internal/normalizer/normalizer.go+++ b/services/ingest/internal/normalizer/normalizer.go@@ -447,6 +447,25 @@ var connectorTypeAliases = map[string]string{// "Syslog / CEF"). Resolution happens once, at the top of Normalize, so the// profile lookup, the alias map, canonicalClassByConnector and the product// identity on the canonical path all agree on one name.+//+// `palo_alto_cortex` was the one entry that named a vendor rather than a+// product, and Palo Alto ships two the platform can ingest: `cortex_xdr` and+// `cortex_xsiam`. It folds onto Cortex XDR, and the record is:+//+// - the spelling entered packages/types in the initial-release commit+// (2026-05-01), before either connector existed — `cortex_xsiam` landed+// 2026-05-07 and `cortex_xdr` 2026-05-08 — so it was aspirational vendor+// vocabulary, not a reference to a connector somebody had wired;+// - outside this map, the union and the changelog, the string appears+// nowhere in the tree and never has: no console code, no saved connector+// instance, no seed row, no API catalog entry, no fixture. Nothing+// emitting it can be misrouted, because nothing emits it;+// - where the console does name the product it says "Cortex XDR" (the+// landing catalog lists it under EDR and lists no XSIAM), which matches+// CortexXDRConnector's own description, "Palo Alto Cortex XDR incidents+// via the public REST API";+// - XSIAM is a SIEM reachable under its own id, so nothing is lost: a+// deployment that means XSIAM says `cortex_xsiam` and always could.var connectorTypeCanonical = map[string]string{"google_chronicle": "chronicle","ibm_qradar": "qradar",diff --git a/tests/test_connector_types_generated.py b/tests/test_connector_types_generated.pynew file mode 100644index 000000000..d7c1fcdcf--- /dev/null+++ b/tests/test_connector_types_generated.py@@ -0,0 +1,225 @@+"""The console's ConnectorType union must be generated, not maintained.++The union was hand-written, and ten of its members named nothing the platform+could ingest. ``ibm_qradar`` was the expensive one: no connector declared it+and no profile was keyed on it, so strict mode rejected those events and+lenient mode minted a vendor called "ibm_qradar" — a second alert source for+the QRadar deployment ``qradar`` already fed.++PR #811 corrected the members and left the mechanism. These tests hold the+three properties that make the mechanism safe to remove:++ * the output is a pure function of the registry, so regeneration is+ idempotent and reordering the registry cannot change a byte;+ * a connector's identity comes from ``connector_id`` on the class, the way+ ``_build_registry()`` resolves it — two connectors in this tree are+ declared under an id their filename does not spell, so a filename slug+ would silently misname them (the defect+ ``scripts/generate_connector_docs.py`` shipped);+ * the canonical folds survive generation, because the console still emits+ ``ibm_qradar`` and the fold is what makes it arrive as ``qradar``.+"""++from __future__ import annotations++import sys+from pathlib import Path++import pytest++REPO_ROOT = Path(__file__).resolve().parent.parent+sys.path.insert(0, str(REPO_ROOT / "scripts"))++import generate_connector_types as gct # noqa: E402++TS_OUT = REPO_ROOT / gct.TS_OUT_REL+JSON_OUT = REPO_ROOT / gct.JSON_OUT_REL+CONSUMER = REPO_ROOT / gct.CONSUMER_REL++[email protected](scope="module")+def inputs() -> dict:+ return gct.load(REPO_ROOT)++[email protected](scope="module")+def payload(inputs: dict) -> dict:+ return gct.build_payload(**inputs)+++# --------------------------------------------------------------------------+# Committed output is current+# --------------------------------------------------------------------------+def test_committed_outputs_match_generation(payload: dict) -> None:+ assert TS_OUT.read_text(encoding="utf-8") == gct.render_typescript(+ payload+ ), f"{gct.TS_OUT_REL} is stale. Run `python3 scripts/generate_connector_types.py` and commit the result."+ assert JSON_OUT.read_text(encoding="utf-8") == gct.render_json(payload)+++def test_check_mode_passes_on_the_committed_tree() -> None:+ assert gct.main(["--check", "--repo-root", str(REPO_ROOT)]) == 0+++def test_self_test_passes() -> None:+ """The generator's own injected-drift suite, run as part of the test suite.++ A `--self-test` nothing invokes is the same defect one level up.+ """+ assert gct.self_test(REPO_ROOT) == 0+++# --------------------------------------------------------------------------+# Idempotence, and nothing positional+# --------------------------------------------------------------------------+def test_generation_is_idempotent(payload: dict, inputs: dict) -> None:+ once = gct.render_typescript(payload)+ twice = gct.render_typescript(gct.build_payload(**gct.load(REPO_ROOT)))+ assert once == twice+++def test_registry_order_does_not_change_the_output(payload: dict, inputs: dict) -> None:+ """Reordering _CONNECTOR_CLASSES must not move a single byte.++ ``scripts/generate_detections.py`` assigned ``det-{category}-{idx:03d}``+ by position, so inserting anywhere but the end renumbered every later+ rule and it needed a committed lock file to become stable. This generator+ assigns no identity — a member *is* its ``connector_id`` — so the same+ class of bug cannot arise, and this is the assertion that says so.+ """+ reversed_registry = dict(reversed(list(inputs["registry"].items())))+ shuffled = dict(inputs, registry=reversed_registry)+ assert gct.render_typescript(gct.build_payload(**shuffled)) == gct.render_typescript(payload)+ assert gct.render_json(gct.build_payload(**shuffled)) == gct.render_json(payload)+++# --------------------------------------------------------------------------+# Identity+# --------------------------------------------------------------------------+def test_identity_comes_from_the_class_not_the_filename(inputs: dict) -> None:+ """``jira_connector.py`` declares ``jira``; ``tenable.py`` declares ``tenable_io``.++ A generator that slugged the filename would emit ``jira_connector`` and+ ``tenable``, neither of which any profile, alias or fold resolves — the+ exact shape of the ten dead union members, reintroduced mechanically.+ """+ registry = inputs["registry"]+ mismatched = {cid: info["module"] for cid, info in registry.items() if info["module"][:-3] != cid}+ assert mismatched, "expected at least one connector whose filename does not spell its id"+ assert mismatched == {"jira": "jira_connector.py", "tenable_io": "tenable.py"}+ for connector_id in mismatched:+ assert connector_id in gct.build_payload(**inputs)["members"]+++def test_registry_matches_the_runtime_registry_key_set(inputs: dict) -> None:+ """The generator's ids are the keys ``CONNECTOR_REGISTRY`` is built from.++ Resolved independently here — walk the tuple, follow each import, read the+ class attribute — so this fails if the generator ever starts inferring+ identity some other way.+ """+ import ast++ init = ast.parse((REPO_ROOT / gct.REGISTRY_REL).read_text(encoding="utf-8"))+ modules = {+ alias.asname or alias.name: node.module.split(".")[-1]+ for node in ast.walk(init)+ if isinstance(node, ast.ImportFrom) and node.module and node.module.startswith("app.connectors.")+ for alias in node.names+ }+ expected = set()+ for class_name in gct._registered_class_names(init):+ tree = ast.parse((REPO_ROOT / gct.CONNECTORS_REL / f"{modules[class_name]}.py").read_text(encoding="utf-8"))+ expected.add(gct._class_attribute(tree, class_name, "connector_id"))+ assert set(inputs["registry"]) == expected+++def test_duplicate_connector_id_is_refused(tmp_path: Path) -> None:+ """``_build_registry()`` raises on a duplicate id; so must the generator.++ Silently collapsing two classes onto one id would shrink the union by one+ member and give no reason.+ """+ pkg = tmp_path / "services/connectors/app/connectors"+ pkg.mkdir(parents=True)+ (pkg / "alpha.py").write_text('class AlphaConnector:\n connector_id = "same"\n')+ (pkg / "beta.py").write_text('class BetaConnector:\n connector_id = "same"\n')+ (pkg / "__init__.py").write_text(+ "from app.connectors.alpha import AlphaConnector\n"+ "from app.connectors.beta import BetaConnector\n"+ "_CONNECTOR_CLASSES = (AlphaConnector, BetaConnector)\n"+ )+ with pytest.raises(gct.GateError, match="duplicate connector_id"):+ gct.parse_registry(tmp_path)+++# --------------------------------------------------------------------------+# The folds+# --------------------------------------------------------------------------+def test_canonical_folds_survive_generation(payload: dict, inputs: dict) -> None:+ assert inputs["canonical"], "the normalizer declares no canonical folds; the parser has drifted"+ rendered = gct.render_typescript(payload)+ for alternate, target in inputs["canonical"].items():+ assert alternate in payload["members"], f"{alternate!r} was flattened out of the union"+ assert target in payload["members"], f"{alternate!r} folds onto {target!r}, which left the union"+ assert f'"{alternate}": "{target}"' in rendered+++def test_the_fold_map_is_the_normalizers(payload: dict) -> None:+ """The five folds PR #811 established, read back off the generated file.++ Two products, one deployment: every source here must denote the same+ vendor product as its target, which is what makes folding safe.+ """+ assert payload["canonical"] == {+ "google_chronicle": "chronicle",+ "ibm_qradar": "qradar",+ "palo_alto_cortex": "cortex_xdr",+ "slack": "slack_audit",+ "syslog": "syslog_cef",+ }+++# --------------------------------------------------------------------------+# Drift detection+# --------------------------------------------------------------------------+def test_check_fails_on_a_hand_edit(payload: dict) -> None:+ """A member deleted by hand must fail --check rather than be regenerated away."""+ good = gct.render_typescript(payload)+ edited = good.replace(' "qualys",', "", 1)+ assert edited != good+ codes = {code for code, _ in gct.evaluate(payload, edited, gct.render_json(payload), CONSUMER.read_text(encoding="utf-8"))}+ assert "output-drifted" in codes+++def test_check_fails_when_the_console_redeclares_the_union(payload: dict) -> None:+ """The generated file can be perfectly current and completely ignored.++ A hand-written union in ``connector.ts`` is what TypeScript resolves, so+ "the generated file exists" is not the property that matters.+ """+ consumer = CONSUMER.read_text(encoding="utf-8") + '\nexport type ConnectorType = "only_this";\n'+ codes = {code for code, _ in gct.evaluate(payload, gct.render_typescript(payload), gct.render_json(payload), consumer)}+ assert "consumer-redeclares" in codes+++def test_new_connector_shows_up_as_drift(payload: dict, inputs: dict) -> None:+ """Adding a connector without regenerating must fail --check.++ This is the direction things actually change: a connector is added far+ more often than the union is edited.+ """+ grown = dict(inputs)+ grown["registry"] = {**inputs["registry"], "acme_xdr_9000": {"class": "AcmeConnector", "module": "acme.py", "name": "Acme"}}+ grown_payload = gct.build_payload(**grown)+ assert "acme_xdr_9000" in grown_payload["members"]+ codes = {+ code+ for code, _ in gct.evaluate(+ grown_payload,+ gct.render_typescript(payload), # the committed file, now stale+ gct.render_json(payload),+ CONSUMER.read_text(encoding="utf-8"),+ )+ }+ assert "output-drifted" in codesdiff --git a/tests/test_gate_coverage_classifier.py b/tests/test_gate_coverage_classifier.pynew file mode 100644index 000000000..bf3a15d07--- /dev/null+++ b/tests/test_gate_coverage_classifier.py@@ -0,0 +1,235 @@+"""A gate must be inventoried for what it does, not what it is called.++``scripts/check_gate_coverage.py`` resolves the workflow-to-check graph so an+orphaned gate cannot hide. It used to decide *which* scripts to resolve by+filename — ``check_*``, ``validate_*``, ``_conformance.py``, plus a list of+five exceptions — which is the same defect one level up: a gate named+something unexpected was not inventoried, and an uninventoried gate is+indistinguishable from one that does not exist.++Nineteen were in exactly that state. Every one is a CI gate today, eleven of+them run by a workflow with ``--check``; deleting any of those steps would+have left the script reporting full coverage over a smaller tree.++These tests pin the structural classifier, in both directions: a gate under a+name no convention would reveal must be found, and a generator that aborts on+an empty read must not be mistaken for one.+"""++from __future__ import annotations++import sys+from pathlib import Path++import pytest++REPO_ROOT = Path(__file__).resolve().parent.parent+sys.path.insert(0, str(REPO_ROOT / "scripts"))++import check_gate_coverage as cgc # noqa: E402++#: What the filename test used to accept. Kept here, in the test rather than+#: in the script, so "the classifier no longer depends on naming" is an+#: assertion instead of a claim.+_OLD_PREFIXES = ("check_", "validate_", "audit_", "lint_", "verify_")+_OLD_SUFFIXES = ("_check.py", "_gates.py", "_conformance.py", "_audit.py")+_OLD_EXTRA = {+ "connector_conformance.py",+ "detection_truth_table.py",+ "openapi_diff.py",+ "readme_gates.py",+ "security_audit.py",+}+++def _named_like_a_check(name: str) -> bool:+ if name in _OLD_EXTRA or Path(name).match("sync_vendored_*.py"):+ return True+ return name.startswith(_OLD_PREFIXES) or name.endswith(_OLD_SUFFIXES)++[email protected](scope="module")+def inventory() -> dict[str, dict[str, str]]:+ return cgc.load(REPO_ROOT)["checks"]+++# --------------------------------------------------------------------------+# A gate under an unconventional name+# --------------------------------------------------------------------------+UNCONVENTIONAL_GATE = '''\+#!/usr/bin/env python3+"""frobnicate_widgets.py — a gate whose name announces nothing."""++import sys+from pathlib import Path++ROOT = Path(__file__).resolve().parent.parent+++def main() -> int:+ offenders = []+ for path in ROOT.glob("docs/**/*.md"):+ if "\\t" in path.read_text(encoding="utf-8", errors="replace"):+ offenders.append(str(path))+ if offenders:+ print(f"{len(offenders)} document(s) contain a literal tab:")+ for path in offenders:+ print(f" {path}")+ return 1+ print("OK")+ return 0+++if __name__ == "__main__":+ sys.exit(main())+'''+++def test_a_gate_planted_under_an_unconventional_name_is_found() -> None:+ """The case the old classifier could not see.++ `frobnicate_widgets.py` matches no prefix, no suffix and no exception+ list, and nothing invokes it — so only an intrinsic signal can find it.+ """+ assert not _named_like_a_check("frobnicate_widgets.py")+ signals = cgc.classify_source(UNCONVENTIONAL_GATE)+ assert signals, "a script that exits non-zero on findings it collected must be inventoried"+ assert "findings-exit" in signals+++def test_the_planted_gate_is_reported_as_unreachable_when_nothing_runs_it(inventory) -> None:+ """Being found is only half of it — the gate must then fail on it.++ A classifier that inventories a script but whose rules never act on the+ inventory is the vacuous case this whole script exists to disprove.+ """+ checks = {**inventory, "frobnicate_widgets.py": cgc.classify_source(UNCONVENTIONAL_GATE)}+ routes = {name: ["workflow:ci.yml"] for name in inventory}+ routes["frobnicate_widgets.py"] = []+ codes = {code for code, _ in cgc.evaluate(checks, routes, [], {})}+ assert "check-unreachable" in codes+++# --------------------------------------------------------------------------+# The inverse: not everything that exits non-zero is a gate+# --------------------------------------------------------------------------+ABORTING_GENERATOR = '''\+#!/usr/bin/env python3+"""Renders a document. Exits non-zero when it has nothing to render."""++import sys+from pathlib import Path+++def main() -> int:+ specs = sorted(Path("specs").glob("*.yaml"))+ if not specs:+ print("no specs found", file=sys.stderr)+ return 1+ Path("out.md").write_text("\\n".join(p.name for p in specs))+ return 0+++if __name__ == "__main__":+ sys.exit(main())+'''++NETWORK_TOOL = '''\+#!/usr/bin/env python3+"""Posts events to a running service. Not a verdict about the tree."""++import sys+import urllib.request++EVENTS = [{"a": 1}, {"b": 2}]+++def main() -> int:+ sent = 0+ for event in EVENTS:+ urllib.request.urlopen("http://localhost:8000", data=b"{}")+ sent += 1+ if sent < len(EVENTS):+ return 1+ return 0+++if __name__ == "__main__":+ sys.exit(main())+'''+++def test_a_generator_that_aborts_on_an_empty_read_is_not_a_check() -> None:+ """Polarity is the whole distinction.++ ``if not specs: return 1`` says "I could not do my job". ``if offenders:+ return 1`` says "I found something". Only the second is a verdict.+ """+ assert cgc.classify_source(ABORTING_GENERATOR) == {}+++def test_a_script_that_only_talks_to_a_service_is_not_a_check() -> None:+ assert cgc.classify_source(NETWORK_TOOL) == {}+++# --------------------------------------------------------------------------+# What the change did to the inventory+# --------------------------------------------------------------------------+def test_structural_classification_is_a_superset_of_the_old_names(inventory) -> None:+ """Nothing the filename test caught may be dropped.++ A classifier that traded nineteen false negatives for one is not an+ improvement, and a silently smaller inventory is how coverage is lost.+ """+ named = {p.name for p in (REPO_ROOT / "scripts").glob("*.py") if _named_like_a_check(p.name)}+ dropped = named - set(inventory)+ assert not dropped, f"structural classification lost {sorted(dropped)}"+++def test_the_inventory_grew_by_the_scripts_the_names_hid(inventory) -> None:+ """The count moved, and this says by how much and why.++ Every added script is classified by an intrinsic signal — what it does —+ not by the fact that something happens to run it.+ """+ named = {p.name for p in (REPO_ROOT / "scripts").glob("*.py") if _named_like_a_check(p.name)}+ added = set(inventory) - named+ assert len(added) >= 15, f"expected the name-based test to have been hiding gates; found {sorted(added)}"+ for name in added:+ assert set(inventory[name]) - {"gates-a-workflow"}, f"{name} is inventoried only because something runs it"+++def test_every_verdict_flag_invocation_in_ci_is_inventoried() -> None:+ """CLASSIFIER -> WORKFLOW, the other direction.++ If a workflow runs a script with ``--check`` it is asking for a verdict.+ A classifier that does not call that script a check has a blind spot, and+ this is the assertion that finds it rather than a reader noticing.+ """+ data = cgc.load(REPO_ROOT)+ missing = sorted(set(data["verdict_invocations"]) - set(data["checks"]))+ assert not missing, f"CI asks these for a verdict but the classifier does not inventory them: {missing}"+++def test_blind_spot_rule_fires(inventory) -> None:+ codes = {+ code+ for code, _ in cgc.evaluate(+ inventory,+ {name: ["workflow:ci.yml"] for name in inventory},+ [],+ {},+ {"some_unclassified_thing.py": "ci.yml"},+ )+ }+ assert "classifier-blind-spot" in codes+++def test_generated_connector_types_is_inventoried(inventory) -> None:+ """The generator this change also added must itself be under the gate."""+ assert "generate_connector_types.py" in inventory+ assert "verdict-flag" in inventory["generate_connector_types.py"]+++def test_self_test_passes() -> None:+ assert cgc.self_test(REPO_ROOT) == 0