Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

Generate the connector-type union, and classify a gate by what it does (#816)

Beenu Arora034e374f761c
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 7e484d152..71f81e070 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -288,6 +288,31 @@ jobs:
# commit the result.
- name: Verify connector count is in sync (Phase 1.1)
run: python scripts/generate_connector_count.py --check
+ # The count comes from the registry; so, now, does the console's
+ # `ConnectorType` union. It used to be written by hand, which is how ten
+ # members came to name nothing the platform ingests — `ibm_qradar`
+ # reached no profile and no connector, so strict mode rejected those
+ # events and lenient mode minted a vendor named after the spelling, a
+ # second alert source for a QRadar deployment `qradar` already fed.
+ #
+ # The self-test runs first and on the same tree: it hand-edits the
+ # generated file, deletes it, puts the union back in connector.ts and
+ # flattens a canonical fold, and requires each to be caught.
+ - name: Connector type union — self-test
+ run: python3 scripts/generate_connector_types.py --self-test
+ - name: Verify the ConnectorType union matches the registry
+ run: python3 scripts/generate_connector_types.py --check
+ # The properties that make generation safe to rely on, asserted rather
+ # than assumed: regeneration is idempotent, reversing the registry
+ # changes no byte (nothing is positional, so no lock file is needed),
+ # and identity is read off the class — `jira_connector.py` declares
+ # `jira` and `tenable.py` declares `tenable_io`, so a filename slug
+ # would misname both, which is how `generate_connector_docs.py` came to
+ # emit duplicate pages while reporting 100% coverage.
+ - name: Connector type union — generation properties
+ run: |
+ python3 -m pip install --quiet pytest
+ python3 -m pytest tests/test_connector_types_generated.py -q
# The count says how many connectors exist; this says whether the name
# each one is registered under still reaches a normalization profile.
# The connectors service declares `crowdstrike` and `okta` while the
@@ -331,6 +356,14 @@ jobs:
# without anyone noticing.
- name: Verify the funnel response contract is documented
run: python scripts/check_funnel_contract.py
+ # Phase 2.1 shipped this gate and left it unwired, with "wire this into
+ # CI itself once the matrix has stabilised" in its own docstring. It
+ # stayed unwired, so the audit it automates — a service that ships tests
+ # no job collects — went back to being something a human had to notice.
+ # Seven services were in that state before Phase 2.1, 971 tests between
+ # them. The matrix has stabilised: all 13 tested services are gated.
+ - name: Every tested Python service is gated by a CI job
+ run: python3 scripts/list_python_services_with_tests.py --check
# A support bundle is the most concentrated pile of configuration a
# deployment produces, and its whole purpose is to leave the building.
# Redaction is allow-list rather than deny-list, so a setting added
@@ -571,12 +604,19 @@ jobs:
# reach. The inverse direction too — a workflow step naming a
# scripts/ path that no longer exists cannot do what it says.
#
+ # Which scripts it resolves is decided structurally — a declared
+ # verdict flag, an exit status derived from findings the script
+ # accumulates, or a workflow job whose output another job branches on.
+ # It used to be decided by filename, which hid nineteen gates, fifteen
+ # of them run by a workflow with `--check`.
+ #
# Self-test first, on the same tree, so a resolver that quietly starts
# returning "reached" for everything fails here rather than printing OK.
run: |
- python -m pip install --quiet pyyaml
+ python -m pip install --quiet pyyaml pytest
python3 scripts/check_gate_coverage.py --self-test
python3 scripts/check_gate_coverage.py --list
+ python3 -m pytest tests/test_gate_coverage_classifier.py -q
- name: Gate store migration runners and their call sites
# Neo4j, ClickHouse and Qdrant hold schema that code depends on, and
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 5cf59fa17..336a368ec 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -77,6 +77,55 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
+- **The console's `ConnectorType` union is generated from the connector
+ registry.** Its members were corrected in the previous release; the
+ mechanism that let them drift was not. Hand-maintenance is how ten of them
+ came to name nothing the platform could ingest, and `ibm_qradar` is what
+ that costs: no connector declared it and no profile was keyed on it, so
+ strict mode rejected those events while lenient mode minted a vendor called
+ "ibm_qradar" — a second alert source for the QRadar deployment `qradar`
+ already fed.
+
+ `scripts/generate_connector_types.py` derives the union from the three
+ places that decide what the normalizer resolves: the `_CONNECTOR_CLASSES`
+ tuple in `services/connectors/app/connectors/__init__.py` (84 ids), the
+ `connectorProfiles` keys in the ingest normalizer (5 more that no connector
+ declares under that spelling, including the legacy `splunk_enterprise`), and
+ the `connectorTypeCanonical` fold sources (5 alternate spellings the console
+ still emits). 94 members, written to
+ `packages/types/src/generated/connector-types.ts` as a `CONNECTOR_TYPES`
+ tuple the union is taken from, alongside a machine-readable
+ `connector-types.json` recording each member's origin. `connector.ts`
+ re-exports rather than redeclares, and `--check` fails if it goes back to
+ declaring its own — a generated file can be perfectly current and completely
+ ignored while a hand-written union three directories away is what TypeScript
+ resolves.
+
+ That set is, by construction, the `resolvable` set
+ `scripts/check_connector_profiles.py` already computed, so the two gates now
+ share one definition instead of holding two. The fold map is emitted with
+ the union as `CONNECTOR_TYPE_CANONICAL`, so the console can resolve a stored
+ spelling to one product name the same way the normalizer does.
+
+ Nothing is positional, so no lock file is needed and the
+ `generate_detections.py` renumbering trap does not apply: a member *is* its
+ `connector_id` and the output is a sorted set. A test reverses the registry
+ and asserts the output does not move a byte. Identity is read off the class,
+ the way `_build_registry()` resolves it — `jira_connector.py` declares
+ `jira` and `tenable.py` declares `tenable_io`, so a filename slug would
+ misname both, which is the defect `generate_connector_docs.py` shipped when
+ it generated duplicate pages for six connectors while reporting 100%
+ coverage.
+
+ The `palo_alto_cortex` fold was the one entry naming a vendor rather than a
+ product, and Palo Alto ships two the platform ingests. It stays on
+ `cortex_xdr`: the spelling entered the union in the initial-release commit
+ before either connector existed, it appears nowhere else in the tree and
+ never has (no console code, no saved instance, no seed, no catalog entry, no
+ fixture), the console's own catalog names "Cortex XDR" under EDR and lists
+ no XSIAM, and XSIAM is reachable under `cortex_xsiam` for any deployment
+ that means it. The evidence is recorded beside the map in `normalizer.go`.
+
- **`scripts/check_gate_coverage.py` — the gate on the gates.** The most
expensive recurring defect in this repository is a mechanism that exists, is
tested, and has no caller on the path that needs it. A gate is the worst case
@@ -339,6 +388,46 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Changed
+- **`check_gate_coverage.py` decides what a check is from what a script does,
+ not what it is called.** It classified by filename — `check_*`,
+ `validate_*`, `_conformance.py`, plus a hand-kept list of five exceptions
+ for the ones whose names did not announce a verdict. That is the same defect
+ the script exists to catch, one level up: a gate named something unexpected
+ was simply not inventoried, and an uninventoried gate is indistinguishable
+ from one that does not exist.
+
+ Nineteen were in that state and every one is a CI gate. Fifteen are run by a
+ workflow with `--check` — `generate_connector_count.py`,
+ `generate_connector_docs.py`, `generate_detections.py`,
+ `generate_slo_alerts.py`, the four `export_*` scripts, `build_marketplace.py`,
+ `build_quarantine_index.py`, `curate_detections.py`, `project_stats.py`,
+ `storage_cost_model.py` and two more. Deleting any of those steps would have
+ left the script reporting full coverage over a smaller tree.
+
+ Classification is now three structural signals, all read from the tree: a
+ declared verdict flag (`--check`, `--verify`, `--strict`, `--fail-*`,
+ `--max-*`, `--self-test`); an exit status derived from findings the script
+ accumulates; or a workflow job whose output another job branches on, which
+ is how `wet_eval_check.py` gates — it always exits 0 by design and publishes
+ its verdict as a JSON status file. The first two are intrinsic, so a gate
+ nothing calls is still inventoried and the "unreachable check" direction
+ does not become vacuous.
+
+ Polarity is the distinction that keeps it from over-reporting: `if
+ offenders: return 1` is a finding, `if not specs: return 1` is a generator
+ aborting on an empty read. A script that only talks to a running service is
+ excluded for the same reason — its non-zero exit is an operational error,
+ not a verdict on the tree.
+
+ The classifier is checked in reverse too: a script CI runs with a verdict
+ flag that the classifier does not inventory now fails as
+ `classifier-blind-spot`, so the classifier cannot silently narrow. The
+ inventory goes 42 → 59, and the first run of the new one found a real
+ orphan — `list_python_services_with_tests.py`, whose own docstring says
+ "wire this into CI itself once the matrix has stabilised" and which had
+ stayed unwired. It is wired now, and passes: all 13 tested Python services
+ are gated.
+
- **A hosted deployment's hostname no longer appears in self-hosted docs as the
reader's own URL.** Seven files under `apps/docs/` pointed at the managed
instance: two `curl` examples told a self-hoster to push their SIEM data to
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 881d6d2da..0a9c9c6d8 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -328,6 +328,17 @@ Add your class to the `_CONNECTOR_CLASSES` tuple and `__all__` in
The registry powers `/connectors/schemas` and the wizard's catalog grid —
no other wiring required.
+Then regenerate the two things derived from that tuple, and commit the result:
+
+```bash
+python3 scripts/generate_connector_count.py # the "N connectors" claims
+python3 scripts/generate_connector_types.py # the console's ConnectorType union
+```
+
+Both have a `--check` mode wired into CI, so forgetting is a red build rather
+than a quiet drift. The union used to be hand-written and ten of its members
+named nothing the platform could ingest.
+
### 3. Add a marketplace plugin manifest
Drop a `plugins/<connector-id>/plugin.yaml` mirroring your `schema()`. See
diff --git a/apps/docs/docs/compliance/evidence-pack.md b/apps/docs/docs/compliance/evidence-pack.md
index 97ec095ef..a44654b5b 100644
--- a/apps/docs/docs/compliance/evidence-pack.md
+++ b/apps/docs/docs/compliance/evidence-pack.md
@@ -33,7 +33,7 @@ and says so.
| Prompt-injection resistance | 32-payload adversarial corpus with a recall floor and a false-positive ceiling | `services/agents/tests/adversarial/` |
| Claim-to-gate traceability | Every product claim mapped to the CI job that fails when it stops being true | `docs/audit/CLAIM_TO_GATE_MATRIX.md` |
-The last row is the one worth reading first. It is the honest index: 113 rows
+The last row is the one worth reading first. It is the honest index: 115 rows
`GATED`, 9 `PARTIAL` with the specific gap named, and a ratchet that refuses
to let a row sit at `NO GATE`.
diff --git a/apps/docs/docs/connectors/hello-connector.md b/apps/docs/docs/connectors/hello-connector.md
index 574ed7850..3b25597d4 100644
--- a/apps/docs/docs/connectors/hello-connector.md
+++ b/apps/docs/docs/connectors/hello-connector.md
@@ -265,11 +265,12 @@ When you're ready to ship a real connector, here is the exact checklist:
1. **Move the file.** From `services/connectors/app/connectors/_examples/<your_connector>.py` to `services/connectors/app/connectors/<your_connector>.py`.
2. **Register it.** Add the class to `_CONNECTOR_CLASSES` in `services/connectors/app/connectors/__init__.py`. Keep the tuple alphabetised by `connector_id` to keep diffs predictable. Add the class name to the `__all__` list at the bottom of the same file.
-3. **Add a marketplace manifest.** Create `plugins/<connector-id>/plugin.yaml` mirroring your `schema()`. The fields and secret flags must match. Use [`plugins/auth0/plugin.yaml`](https://github.com/beenuar/AiSOC/blob/main/plugins/auth0/plugin.yaml) as a template.
-4. **Sync the marketplace.** Run `pnpm marketplace:sync` from the repo root. This regenerates the static catalog under `apps/web/public/marketplace/` so the catalog grid in the UI picks up the new entry.
-5. **Write a docs page.** Add `apps/docs/docs/connectors/<your-connector>.md` and wire it into `apps/docs/sidebars.ts` under the `Connectors` category. Use [`apps/docs/docs/connectors/cloudflare.md`](https://github.com/beenuar/AiSOC/blob/main/apps/docs/docs/connectors/cloudflare.md) as a template — `What you get` table → `Prerequisites` → `Setup walkthrough` → `Polling details` → `Severity heuristics` → `Troubleshooting`.
-6. **Add real tests.** Use `respx` to mock the vendor API. At minimum, exercise `test_connection()` (success + auth failure + network failure), `fetch_alerts()` (happy path + empty + 5xx + auth expiry), and `normalize()` (every severity branch).
-7. **Open a PR.** The code review will check that schema fields are wizard-renderable, that secrets are marked `secret=True`, that the category fits the existing taxonomy, that polling failures are non-terminal, and that the docs page exists.
+3. **Regenerate what the registry derives.** Run `python3 scripts/generate_connector_count.py` and `python3 scripts/generate_connector_types.py`, then commit the result. The first keeps every "N connectors" claim honest; the second regenerates the console's `ConnectorType` union from your `connector_id`. Both have a `--check` mode in CI, so skipping this is a red build rather than a silent drift.
+4. **Add a marketplace manifest.** Create `plugins/<connector-id>/plugin.yaml` mirroring your `schema()`. The fields and secret flags must match. Use [`plugins/auth0/plugin.yaml`](https://github.com/beenuar/AiSOC/blob/main/plugins/auth0/plugin.yaml) as a template.
+5. **Sync the marketplace.** Run `pnpm marketplace:sync` from the repo root. This regenerates the static catalog under `apps/web/public/marketplace/` so the catalog grid in the UI picks up the new entry.
+6. **Write a docs page.** Add `apps/docs/docs/connectors/<your-connector>.md` and wire it into `apps/docs/sidebars.ts` under the `Connectors` category. Use [`apps/docs/docs/connectors/cloudflare.md`](https://github.com/beenuar/AiSOC/blob/main/apps/docs/docs/connectors/cloudflare.md) as a template — `What you get` table → `Prerequisites` → `Setup walkthrough` → `Polling details` → `Severity heuristics` → `Troubleshooting`.
+7. **Add real tests.** Use `respx` to mock the vendor API. At minimum, exercise `test_connection()` (success + auth failure + network failure), `fetch_alerts()` (happy path + empty + 5xx + auth expiry), and `normalize()` (every severity branch).
+8. **Open a PR.** The code review will check that schema fields are wizard-renderable, that secrets are marked `secret=True`, that the category fits the existing taxonomy, that polling failures are non-terminal, and that the docs page exists.
That's the full path. If any of those steps feels heavier than you expected, file a bug — the goal is for adding a connector to feel mechanical, not heroic.
diff --git a/docs/audit/CLAIM_TO_GATE_MATRIX.md b/docs/audit/CLAIM_TO_GATE_MATRIX.md
index 14dbcc7a5..80e5cac7e 100644
--- a/docs/audit/CLAIM_TO_GATE_MATRIX.md
+++ b/docs/audit/CLAIM_TO_GATE_MATRIX.md
@@ -50,6 +50,7 @@ Statuses: `GATED` (a CI job fails when the claim stops being true) · `PARTIAL`
| Unified Data Explorer (NL + SQL across the lake) replaces the SIEM context-switch | README (data explorer) | `ci.yml` web-test job (`ExploreView.test.tsx` — NL question → `nl-query/translate` → `lake/sql` → BI table; raw-SQL run; source pivots to identity/graph/intel; error surfaced) + web-lint type-check | GATED | - |
| Autonomy posture is copilot-by-default with a visible per-action/blast scorecard | README (autopilot/copilot) | `ci.yml` web-test job (`AutonomyScorecard.test.tsx` — posture stays Copilot unless a high/critical-blast action auto-executes; distribution + override counts; rendered badge) | GATED | - |
| Broad SIEM/NDR/edge coverage (QRadar, Exabeam, Securonix, Devo, Netskope, Windows/Sysmon, Zeek/Suricata, syslog/CEF) | README (connector count) | `ci.yml` connectors job (`test_schemas.py` + `test_conformance.py` gate every connector; `test_siem_expansion.py` + `test_ndr_edge_expansion.py` pin each new connector's severity-ladder mapping) + `generate_connector_count.py --check` (registry-derived count, README phrase) | GATED | - |
+| The console's `connector_type` vocabulary names only connectors the platform can ingest | `packages/types/src/generated/connector-types.ts` (the exported `ConnectorType` union) | `ci.yml` python-lint job (`scripts/generate_connector_types.py --self-test` then `--check` — the union is generated from `_CONNECTOR_CLASSES`, the `connectorProfiles` keys and the `connectorTypeCanonical` folds, so it cannot be hand-edited into naming something nothing resolves; the self-test hand-edits the output, deletes it, puts a union back in `connector.ts` and flattens a fold, and requires each to be caught) + `check_connector_profiles.py` (every member resolved against the normalizer and the registry, in both directions) + `tests/test_connector_types_generated.py` (13 cases: regeneration is idempotent, reversing the registry changes no byte, identity is read off the class so `jira_connector.py` -> `jira` and `tenable.py` -> `tenable_io` are not misnamed, and a new connector shows up as drift) | GATED | - |
| AI/LLM-usage governance: audit connector + detections + tiered lake storage | README (AI usage / storage) | `ci.yml` connectors job (`test_llm_usage.py` — OpenAI/Anthropic audit connector emits the dotted `event_type` the `llm-*` rules match) + `validate-detections.yml` (8 `llm-*` native rules replay) + api job (`test_storage_tiering.py` — hot/cold TTL-move policy + DDL gate; verified on live ClickHouse) | GATED | - |
| Outcomes compound: repeat alerts auto-suppressed from prior dispositions (measured, not estimated) | README (closed loop) | `ci.yml` agents job (`test_outcome_memory.py` — every durable triage outcome is written back as a per-signature prior; a later alert matching a trusted prior benign/FP disposition is auto-resolved WITHOUT re-triage; human priors trusted immediately, AI priors require corroboration, a prior TP never auto-closes) + api funnel `repeat_alerts_suppressed`/`repeat_suppression_rate` measured from `aisoc_outcome_suppressions` | GATED | - |
| Institutional memory nudges fuse-time confidence (bounded, per-tenant) | README (compounding memory) | `ci.yml` fusion job (`test_memory_nudge.py` — distilled per-signature priors nudge new alerts of that signature ±0.10 in `ConfidenceScorer`; unknown signatures unchanged; `MemoryPriorProvider` distils per-tenant priors from disposition history so one tenant never nudges another) | GATED | - |
@@ -130,10 +131,11 @@ Statuses: `GATED` (a CI job fails when the claim stops being true) · `PARTIAL`
| A step a playbook engine cannot run never reports that it ran | concepts/playbooks.md (error handling) | `ci.yml` agents job (`test_playbook_engine_correctness.py::TestUnimplementedStepTypesFailClosed` — an `approval` gate the engine cannot honour halts the run instead of passing it, every handler-less `StepType` is asserted to fail rather than succeed, a missing handler is not retried, an explicit `on_failure: continue` is still honoured, and a dry run reports `would_fail` rather than a bare `dry_run: true`) | GATED | - |
| Every `ActionType` resolves a capability contract, so the confidence matrix is never skipped | concepts/live-actions.md (the one alias, and when it goes) | `ci.yml` actions job (`scripts/check_action_contract.py` directions 7 and 8 — every `ActionType` must resolve a contract by name or alias, and every alias must key a real `ActionType`, point at a real contract, not shadow a capability of the same name, and describe a bridge some adapter actually implements) + `test_approval_gate_wiring.py` (`notify_slack` at 10% confidence was approved for auto-execution and now requires an analyst; the alias is asserted unable to *lower* a requirement) | GATED | one alias entry (`notify_slack` -> `notify`), with its retirement condition recorded: it goes when `ActionType` does |
| Every check this matrix names is actually run by CI | this matrix (each row names a gate) | `ci.yml` (`scripts/check_gate_coverage.py` — resolves the whole graph rather than reading workflow names: workflows, `make` recipes they invoke, the pytest suites they collect and the scripts those reach, so a check wired only through a test still counts. Both directions — a check no workflow reaches, and a workflow step naming a `scripts/` path that does not exist. `--self-test` runs first and asserts the resolver discriminates rather than returning "reached" for everything, and the corpus counts are printed so it cannot report on a tree it never opened) | GATED | the ratchet for a deliberately-unwired check is empty; it may shrink, never grow |
+| What counts as a check is decided by what a script does, not what it is named | `scripts/check_gate_coverage.py` (the inventory it reports on) | `ci.yml` (three structural signals, all read from the tree: a declared verdict flag, an exit status derived from findings the script accumulates, and a workflow job whose output another job branches on. The first two are intrinsic, so an uncalled gate is still inventoried and the `SCRIPT -> WORKFLOW` direction is not vacuous. The classifier is also checked in reverse — a script CI runs with `--check` that the classifier does not inventory fails as `classifier-blind-spot`) + `tests/test_gate_coverage_classifier.py` (10 cases, including a gate planted under a name no convention would reveal, and a generator that aborts on an empty read asserted *not* to be a check) | GATED | - |
## Summary
-- GATED: 113
+- GATED: 115
- PARTIAL: 9
- NO GATE: 0 (**every claim is backed by a failing test.** The last NO GATE — the weekly benchmark scoreboard running live against `main` — closed in Phase E1: `scripts/check_scoreboard.py` ties the published scoreboard to a deterministic per-PR live-agent MITRE-accuracy run, while the funded weekly `wet-eval.yml` appends the LLM-tier rows. The remaining PARTIAL rows are honest, named deferrals — each states the specific gap and what closes it — not unproven claims.)
diff --git a/packages/types/src/connector.ts b/packages/types/src/connector.ts
index 73d55949b..84a713849 100644
--- a/packages/types/src/connector.ts
+++ b/packages/types/src/connector.ts
@@ -2,45 +2,42 @@
* Connector / Integration types
*/
+import type { ConnectorType } from "./generated/connector-types";
+
/**
* The `connector_type` values the ingest normalizer resolves.
*
- * This union is the console's vocabulary, and it is a third name space beside
- * the ids `services/connectors` declares and the keys `connectorProfiles` in
- * `services/ingest/internal/normalizer/normalizer.go` uses. Ten members used to
- * name nothing in either: `ibm_qradar` was not a profile key and no connector
+ * This union was hand-written, and hand-maintenance is what let it drift: it
+ * was a third name space beside the ids `services/connectors` declares and the
+ * keys `connectorProfiles` in
+ * `services/ingest/internal/normalizer/normalizer.go` uses, and ten members
+ * named nothing in either. `ibm_qradar` was not a profile key and no connector
* declared it, so strict mode rejected it and lenient mode minted a vendor
* called "ibm_qradar" — a second alert source for the QRadar deployment
* `qradar` already fed.
*
- * Six were the same product under a longer name and now fold onto the declared
- * id through `connectorTypeCanonical`. Four named nothing the platform ingests
- * and are gone: `vectra_ai` (no Vectra connector exists), `teams` (a ChatOps
- * destination, not a source), and `custom_webhook` / `http_pull` / `kafka`
- * (transports — the webhook path is the tenant inbox, which keys off a template
- * id and never sets `connector_type`).
+ * It is now generated from the registry that decides what the platform can
+ * ingest. Five alternate spellings (`ibm_qradar`, `google_chronicle`,
+ * `palo_alto_cortex`, `slack`, `syslog`) remain members and fold onto the
+ * declared id through `connectorTypeCanonical`, which the generated module
+ * mirrors as `CONNECTOR_TYPE_CANONICAL`. Five named nothing the platform
+ * ingests and are gone: `vectra_ai` (no Vectra connector exists), `teams` (a
+ * ChatOps destination, not a source), and `custom_webhook` / `http_pull` /
+ * `kafka` (transports — the webhook path is the tenant inbox, which keys off a
+ * template id and never sets `connector_type`).
*
- * `scripts/check_connector_profiles.py` gates every member of this union
- * against the normalizer and the connector registry, in both directions.
+ * Do not redeclare the union here. `scripts/generate_connector_types.py
+ * --check` fails if this file declares its own, because a hand-written union
+ * three directories from the generated one is what TypeScript would resolve.
+ * `scripts/check_connector_profiles.py` gates every member against the
+ * normalizer and the connector registry, in both directions.
*/
-export type ConnectorType =
- | "crowdstrike_falcon"
- | "microsoft_sentinel"
- | "splunk_enterprise"
- | "aws_security_hub"
- | "okta_system_log"
- | "sentinelone"
- | "palo_alto_cortex"
- | "google_chronicle"
- | "ibm_qradar"
- | "darktrace"
- | "tenable_io"
- | "qualys"
- | "jira"
- | "servicenow"
- | "pagerduty"
- | "slack"
- | "syslog";
+export type { ConnectorType } from "./generated/connector-types";
+export {
+ CONNECTOR_TYPES,
+ CONNECTOR_TYPE_CANONICAL,
+ canonicalConnectorType,
+} from "./generated/connector-types";
export type ConnectorCategory =
| "edr"
diff --git a/packages/types/src/generated/connector-types.json b/packages/types/src/generated/connector-types.json
new file mode 100644
index 000000000..adbe337d8
--- /dev/null
+++ b/packages/types/src/generated/connector-types.json
@@ -0,0 +1,213 @@
+{
+ "canonical": {
+ "google_chronicle": "chronicle",
+ "ibm_qradar": "qradar",
+ "palo_alto_cortex": "cortex_xdr",
+ "slack": "slack_audit",
+ "syslog": "syslog_cef"
+ },
+ "counts": {
+ "aliases": 2,
+ "canonical-fold": 5,
+ "profile": 5,
+ "registry": 84,
+ "total": 94
+ },
+ "generatedFrom": [
+ "services/connectors/app/connectors/__init__.py",
+ "services/ingest/internal/normalizer/normalizer.go"
+ ],
+ "members": [
+ "abnormal_security",
+ "ai_gateway",
+ "ai_guardrail",
+ "ai_runtime",
+ "auditd",
+ "auth0",
+ "aws_cloudtrail",
+ "aws_guardduty",
+ "aws_security_hub",
+ "aws_vpc_flow",
+ "azure_activity",
+ "azure_defender",
+ "azure_entra",
+ "box",
+ "carbon_black",
+ "chronicle",
+ "cisco_umbrella",
+ "cloudflare",
+ "cloudflare_zt",
+ "confluence_audit",
+ "cortex_xdr",
+ "cortex_xsiam",
+ "crowdstrike",
+ "crowdstrike_falcon",
+ "darktrace",
+ "datadog",
+ "datadog_cloud_siem",
+ "devo",
+ "dropbox",
+ "duo_security",
+ "elastic",
+ "email_inbox",
+ "exabeam",
+ "falco",
+ "fleetdm",
+ "gcp_cloud_audit",
+ "gcp_scc",
+ "github",
+ "gitlab",
+ "google_chronicle",
+ "google_workspace",
+ "greynoise",
+ "ibm_qradar",
+ "imperva",
+ "jira",
+ "jumpcloud",
+ "kubernetes_audit",
+ "lacework",
+ "llm_usage",
+ "m365_audit",
+ "microsoft_sentinel",
+ "mimecast",
+ "netskope",
+ "oci",
+ "okta",
+ "okta_system_log",
+ "onepassword",
+ "opsgenie",
+ "orca",
+ "osctrl",
+ "pagerduty",
+ "palo_alto_cortex",
+ "prisma_cloud",
+ "proofpoint",
+ "qradar",
+ "qualys",
+ "rapid7_insightidr",
+ "salesforce",
+ "securonix",
+ "sentinelone",
+ "servicenow",
+ "slack",
+ "slack_audit",
+ "snowflake",
+ "snyk",
+ "splunk",
+ "splunk_enterprise",
+ "sublime_security",
+ "sumo_logic",
+ "sysdig",
+ "syslog",
+ "syslog_cef",
+ "tailscale",
+ "tenable_io",
+ "tines",
+ "torq",
+ "trellix_helix",
+ "trend_vision_one",
+ "vault",
+ "wazuh",
+ "windows_event",
+ "wiz",
+ "zeek_suricata",
+ "zscaler"
+ ],
+ "origins": {
+ "abnormal_security": "registry",
+ "ai_gateway": "registry",
+ "ai_guardrail": "profile",
+ "ai_runtime": "profile",
+ "auditd": "registry",
+ "auth0": "registry",
+ "aws_cloudtrail": "registry",
+ "aws_guardduty": "registry",
+ "aws_security_hub": "registry",
+ "aws_vpc_flow": "registry",
+ "azure_activity": "registry",
+ "azure_defender": "registry",
+ "azure_entra": "registry",
+ "box": "registry",
+ "carbon_black": "registry",
+ "chronicle": "registry",
+ "cisco_umbrella": "registry",
+ "cloudflare": "registry",
+ "cloudflare_zt": "registry",
+ "confluence_audit": "registry",
+ "cortex_xdr": "registry",
+ "cortex_xsiam": "registry",
+ "crowdstrike": "registry",
+ "crowdstrike_falcon": "profile",
+ "darktrace": "registry",
+ "datadog": "registry",
+ "datadog_cloud_siem": "registry",
+ "devo": "registry",
+ "dropbox": "registry",
+ "duo_security": "registry",
+ "elastic": "registry",
+ "email_inbox": "registry",
+ "exabeam": "registry",
+ "falco": "registry",
+ "fleetdm": "registry",
+ "gcp_cloud_audit": "registry",
+ "gcp_scc": "registry",
+ "github": "registry",
+ "gitlab": "registry",
+ "google_chronicle": "canonical-fold",
+ "google_workspace": "registry",
+ "greynoise": "registry",
+ "ibm_qradar": "canonical-fold",
+ "imperva": "registry",
+ "jira": "registry",
+ "jumpcloud": "registry",
+ "kubernetes_audit": "registry",
+ "lacework": "registry",
+ "llm_usage": "registry",
+ "m365_audit": "registry",
+ "microsoft_sentinel": "registry",
+ "mimecast": "registry",
+ "netskope": "registry",
+ "oci": "registry",
+ "okta": "registry",
+ "okta_system_log": "profile",
+ "onepassword": "registry",
+ "opsgenie": "registry",
+ "orca": "registry",
+ "osctrl": "registry",
+ "pagerduty": "registry",
+ "palo_alto_cortex": "canonical-fold",
+ "prisma_cloud": "registry",
+ "proofpoint": "registry",
+ "qradar": "registry",
+ "qualys": "registry",
+ "rapid7_insightidr": "registry",
+ "salesforce": "registry",
+ "securonix": "registry",
+ "sentinelone": "registry",
+ "servicenow": "registry",
+ "slack": "canonical-fold",
+ "slack_audit": "registry",
+ "snowflake": "registry",
+ "snyk": "registry",
+ "splunk": "registry",
+ "splunk_enterprise": "profile",
+ "sublime_security": "registry",
+ "sumo_logic": "registry",
+ "sysdig": "registry",
+ "syslog": "canonical-fold",
+ "syslog_cef": "registry",
+ "tailscale": "registry",
+ "tenable_io": "registry",
+ "tines": "registry",
+ "torq": "registry",
+ "trellix_helix": "registry",
+ "trend_vision_one": "registry",
+ "vault": "registry",
+ "wazuh": "registry",
+ "windows_event": "registry",
+ "wiz": "registry",
+ "zeek_suricata": "registry",
+ "zscaler": "registry"
+ },
+ "regenerateWith": "python3 scripts/generate_connector_types.py"
+}
diff --git a/packages/types/src/generated/connector-types.ts b/packages/types/src/generated/connector-types.ts
new file mode 100644
index 000000000..aff04ec89
--- /dev/null
+++ b/packages/types/src/generated/connector-types.ts
@@ -0,0 +1,134 @@
+// AUTO-GENERATED by scripts/generate_connector_types.py. Do not edit by hand.
+// Run `python3 scripts/generate_connector_types.py` and commit the result.
+//
+// Every `connector_type` the ingest normalizer resolves, derived from the
+// registry that decides what the platform can ingest:
+//
+// 84 connector ids declared in services/connectors (_CONNECTOR_CLASSES)
+// 5 connectorProfiles keys no connector declares under that spelling
+// 5 alternate spellings connectorTypeCanonical folds onto a declared id
+// 94 total
+//
+// The union used to be written by hand, which is how ten members came to name
+// nothing: `ibm_qradar` reached no profile and no connector, so strict mode
+// rejected those events and lenient mode minted a vendor called "ibm_qradar" —
+// a second alert source for the QRadar deployment `qradar` already fed.
+
+/** Every `connector_type` value the ingest normalizer resolves. */
+export const CONNECTOR_TYPES = [
+ "abnormal_security", // registry
+ "ai_gateway", // registry
+ "ai_guardrail", // profile
+ "ai_runtime", // profile
+ "auditd", // registry
+ "auth0", // registry
+ "aws_cloudtrail", // registry
+ "aws_guardduty", // registry
+ "aws_security_hub", // registry
+ "aws_vpc_flow", // registry
+ "azure_activity", // registry
+ "azure_defender", // registry
+ "azure_entra", // registry
+ "box", // registry
+ "carbon_black", // registry
+ "chronicle", // registry
+ "cisco_umbrella", // registry
+ "cloudflare", // registry
+ "cloudflare_zt", // registry
+ "confluence_audit", // registry
+ "cortex_xdr", // registry
+ "cortex_xsiam", // registry
+ "crowdstrike", // registry
+ "crowdstrike_falcon", // profile
+ "darktrace", // registry
+ "datadog", // registry
+ "datadog_cloud_siem", // registry
+ "devo", // registry
+ "dropbox", // registry
+ "duo_security", // registry
+ "elastic", // registry
+ "email_inbox", // registry
+ "exabeam", // registry
+ "falco", // registry
+ "fleetdm", // registry
+ "gcp_cloud_audit", // registry
+ "gcp_scc", // registry
+ "github", // registry
+ "gitlab", // registry
+ "google_chronicle", // canonical-fold
+ "google_workspace", // registry
+ "greynoise", // registry
+ "ibm_qradar", // canonical-fold
+ "imperva", // registry
+ "jira", // registry
+ "jumpcloud", // registry
+ "kubernetes_audit", // registry
+ "lacework", // registry
+ "llm_usage", // registry
+ "m365_audit", // registry
+ "microsoft_sentinel", // registry
+ "mimecast", // registry
+ "netskope", // registry
+ "oci", // registry
+ "okta", // registry
+ "okta_system_log", // profile
+ "onepassword", // registry
+ "opsgenie", // registry
+ "orca", // registry
+ "osctrl", // registry
+ "pagerduty", // registry
+ "palo_alto_cortex", // canonical-fold
+ "prisma_cloud", // registry
+ "proofpoint", // registry
+ "qradar", // registry
+ "qualys", // registry
+ "rapid7_insightidr", // registry
+ "salesforce", // registry
+ "securonix", // registry
+ "sentinelone", // registry
+ "servicenow", // registry
+ "slack", // canonical-fold
+ "slack_audit", // registry
+ "snowflake", // registry
+ "snyk", // registry
+ "splunk", // registry
+ "splunk_enterprise", // profile
+ "sublime_security", // registry
+ "sumo_logic", // registry
+ "sysdig", // registry
+ "syslog", // canonical-fold
+ "syslog_cef", // registry
+ "tailscale", // registry
+ "tenable_io", // registry
+ "tines", // registry
+ "torq", // registry
+ "trellix_helix", // registry
+ "trend_vision_one", // registry
+ "vault", // registry
+ "wazuh", // registry
+ "windows_event", // registry
+ "wiz", // registry
+ "zeek_suricata", // registry
+ "zscaler", // registry
+] as const;
+
+export type ConnectorType = (typeof CONNECTOR_TYPES)[number];
+
+/**
+ * Alternate spellings the normalizer folds onto the id services/connectors
+ * declares, before anything keys off it. Mirrors `connectorTypeCanonical` in
+ * services/ingest/internal/normalizer/normalizer.go so the console can resolve
+ * a stored value to one product name instead of two.
+ */
+export const CONNECTOR_TYPE_CANONICAL: Readonly<Record<string, ConnectorType>> = {
+ "google_chronicle": "chronicle",
+ "ibm_qradar": "qradar",
+ "palo_alto_cortex": "cortex_xdr",
+ "slack": "slack_audit",
+ "syslog": "syslog_cef",
+};
+
+/** Resolve an alternate spelling to the declared connector id. */
+export function canonicalConnectorType(connectorType: string): string {
+ return CONNECTOR_TYPE_CANONICAL[connectorType] ?? connectorType;
+}
diff --git a/scripts/check_connector_profiles.py b/scripts/check_connector_profiles.py
index 4527d54d9..110c22558 100644
--- a/scripts/check_connector_profiles.py
+++ b/scripts/check_connector_profiles.py
@@ -81,20 +81,26 @@ PUSH_ONLY_PROFILES: dict[str, str] = {
# checked against those files rather than asserted here.
LEGACY_PROFILE_KEYS = {"crowdstrike_falcon", "okta_system_log", "splunk_enterprise"}
-# packages/types' ConnectorType union is a third name space, and it predates
-# the connectors registry. Ten members named no connector the service declares
-# and no profile: the console's older vocabulary (``google_chronicle`` for
-# ``chronicle``, ``ibm_qradar`` for ``qradar``) plus four generic transports
-# that were never connectors at all.
+# packages/types' ConnectorType union used to be a third name space written by
+# hand, and it predated the connectors registry. Ten members named no connector
+# the service declares and no profile: the console's older vocabulary
+# (``google_chronicle`` for ``chronicle``, ``ibm_qradar`` for ``qradar``) plus
+# transports that were never connectors at all.
#
-# All ten are now closed. Six fold onto their declared connector through
-# ``connectorTypeCanonical``; four denoted nothing the platform ingests and
-# were removed from the union. This stays as an empty ratchet rather than being
-# deleted, because the drift it recorded is the kind that comes back: it may
-# shrink, never grow, and a new unresolvable union member fails the gate.
+# All ten are closed. Five fold onto their declared connector through
+# ``connectorTypeCanonical``; five denoted nothing the platform ingests and
+# were removed. The union is now generated by
+# ``scripts/generate_connector_types.py`` from the registry, the profile keys
+# and the fold map, so drift of this shape cannot be reintroduced by hand —
+# but the ratchet stays rather than being deleted, because it also catches a
+# generator that starts emitting a name nothing resolves. It may shrink, never
+# grow, and a new unresolvable union member fails the gate.
KNOWN_UNION_ONLY_TYPES: set[str] = set()
-TS_TYPES_REL = Path("packages/types/src/connector.ts")
+# The generated module, not the hand-written ``connector.ts`` that re-exports
+# it. Reading the consumer would mean this gate silently stopped inspecting
+# anything the day the union moved.
+TS_TYPES_REL = Path("packages/types/src/generated/connector-types.ts")
# Identity destinations the generic path depends on. Losing any one of these
# is the defect this gate exists for.
@@ -271,11 +277,11 @@ def parse_doc_examples(root: Path) -> dict[str, list[str]]:
# Gate
# --------------------------------------------------------------------------
def parse_union_types(path: Path) -> set[str]:
- """Members of packages/types' ConnectorType union."""
+ """Members of the generated CONNECTOR_TYPES tuple the union is built from."""
src = path.read_text(encoding="utf-8")
- match = re.search(r"export type ConnectorType\s*=(.*?);", src, re.DOTALL)
+ match = re.search(r"export const CONNECTOR_TYPES = \[(.*?)\] as const;", src, re.DOTALL)
if not match:
- raise GateError(f"could not find the ConnectorType union in {path}")
+ raise GateError(f"could not find the CONNECTOR_TYPES tuple in {path}")
return set(re.findall(r'"([^"]+)"', match.group(1)))
diff --git a/scripts/check_gate_coverage.py b/scripts/check_gate_coverage.py
index 0468c2bdc..5e9adefd3 100644
--- a/scripts/check_gate_coverage.py
+++ b/scripts/check_gate_coverage.py
@@ -51,9 +51,11 @@ opened is worse than no gate.
from __future__ import annotations
import argparse
+import ast
import json
import re
import sys
+from collections import Counter
from pathlib import Path
import yaml
@@ -65,23 +67,53 @@ MAKEFILE_REL = Path("Makefile")
#: Directories whose pytest suites a workflow can name.
TEST_ROOTS = ("tests", "services/*/tests", "packages/*/tests", "apps/*/tests")
-#: A script is a "check" when its name says it decides something. Generators,
-#: exporters and fixtures are out of scope: they have no verdict to report.
-_CHECK_PREFIXES = ("check_", "validate_", "audit_", "lint_", "verify_")
-_CHECK_SUFFIXES = ("_check.py", "_gates.py", "_conformance.py", "_audit.py")
-
-#: Named individually because their filenames do not announce a verdict, but
-#: they exit non-zero on a real finding and are relied on as gates.
-_CHECK_EXTRA = {
- "connector_conformance.py",
- "detection_truth_table.py",
- "openapi_diff.py",
- "readme_gates.py",
- "security_audit.py",
-}
-
-#: `sync_vendored_*.py --check` are drift gates; the sync half is a generator.
-_CHECK_GLOBS = ("sync_vendored_*.py",)
+# --------------------------------------------------------------------------
+# What counts as a check
+# --------------------------------------------------------------------------
+# This used to be a filename test: `check_*`, `validate_*`, `_conformance.py`
+# and a hand-kept list of five exceptions for the ones whose names did not
+# announce a verdict. That is the same defect the script exists to catch, one
+# level up — a gate named something unexpected was simply not inventoried, and
+# an uninventoried gate is indistinguishable from one that does not exist.
+# Nineteen were in that state, every one of them a CI gate: eleven
+# `generate_*`/`export_*`/`build_*` scripts a workflow runs with `--check`,
+# `project_stats.py`, `storage_cost_model.py`, `curate_detections.py` and the
+# rest. Deleting any of their workflow steps would have left this script
+# reporting full coverage.
+#
+# Classification is now structural: what a script *does*.
+#
+# verdict-flag it declares a CLI option whose only purpose is to turn
+# the run into a verdict.
+# findings-exit its exit status is derived from findings it accumulates
+# — non-zero when the accumulator is non-empty, or zero
+# when it is empty. The polarity matters: `if not specs:
+# return 1` is a generator aborting on an empty read, not
+# a gate reporting a finding.
+# gates-a-workflow a workflow job that runs it publishes an output another
+# job branches on. `wet_eval_check.py` is a preflight that
+# always exits 0 by design and reports its verdict in a
+# JSON status file; the job graph is where that shows.
+#
+# All three are read from the tree, never from the name. The first two are
+# intrinsic, so a gate is inventoried whether or not anything calls it — which
+# is what keeps the SCRIPT -> WORKFLOW direction below from being vacuous.
+
+#: Flags that exist only so a caller can act on the result.
+_VERDICT_FLAGS = ("--check", "--check-only", "--verify", "--strict", "--self-test")
+_VERDICT_PREFIXES = ("--fail-", "--max-", "--require-", "--assert-")
+
+#: A check reports on the repository. Scripts that talk only to a running
+#: service (`inject_scenario.py` posts alerts, `generate_runbook.py` queries a
+#: tracing backend) exit non-zero when the network call fails, which is an
+#: operational error and not a finding about the tree.
+_INSPECTS_TREE = re.compile(r"Path\(__file__\)|\.glob\(|\.rglob\(|\.iterdir\(|\.read_text\(|\.read_bytes\(|os\.walk\(")
+
+#: Calls that grow a collection.
+_MUTATORS = {"append", "add", "extend", "update"}
+
+#: Annotations that say a name holds a collection of findings.
+_COLLECTION_TYPES = {"list", "set", "dict", "tuple", "List", "Set", "Dict", "Counter", "defaultdict"}
#: Checks that are deliberately not wired, with the reason. Shrink-only: a
#: name here that turns out to be reachable fails the gate, so the list cannot
@@ -96,19 +128,231 @@ class GateError(RuntimeError):
# --------------------------------------------------------------------------
# Inventory
# --------------------------------------------------------------------------
-def is_check(name: str) -> bool:
- if name in _CHECK_EXTRA:
- return True
- if any(Path(name).match(g) for g in _CHECK_GLOBS):
- return True
- return name.startswith(_CHECK_PREFIXES) or name.endswith(_CHECK_SUFFIXES)
+def _with_parents(tree: ast.Module) -> ast.Module:
+ for node in ast.walk(tree):
+ for child in ast.iter_child_nodes(node):
+ child.parent = node # type: ignore[attr-defined]
+ return tree
+
+
+def _inside_except(node: ast.AST) -> bool:
+ parent = getattr(node, "parent", None)
+ while parent is not None:
+ if isinstance(parent, ast.ExceptHandler):
+ return True
+ parent = getattr(parent, "parent", None)
+ return False
+
+
+def _verdict_flags(tree: ast.Module) -> list[str]:
+ found = set()
+ for node in ast.walk(tree):
+ if isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute) and node.func.attr == "add_argument":
+ for arg in node.args:
+ if not (isinstance(arg, ast.Constant) and isinstance(arg.value, str) and arg.value.startswith("--")):
+ continue
+ if arg.value in _VERDICT_FLAGS or arg.value.startswith(_VERDICT_PREFIXES):
+ found.add(arg.value)
+ return sorted(found)
+
+
+def _is_collection_annotation(node: ast.AST | None) -> bool:
+ if node is None:
+ return False
+ base = node.value if isinstance(node, ast.Subscript) else node
+ return isinstance(base, ast.Name) and base.id in _COLLECTION_TYPES
+
+
+def _accumulators(tree: ast.Module) -> set[str]:
+ """Names and attributes the script collects findings into.
+
+ Attributes are included because a report object is the other common shape:
+ `security_audit.py` decides on `report.high_critical` and
+ `report.unscanned`, which are list fields it appends to.
+ """
+ assigned: set[str] = set()
+ zeroed: set[str] = set()
+ found: set[str] = set()
+ for node in ast.walk(tree):
+ targets: list[ast.AST] = []
+ value: ast.AST | None = None
+ annotation: ast.AST | None = None
+ if isinstance(node, ast.Assign):
+ targets, value = list(node.targets), node.value
+ elif isinstance(node, ast.AnnAssign):
+ targets, value, annotation = [node.target], node.value, node.annotation
+ for target in targets:
+ key = target.id if isinstance(target, ast.Name) else (target.attr if isinstance(target, ast.Attribute) else None)
+ if key is None:
+ continue
+ if isinstance(target, ast.Name):
+ assigned.add(key)
+ if isinstance(value, ast.List | ast.Set | ast.Dict | ast.ListComp | ast.SetComp | ast.DictComp | ast.GeneratorExp):
+ found.add(key)
+ if isinstance(value, ast.Call) and isinstance(value.func, ast.Name) and value.func.id in {"list", "set", "dict", "sorted"}:
+ found.add(key)
+ if _is_collection_annotation(annotation):
+ found.add(key)
+ if isinstance(value, ast.Constant) and value.value == 0:
+ zeroed.add(key)
+ if isinstance(node, ast.FunctionDef | ast.AsyncFunctionDef) and _is_collection_annotation(node.returns):
+ found.add(node.name)
+ if isinstance(node, ast.AugAssign) and isinstance(node.target, ast.Name) and node.target.id in zeroed:
+ found.add(node.target.id) # a counter of findings
+ if isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute) and node.func.attr in _MUTATORS:
+ container = node.func.value
+ if isinstance(container, ast.Name):
+ found.add(container.id)
+ elif isinstance(container, ast.Attribute):
+ found.add(container.attr)
+ if isinstance(node, ast.For):
+ if isinstance(node.iter, ast.Name) and node.iter.id in assigned:
+ found.add(node.iter.id)
+ elif isinstance(node.iter, ast.Attribute):
+ found.add(node.iter.attr)
+ if isinstance(node, ast.Call) and isinstance(node.func, ast.Name) and node.func.id == "len":
+ for arg in node.args:
+ if isinstance(arg, ast.Name) and arg.id in assigned:
+ found.add(arg.id)
+ elif isinstance(arg, ast.Attribute):
+ found.add(arg.attr)
+ return found
+
+
+def _referenced(node: ast.AST) -> set[str]:
+ out: set[str] = set()
+ for sub in ast.walk(node):
+ if isinstance(sub, ast.Name):
+ out.add(sub.id)
+ elif isinstance(sub, ast.Attribute):
+ out.add(sub.attr)
+ return out
+
+
+def _negated(test: ast.AST, names: set[str]) -> bool:
+ """Whether every accumulator in `test` appears under a `not`."""
+ positive = set()
+ for node in ast.walk(test):
+ if isinstance(node, ast.UnaryOp) and isinstance(node.op, ast.Not):
+ continue
+ if isinstance(node, ast.Name) and node.id in names:
+ positive.add(node.id)
+ elif isinstance(node, ast.Attribute) and node.attr in names:
+ positive.add(node.attr)
+ negated = set()
+ for node in ast.walk(test):
+ if isinstance(node, ast.UnaryOp) and isinstance(node.op, ast.Not):
+ negated |= _referenced(node.operand) & names
+ return bool(negated) and not (positive - negated)
+
+
+def _exit_status(node: ast.AST) -> ast.AST | None:
+ if isinstance(node, ast.Return):
+ return node.value
+ if isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute) and node.func.attr == "exit" and node.args:
+ return node.args[0]
+ if isinstance(node, ast.Raise) and isinstance(node.exc, ast.Call) and isinstance(node.exc.func, ast.Name):
+ if node.exc.func.id == "SystemExit" and node.exc.args:
+ return node.exc.args[0]
+ return None
+
+
+def _is_int(node: ast.AST | None, *, zero: bool) -> bool:
+ if not (isinstance(node, ast.Constant) and isinstance(node.value, int) and not isinstance(node.value, bool)):
+ return False
+ return (node.value == 0) if zero else (node.value != 0)
+
+
+def _exit_path(tree: ast.Module) -> list[ast.AST]:
+ """Functions whose return value becomes the process exit status.
+
+ Bounded transitive closure from whatever is passed to `sys.exit(...)`,
+ because dispatchers are common: `security_audit.py`'s `main` returns
+ `handlers[args.command](args)` and the verdict is three frames down. A
+ module with no exit path at all is a library, not a check.
+ """
+ funcs = {n.name: n for n in ast.walk(tree) if isinstance(n, ast.FunctionDef | ast.AsyncFunctionDef)}
+ entries: set[str] = set()
+ for node in ast.walk(tree):
+ args = None
+ if isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute) and node.func.attr == "exit":
+ args = node.args
+ elif isinstance(node, ast.Raise) and isinstance(node.exc, ast.Call) and isinstance(node.exc.func, ast.Name):
+ args = node.exc.args if node.exc.func.id == "SystemExit" else None
+ for arg in args or []:
+ entries |= {n.id for n in ast.walk(arg) if isinstance(n, ast.Name)} & set(funcs)
+ if not entries and "main" in funcs:
+ entries = {"main"}
+
+ seen: set[str] = set()
+ frontier, depth = entries, 0
+ while frontier and depth < 4:
+ seen |= frontier
+ nxt: set[str] = set()
+ for name in frontier:
+ nxt |= ({n.id for n in ast.walk(funcs[name]) if isinstance(n, ast.Name)} & set(funcs)) - seen
+ frontier, depth = nxt, depth + 1
+ return [funcs[name] for name in sorted(seen)]
+
+
+def _findings_exit(tree: ast.Module) -> str | None:
+ """The guard that turns accumulated findings into an exit status."""
+ accumulators = _accumulators(tree)
+ if not accumulators:
+ return None
+ for function in _exit_path(tree):
+ for node in ast.walk(function):
+ if isinstance(node, ast.IfExp) and (_is_int(node.body, zero=False) or _is_int(node.orelse, zero=False)):
+ if _referenced(node.test) & accumulators:
+ return ast.unparse(node)[:72]
+ continue
+ status = _exit_status(node)
+ if status is None or _inside_except(node):
+ continue
+ nonzero = _is_int(status, zero=False)
+ if not nonzero and not _is_int(status, zero=True):
+ continue
+ parent = getattr(node, "parent", None)
+ while parent is not None:
+ if isinstance(parent, ast.If) and (_referenced(parent.test) & accumulators):
+ # Findings present -> fail. The inverse (`if not specs:
+ # return 1`) is an abort on an empty read: the script
+ # could not do its job, which is not a verdict on the tree.
+ if nonzero != _negated(parent.test, accumulators):
+ return ast.unparse(parent.test)[:72]
+ break
+ parent = getattr(parent, "parent", None)
+ return None
+
+
+def classify_source(source: str) -> dict[str, str]:
+ """Structural signals that make a script a check. Empty means it is not."""
+ try:
+ tree = _with_parents(ast.parse(source))
+ except SyntaxError:
+ return {}
+ if not _INSPECTS_TREE.search(source):
+ return {}
+ signals: dict[str, str] = {}
+ if flags := _verdict_flags(tree):
+ signals["verdict-flag"] = " ".join(flags)
+ if guard := _findings_exit(tree):
+ signals["findings-exit"] = guard
+ return signals
-def collect_checks(root: Path) -> list[str]:
+def collect_checks(root: Path, gating: dict[str, str] | None = None) -> dict[str, dict[str, str]]:
+ """script name -> the structural signals that classify it as a check."""
scripts = root / SCRIPTS_REL
if not scripts.is_dir():
raise GateError(f"no scripts directory at {scripts}")
- found = sorted(p.name for p in scripts.glob("*.py") if is_check(p.name))
+ found: dict[str, dict[str, str]] = {}
+ for path in sorted(scripts.glob("*.py")):
+ signals = classify_source(path.read_text(encoding="utf-8", errors="replace"))
+ if gating and path.name in gating:
+ signals["gates-a-workflow"] = gating[path.name]
+ if signals:
+ found[path.name] = signals
if not found:
raise GateError(f"parsed zero check scripts from {scripts} — refusing to report a clean tree from an empty read")
return found
@@ -161,12 +405,38 @@ def _command_text(step: dict) -> str:
return "\n".join(parts)
+_NEEDS_OUTPUT = re.compile(r"needs\.([A-Za-z0-9_-]+)\.outputs")
+_SCRIPT_IN_JOB = re.compile(r"scripts/([\w.-]+\.py)")
+
+
+def _gating_jobs(doc: dict, workflow: str) -> dict[str, str]:
+ """script -> "workflow:job" for jobs whose output another job branches on.
+
+ A workflow that merely runs a script is not evidence the script is a gate.
+ A workflow that publishes a job output and makes a downstream job
+ conditional on it has, structurally, asked the script for a verdict.
+ """
+ jobs = doc.get("jobs") or {}
+ branched: set[str] = set()
+ for job in jobs.values():
+ if isinstance(job, dict):
+ branched.update(_NEEDS_OUTPUT.findall(yaml.safe_dump(job, default_flow_style=False)))
+ out: dict[str, str] = {}
+ for job_id, job in jobs.items():
+ if job_id not in branched or not isinstance(job, dict) or not job.get("outputs"):
+ continue
+ for name in _SCRIPT_IN_JOB.findall(yaml.safe_dump(job, default_flow_style=False)):
+ out.setdefault(name, f"{workflow}:{job_id}")
+ return out
+
+
class Surface:
"""The text a workflow executes, with working directories resolved."""
def __init__(self, root: Path):
self.root = root
self.by_workflow: dict[str, list[tuple[str, str]]] = {}
+ self.gating: dict[str, str] = {}
wf_dir = root / WORKFLOWS_REL
if not wf_dir.is_dir():
raise GateError(f"no workflows directory at {wf_dir}")
@@ -186,11 +456,28 @@ class Surface:
for concrete in _expand_matrix(text, matrix):
rows.append((concrete_wd.strip().strip("./"), concrete))
self.by_workflow[path.name] = rows
+ for name, origin in _gating_jobs(doc, path.name).items():
+ self.gating.setdefault(name, origin)
self.workflow_count = len(files)
def texts(self) -> list[tuple[str, str, str]]:
return [(wf, wd, text) for wf, rows in self.by_workflow.items() for wd, text in rows]
+ def invoked_with_verdict_flag(self) -> dict[str, str]:
+ """script -> the workflow that runs it with a verdict flag.
+
+ The other direction on the classifier itself. If a workflow asks a
+ script for a verdict and the classifier does not call it a check, the
+ classifier has a blind spot — which is the defect this file is about.
+ """
+ flags = "|".join(re.escape(f) for f in (*_VERDICT_FLAGS, *_VERDICT_PREFIXES))
+ pattern = re.compile(rf"scripts/([\w.-]+\.py)[^\n;&|]*?\s(?:{flags})")
+ out: dict[str, str] = {}
+ for workflow, _wd, text in self.texts():
+ for name in pattern.findall(text):
+ out.setdefault(name, workflow)
+ return out
+
# --------------------------------------------------------------------------
# Reachability
@@ -339,10 +626,11 @@ def missing_script_paths(root: Path, surface: Surface) -> list[tuple[str, str]]:
def evaluate(
- checks: list[str],
+ checks: dict[str, dict[str, str]],
routes: dict[str, list[str]],
dangling: list[tuple[str, str]],
ratchet: dict[str, str],
+ verdict_invocations: dict[str, str] | None = None,
) -> list[tuple[str, str]]:
failures: list[tuple[str, str]] = []
@@ -352,9 +640,9 @@ def evaluate(
failures.append(
(
"check-unreachable",
- f"{name} is a check and no workflow reaches it, directly or through make, "
- "another script or a test suite CI runs. A gate that cannot fail is "
- "indistinguishable from no gate, while its presence implies coverage.",
+ f"{name} is a check ({', '.join(sorted(checks[name]))}) and no workflow reaches it, "
+ "directly or through make, another script or a test suite CI runs. A gate that cannot "
+ "fail is indistinguishable from no gate, while its presence implies coverage.",
)
)
if reached and name in ratchet:
@@ -371,18 +659,33 @@ def evaluate(
for wf, rel in dangling:
failures.append(("workflow-path-missing", f"{wf} names {rel}, which does not exist — the step cannot do what it says"))
+ # CLASSIFIER -> WORKFLOW. The classifier read in the other direction: a
+ # script CI runs with `--check` is being asked for a verdict, so anything
+ # the classifier leaves out is a blind spot in the classifier, not a
+ # script that stopped being a gate.
+ for name, workflow in sorted((verdict_invocations or {}).items()):
+ if name not in checks:
+ failures.append(
+ (
+ "classifier-blind-spot",
+ f"{workflow} runs {name} with a verdict flag, but no structural signal classifies it as a check — "
+ "it would not be inventoried, and losing its workflow step would go unnoticed",
+ )
+ )
+
return failures
def load(root: Path) -> dict:
surface = Surface(root)
- checks = collect_checks(root)
+ checks = collect_checks(root, surface.gating)
routes, _, corpus = resolve(root, surface)
return {
"checks": checks,
"routes": routes,
"dangling": missing_script_paths(root, surface),
"corpus": corpus,
+ "verdict_invocations": surface.invoked_with_verdict_flag(),
}
@@ -405,7 +708,7 @@ def main(argv: list[str] | None = None) -> int:
return 2
checks, routes, corpus = data["checks"], data["routes"], data["corpus"]
- failures = evaluate(checks, routes, data["dangling"], KNOWN_UNREACHED)
+ failures = evaluate(checks, routes, data["dangling"], KNOWN_UNREACHED, data["verdict_invocations"])
if args.json:
print(
@@ -413,9 +716,10 @@ def main(argv: list[str] | None = None) -> int:
{
"repo_root": str(root),
"corpus": corpus,
- "checks": {name: routes.get(name, []) for name in checks},
+ "checks": {name: {"signals": checks[name], "routes": routes.get(name, [])} for name in checks},
"dangling_workflow_paths": [{"workflow": w, "path": p} for w, p in data["dangling"]],
"known_unreached": KNOWN_UNREACHED,
+ "verdict_invocations": data["verdict_invocations"],
"failures": [{"code": c, "detail": d} for c, d in failures],
},
indent=2,
@@ -424,11 +728,15 @@ def main(argv: list[str] | None = None) -> int:
)
return 1 if failures else 0
+ by_signal = Counter(signal for signals in checks.values() for signal in signals)
+ intrinsic = sum(1 for s in checks.values() if set(s) - {"gates-a-workflow"})
print(f"repo root {root}")
print(f"workflows {WORKFLOWS_REL} ({corpus['workflows']} files)")
print(f"scripts {SCRIPTS_REL} ({corpus['scripts']} python files, {len(checks)} of them checks)")
print(f"make recipes {MAKEFILE_REL} ({corpus['make_recipes']} targets)")
print(f"reachable nodes {corpus['reachable_nodes']} (workflows + make recipes + CI-collected tests + scripts they reach)")
+ print("classified by " + ", ".join(f"{signal} {count}" for signal, count in sorted(by_signal.items())))
+ print(f" {intrinsic} of {len(checks)} from an intrinsic signal, so they are inventoried with or without a caller")
print(f"ratchet {len(KNOWN_UNREACHED)} check(s) deliberately unwired")
print()
@@ -437,7 +745,8 @@ def main(argv: list[str] | None = None) -> int:
for name in checks:
hits = routes.get(name, [])
print(f" {name:{width}} {hits[0] if hits else '— NO WORKFLOW —'}")
- for extra in hits[1:4]:
+ print(f" {'':{width}} signals: {', '.join(sorted(checks[name]))}")
+ for extra in hits[1:3]:
print(f" {'':{width}} {extra}")
print()
@@ -462,24 +771,25 @@ def self_test(root: Path) -> int:
print(f"self-test: cannot read the tree: {exc}", file=sys.stderr)
return 2
- clean = evaluate(base["checks"], base["routes"], base["dangling"], KNOWN_UNREACHED)
+ clean = evaluate(base["checks"], base["routes"], base["dangling"], KNOWN_UNREACHED, base["verdict_invocations"])
if clean:
print("self-test: the unmodified tree already fails; fix that first", file=sys.stderr)
for code, detail in clean:
print(f" [{code}] {detail}", file=sys.stderr)
return 1
- def case(checks=None, routes=None, dangling=None, ratchet=None):
+ def case(checks=None, routes=None, dangling=None, ratchet=None, invocations=None):
return (
checks if checks is not None else base["checks"],
routes if routes is not None else base["routes"],
dangling if dangling is not None else base["dangling"],
ratchet if ratchet is not None else KNOWN_UNREACHED,
+ invocations if invocations is not None else base["verdict_invocations"],
)
- orphan_checks = [*base["checks"], "check_brand_new_thing.py"]
- orphan_routes = {**base["routes"], "check_brand_new_thing.py": []}
-
+ orphan = "check_brand_new_thing.py"
+ orphan_checks = {**base["checks"], orphan: {"verdict-flag": "--check"}}
+ orphan_routes = {**base["routes"], orphan: []}
reached_name = next(n for n in base["checks"] if base["routes"].get(n))
cases: list[tuple[str, str, tuple]] = [
@@ -508,6 +818,11 @@ def self_test(root: Path) -> int:
"ratchet-names-nothing",
case(ratchet={"check_ghost.py": "example"}),
),
+ (
+ "CLASSIFIER -> WORKFLOW: CI asks a script for a verdict the classifier does not inventory",
+ "classifier-blind-spot",
+ case(invocations={**base["verdict_invocations"], "some_unclassified_thing.py": "ci.yml"}),
+ ),
]
print(f"self-test against {root}")
@@ -520,6 +835,53 @@ def self_test(root: Path) -> int:
print(f" {'PASS' if caught else 'FAIL'} {description}")
print(f" expected [{expected}] got {sorted(codes) or 'nothing'}")
+ # The classifier itself. The defect this replaced was a filename test, so
+ # the case that matters is a gate whose name announces nothing: a
+ # structural classifier must find it, and the old one could not.
+ planted = '''\
+"""A gate that no naming convention would reveal."""
+import sys
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parent.parent
+
+def main() -> int:
+ offenders = [p.name for p in ROOT.glob("*.md") if "\\t" in p.read_text(errors="replace")]
+ if offenders:
+ print("tabs in markdown:", offenders)
+ return 1
+ return 0
+
+sys.exit(main())
+'''
+ signals = classify_source(planted)
+ print(f" {'PASS' if signals else 'FAIL'} CLASSIFIER: a gate planted under an unconventional name is still found")
+ print(f" `frobnicate_widgets.py` matches no check_/validate_/audit_ prefix; signals {sorted(signals) or 'none'}")
+ ok &= bool(signals)
+
+ # And the inverse, because a classifier that said yes to everything would
+ # pass the case above: a generator that aborts on an empty read is not a
+ # gate, and the polarity of its guard is the only thing that says so.
+ generator = '''\
+"""Writes a document. Not a gate."""
+import sys
+from pathlib import Path
+
+def main() -> int:
+ specs = [p for p in Path("specs").glob("*.yaml")]
+ if not specs:
+ print("nothing to render")
+ return 1
+ Path("out.md").write_text("\\n".join(p.name for p in specs))
+ return 0
+
+sys.exit(main())
+'''
+ rejected = not classify_source(generator)
+ print(f" {'PASS' if rejected else 'FAIL'} CLASSIFIER: a generator that aborts on an empty read is not a check")
+ print(f" `if not specs: return 1` is an abort, not a finding; signals {sorted(classify_source(generator)) or 'none'}")
+ ok &= rejected
+
# The reachability resolver itself, not just the rules on top of it: a
# resolver that returned "reached" for everything would pass every case
# above. This asserts it discriminates on the real tree.
diff --git a/scripts/competitor_names.toml b/scripts/competitor_names.toml
index ead4a2bf6..cb1dea6d7 100644
--- a/scripts/competitor_names.toml
+++ b/scripts/competitor_names.toml
@@ -130,6 +130,11 @@ paths = ["marketplace/index.json", "apps/web/public/marketplace/index.json"]
names = ["Torq", "Tines", "Wazuh"]
reason = "Generated marketplace catalog; entries are the connector manifests above."
+[[allow]]
+paths = ["packages/types/src/generated/connector-types.ts", "packages/types/src/generated/connector-types.json"]
+names = ["Torq", "Tines", "Wazuh"]
+reason = "The console's connector_type vocabulary, generated from the connector registry by scripts/generate_connector_types.py. Each name here is a connector id the platform ingests under; removing one would make the type reject events from a connector that ships."
+
[[allow]]
paths = ["apps/docs/sidebars.ts"]
names = ["Torq", "Tines", "Wazuh"]
diff --git a/scripts/generate_connector_types.py b/scripts/generate_connector_types.py
new file mode 100644
index 000000000..370cc42da
--- /dev/null
+++ b/scripts/generate_connector_types.py
@@ -0,0 +1,541 @@
+#!/usr/bin/env python3
+"""Generate the console's ``ConnectorType`` union from the connector registry.
+
+Why this exists
+---------------
+``packages/types/src/connector.ts`` carried a hand-written union of
+``connector_type`` values. Hand-maintenance is what let it drift: ten members
+named nothing the platform could ingest, and ``ibm_qradar`` was the expensive
+one — no connector declared it and no profile was keyed on it, so strict mode
+rejected those events and lenient mode minted a vendor called "ibm_qradar", a
+second alert source for the same QRadar deployment ``qradar`` already fed.
+
+PR #811 corrected the members. It did not remove the mechanism, so this does:
+the union is now derived from the registry that decides what the platform can
+ingest, and ``--check`` fails when the committed file no longer matches.
+
+What the union means
+--------------------
+"Every ``connector_type`` the ingest normalizer resolves", which is exactly
+three sets, each read from its own source of truth:
+
+ registry ``_CONNECTOR_CLASSES`` in
+ ``services/connectors/app/connectors/__init__.py``, resolved
+ to ``cls.connector_id`` the same way ``_build_registry()``
+ does — through the class, never through the filename.
+ profile ``connectorProfiles`` keys in
+ ``services/ingest/internal/normalizer/normalizer.go``. Some
+ are longer legacy spellings a declared id reaches through
+ ``connectorTypeAliases`` (``crowdstrike`` ->
+ ``crowdstrike_falcon``); ``splunk_enterprise`` is reachable
+ only because the union carries it, which is why dropping it
+ would fail ``check_connector_profiles.py``.
+ canonical-fold ``connectorTypeCanonical`` sources — alternate spellings the
+ normalizer folds onto a declared id before anything keys off
+ it. Generation must preserve these, not flatten them away:
+ the console still emits ``ibm_qradar`` and the fold is what
+ makes it arrive as ``qradar``.
+
+That set is, by construction, the ``resolvable`` set
+``scripts/check_connector_profiles.py`` computes, so the two gates now agree
+on one definition instead of two.
+
+Nothing here is positional
+--------------------------
+``scripts/generate_detections.py`` was not idempotent because it assigned
+``det-{category}-{idx:03d}`` by position, so inserting anywhere but the end
+moved every later id; it needed a committed lock file. This generator assigns
+no identity at all. A member *is* its ``connector_id``, taken from the class,
+and the output is a sorted set — so reordering the registry cannot change a
+byte of the output and no lock file is required. ``tests/`` proves both
+properties rather than asserting them here.
+
+Identity comes from the class, not the filename
+-----------------------------------------------
+``scripts/generate_connector_docs.py`` derived a connector's identity from its
+filename and produced thin duplicate pages for the six connectors whose files
+are spelled with underscores, while the coverage gate reported 100%. Two
+connectors in this tree would break the same way: ``jira_connector.py``
+declares ``jira`` and ``tenable.py`` declares ``tenable_io``. This generator
+reads ``connector_id`` off the class, which is what ``CONNECTOR_REGISTRY`` is
+keyed by, and refuses a class with an empty or duplicate id exactly as
+``_build_registry()`` does.
+
+Usage
+-----
+ python3 scripts/generate_connector_types.py # write outputs
+ python3 scripts/generate_connector_types.py --check # fail on drift
+ python3 scripts/generate_connector_types.py --json
+ python3 scripts/generate_connector_types.py --self-test
+
+``--repo-root`` overrides the tree under inspection. The resolved root, every
+file read and every count are printed before the verdict, and an empty parse
+is a hard error rather than a quiet zero: a generator that reports a clean
+tree it never opened is worse than no generator.
+"""
+
+from __future__ import annotations
+
+import argparse
+import ast
+import json
+import re
+import sys
+from pathlib import Path
+
+REGISTRY_REL = Path("services/connectors/app/connectors/__init__.py")
+CONNECTORS_REL = Path("services/connectors/app/connectors")
+NORMALIZER_REL = Path("services/ingest/internal/normalizer/normalizer.go")
+TS_OUT_REL = Path("packages/types/src/generated/connector-types.ts")
+JSON_OUT_REL = Path("packages/types/src/generated/connector-types.json")
+CONSUMER_REL = Path("packages/types/src/connector.ts")
+
+#: Where each union member comes from, in the order a member is attributed to
+#: the first source that claims it.
+ORIGIN_REGISTRY = "registry"
+ORIGIN_PROFILE = "profile"
+ORIGIN_FOLD = "canonical-fold"
+
+
+class GateError(RuntimeError):
+ """An input could not be read. Never downgraded to a passing result."""
+
+
+# --------------------------------------------------------------------------
+# Registry (Python, via AST — no runtime import of the connectors package)
+# --------------------------------------------------------------------------
+def _registered_class_names(tree: ast.Module) -> list[str]:
+ for node in tree.body:
+ value = None
+ if isinstance(node, ast.AnnAssign) and getattr(node.target, "id", None) == "_CONNECTOR_CLASSES":
+ value = node.value
+ elif isinstance(node, ast.Assign) and any(getattr(t, "id", None) == "_CONNECTOR_CLASSES" for t in node.targets):
+ value = node.value
+ if isinstance(value, ast.Tuple):
+ names = [e.id for e in value.elts if isinstance(e, ast.Name)]
+ if len(names) != len(value.elts):
+ raise GateError("_CONNECTOR_CLASSES contains an entry that is not a bare class name")
+ return names
+ raise GateError(f"could not find the _CONNECTOR_CLASSES tuple in {REGISTRY_REL}")
+
+
+def _import_modules(tree: ast.Module) -> dict[str, str]:
+ """Imported class name -> the module in app.connectors it comes from."""
+ out: dict[str, str] = {}
+ for node in ast.walk(tree):
+ if isinstance(node, ast.ImportFrom) and node.module and node.module.startswith("app.connectors."):
+ for alias in node.names:
+ out[alias.asname or alias.name] = node.module.split(".")[-1]
+ return out
+
+
+def _class_attribute(tree: ast.Module, class_name: str, attribute: str) -> str | None:
+ for cls in (n for n in ast.walk(tree) if isinstance(n, ast.ClassDef) and n.name == class_name):
+ for stmt in cls.body:
+ target = None
+ if isinstance(stmt, ast.Assign) and len(stmt.targets) == 1:
+ target = getattr(stmt.targets[0], "id", None)
+ elif isinstance(stmt, ast.AnnAssign):
+ target = getattr(stmt.target, "id", None)
+ if target == attribute and isinstance(stmt.value, ast.Constant) and isinstance(stmt.value.value, str):
+ return stmt.value.value
+ return None
+
+
+def parse_registry(root: Path) -> dict[str, dict[str, str]]:
+ """connector_id -> {class, module, name}, resolved as _build_registry() does."""
+ registry_file = root / REGISTRY_REL
+ connectors_dir = root / CONNECTORS_REL
+ if not registry_file.exists():
+ raise GateError(f"expected input does not exist: {registry_file}")
+
+ init_tree = ast.parse(registry_file.read_text(encoding="utf-8"))
+ modules = _import_modules(init_tree)
+ out: dict[str, dict[str, str]] = {}
+ for class_name in _registered_class_names(init_tree):
+ module = modules.get(class_name)
+ if module is None:
+ raise GateError(f"{class_name} is in _CONNECTOR_CLASSES but is not imported from app.connectors.*")
+ path = connectors_dir / f"{module}.py"
+ if not path.exists():
+ raise GateError(f"{class_name} is imported from app.connectors.{module}, which does not exist")
+ tree = ast.parse(path.read_text(encoding="utf-8"))
+ connector_id = _class_attribute(tree, class_name, "connector_id")
+ # The same two refusals _build_registry() makes, for the same reasons.
+ if not connector_id:
+ raise GateError(f"connector class {class_name} has an empty connector_id; refusing to register")
+ if connector_id in out:
+ raise GateError(f"duplicate connector_id {connector_id!r} between {out[connector_id]['class']} and {class_name}")
+ out[connector_id] = {
+ "class": class_name,
+ "module": f"{module}.py",
+ "name": _class_attribute(tree, class_name, "connector_name") or "",
+ }
+ if not out:
+ raise GateError("parsed zero connector ids — refusing to generate from an empty read")
+ return out
+
+
+# --------------------------------------------------------------------------
+# Normalizer (Go, via brace-balanced literals)
+# --------------------------------------------------------------------------
+def _go_block(src: str, header: str) -> str:
+ start = src.find(header)
+ if start == -1:
+ raise GateError(f"could not find `{header}` in {NORMALIZER_REL}")
+ depth, i = 0, src.index("{", start)
+ for j in range(i, len(src)):
+ if src[j] == "{":
+ depth += 1
+ elif src[j] == "}":
+ depth -= 1
+ if depth == 0:
+ return src[i : j + 1]
+ raise GateError(f"unbalanced braces after `{header}` in {NORMALIZER_REL}")
+
+
+def parse_normalizer(root: Path) -> tuple[set[str], dict[str, str], dict[str, str]]:
+ """(profile keys, connectorTypeAliases, connectorTypeCanonical)."""
+ path = root / NORMALIZER_REL
+ if not path.exists():
+ raise GateError(f"expected input does not exist: {path}")
+ src = path.read_text(encoding="utf-8")
+
+ profiles = set(re.findall(r'^\t"([^"]+)":\s*\{', _go_block(src, "var connectorProfiles = map[string]connectorProfile{"), re.MULTILINE))
+ if not profiles:
+ raise GateError("parsed zero connectorProfiles keys — refusing to generate from an empty read")
+
+ def pairs(header: str) -> dict[str, str]:
+ if header not in src:
+ return {}
+ return dict(re.findall(r'"([^"]+)":\s*"([^"]+)"', _go_block(src, header)))
+
+ aliases = pairs("var connectorTypeAliases = map[string]string{")
+ canonical = pairs("var connectorTypeCanonical = map[string]string{")
+ return profiles, aliases, canonical
+
+
+# --------------------------------------------------------------------------
+# Payload
+# --------------------------------------------------------------------------
+def build_payload(registry: dict[str, dict[str, str]], profiles: set[str], aliases: dict[str, str], canonical: dict[str, str]) -> dict:
+ """The union and its provenance. A pure function of the three inputs."""
+ origins: dict[str, str] = {}
+ for connector_id in registry:
+ origins[connector_id] = ORIGIN_REGISTRY
+ for key in sorted(profiles):
+ origins.setdefault(key, ORIGIN_PROFILE)
+ for alternate in sorted(canonical):
+ origins.setdefault(alternate, ORIGIN_FOLD)
+
+ members = sorted(origins)
+ return {
+ "members": members,
+ "origins": {m: origins[m] for m in members},
+ "canonical": dict(sorted(canonical.items())),
+ "counts": {
+ "total": len(members),
+ ORIGIN_REGISTRY: sum(1 for m in members if origins[m] == ORIGIN_REGISTRY),
+ ORIGIN_PROFILE: sum(1 for m in members if origins[m] == ORIGIN_PROFILE),
+ ORIGIN_FOLD: sum(1 for m in members if origins[m] == ORIGIN_FOLD),
+ "aliases": len(aliases),
+ },
+ "generatedFrom": [str(REGISTRY_REL), str(NORMALIZER_REL)],
+ "regenerateWith": "python3 scripts/generate_connector_types.py",
+ }
+
+
+def render_json(payload: dict) -> str:
+ return json.dumps(payload, indent=2, sort_keys=True) + "\n"
+
+
+def render_typescript(payload: dict) -> str:
+ members = payload["members"]
+ origins = payload["origins"]
+ canonical = payload["canonical"]
+ counts = payload["counts"]
+
+ lines: list[str] = [
+ "// AUTO-GENERATED by scripts/generate_connector_types.py. Do not edit by hand.",
+ "// Run `python3 scripts/generate_connector_types.py` and commit the result.",
+ "//",
+ "// Every `connector_type` the ingest normalizer resolves, derived from the",
+ "// registry that decides what the platform can ingest:",
+ "//",
+ f"// {counts[ORIGIN_REGISTRY]:>3} connector ids declared in services/connectors (_CONNECTOR_CLASSES)",
+ f"// {counts[ORIGIN_PROFILE]:>3} connectorProfiles keys no connector declares under that spelling",
+ f"// {counts[ORIGIN_FOLD]:>3} alternate spellings connectorTypeCanonical folds onto a declared id",
+ f"// {counts['total']:>3} total",
+ "//",
+ "// The union used to be written by hand, which is how ten members came to name",
+ "// nothing: `ibm_qradar` reached no profile and no connector, so strict mode",
+ '// rejected those events and lenient mode minted a vendor called "ibm_qradar" —',
+ "// a second alert source for the QRadar deployment `qradar` already fed.",
+ "",
+ "/** Every `connector_type` value the ingest normalizer resolves. */",
+ "export const CONNECTOR_TYPES = [",
+ ]
+ width = max(len(m) for m in members) + 2
+ for member in members:
+ literal = f'"{member}",'
+ lines.append(f" {literal:<{width + 1}} // {origins[member]}")
+ lines += [
+ "] as const;",
+ "",
+ "export type ConnectorType = (typeof CONNECTOR_TYPES)[number];",
+ "",
+ "/**",
+ " * Alternate spellings the normalizer folds onto the id services/connectors",
+ " * declares, before anything keys off it. Mirrors `connectorTypeCanonical` in",
+ " * services/ingest/internal/normalizer/normalizer.go so the console can resolve",
+ " * a stored value to one product name instead of two.",
+ " */",
+ "export const CONNECTOR_TYPE_CANONICAL: Readonly<Record<string, ConnectorType>> = {",
+ ]
+ for alternate, target in canonical.items():
+ lines.append(f' "{alternate}": "{target}",')
+ lines += [
+ "};",
+ "",
+ "/** Resolve an alternate spelling to the declared connector id. */",
+ "export function canonicalConnectorType(connectorType: string): string {",
+ " return CONNECTOR_TYPE_CANONICAL[connectorType] ?? connectorType;",
+ "}",
+ "",
+ ]
+ return "\n".join(lines)
+
+
+# --------------------------------------------------------------------------
+# Gate
+# --------------------------------------------------------------------------
+#: The consumer must re-export the generated union rather than redeclare it.
+#: Without this rule the generated file can sit in the tree, current and
+#: ignored, while a hand-written union three directories away is what
+#: TypeScript actually resolves — the drift this change exists to end.
+_REDECLARATION = re.compile(r"^\s*export\s+type\s+ConnectorType\s*=", re.MULTILINE)
+_REEXPORT = re.compile(r"""export\s+type\s*\{[^}]*\bConnectorType\b[^}]*\}\s*from\s*["'][^"']*generated/connector-types["']""")
+
+
+def evaluate(payload: dict, on_disk_ts: str | None, on_disk_json: str | None, consumer: str | None) -> list[tuple[str, str]]:
+ failures: list[tuple[str, str]] = []
+
+ if on_disk_ts is None:
+ failures.append(("output-missing", f"{TS_OUT_REL} does not exist; run the generator"))
+ elif on_disk_ts != render_typescript(payload):
+ failures.append(("output-drifted", f"{TS_OUT_REL} is not what generation produces; run the generator and commit the result"))
+
+ if on_disk_json is None:
+ failures.append(("output-missing", f"{JSON_OUT_REL} does not exist; run the generator"))
+ elif on_disk_json != render_json(payload):
+ failures.append(("output-drifted", f"{JSON_OUT_REL} is not what generation produces; run the generator and commit the result"))
+
+ if consumer is None:
+ failures.append(("consumer-missing", f"{CONSUMER_REL} does not exist"))
+ else:
+ if _REDECLARATION.search(consumer):
+ failures.append(
+ (
+ "consumer-redeclares",
+ f"{CONSUMER_REL} declares its own ConnectorType union; the generated one would be ignored "
+ "and the hand-maintained list is the mechanism that drifted",
+ )
+ )
+ if not _REEXPORT.search(consumer):
+ failures.append(("consumer-not-reexporting", f"{CONSUMER_REL} does not re-export ConnectorType from {TS_OUT_REL.stem}"))
+
+ # The folds are the part generation is most likely to flatten away, so
+ # they are checked against the rendered output rather than the payload.
+ rendered = on_disk_ts or ""
+ for alternate, target in payload["canonical"].items():
+ if alternate not in payload["members"]:
+ failures.append(
+ (
+ "fold-source-dropped",
+ f"{alternate!r} folds onto {target!r} but is not a union member; the console's spelling would not type-check",
+ )
+ )
+ if target not in payload["members"]:
+ failures.append(("fold-target-dropped", f"{alternate!r} folds onto {target!r}, which is not a union member"))
+ if on_disk_ts is not None and f'"{alternate}": "{target}"' not in rendered:
+ failures.append(("fold-not-emitted", f"the {alternate!r} -> {target!r} fold is missing from {TS_OUT_REL}"))
+ return failures
+
+
+def load(root: Path) -> dict:
+ registry = parse_registry(root)
+ profiles, aliases, canonical = parse_normalizer(root)
+ return {"registry": registry, "profiles": profiles, "aliases": aliases, "canonical": canonical}
+
+
+def _read(path: Path) -> str | None:
+ return path.read_text(encoding="utf-8") if path.exists() else None
+
+
+def main(argv: list[str] | None = None) -> int:
+ parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+ parser.add_argument("--repo-root", type=Path, default=Path(__file__).resolve().parent.parent)
+ parser.add_argument("--check", action="store_true", help="fail (exit 1) on drift instead of rewriting the outputs")
+ parser.add_argument("--json", action="store_true")
+ parser.add_argument("--self-test", action="store_true", help="prove the gate detects injected drift in each direction")
+ args = parser.parse_args(argv)
+
+ root = args.repo_root.resolve()
+ if args.self_test:
+ return self_test(root)
+
+ try:
+ inputs = load(root)
+ except GateError as exc:
+ print(f"generate_connector_types: FAILED to read the tree: {exc}", file=sys.stderr)
+ return 2
+
+ payload = build_payload(**inputs)
+ ts_path, json_path = root / TS_OUT_REL, root / JSON_OUT_REL
+
+ if not args.check:
+ ts_path.parent.mkdir(parents=True, exist_ok=True)
+ ts_path.write_text(render_typescript(payload), encoding="utf-8")
+ json_path.write_text(render_json(payload), encoding="utf-8")
+
+ failures = evaluate(payload, _read(ts_path), _read(json_path), _read(root / CONSUMER_REL))
+
+ if args.json:
+ print(
+ json.dumps(
+ {"repo_root": str(root), **payload, "failures": [{"code": c, "detail": d} for c, d in failures]}, indent=2, sort_keys=True
+ )
+ )
+ return 1 if failures else 0
+
+ counts = payload["counts"]
+ print(f"repo root {root}")
+ print(f"registry {REGISTRY_REL} ({len(inputs['registry'])} declared connector ids)")
+ print(
+ f"normalizer {NORMALIZER_REL} ({len(inputs['profiles'])} profiles, "
+ f"{counts['aliases']} type aliases, {len(inputs['canonical'])} canonical folds)"
+ )
+ print(f"outputs {TS_OUT_REL}")
+ print(f" {JSON_OUT_REL}")
+ print(f"consumer {CONSUMER_REL}")
+ print()
+ print(
+ f"union {counts['total']} members = {counts[ORIGIN_REGISTRY]} registry "
+ f"+ {counts[ORIGIN_PROFILE]} profile + {counts[ORIGIN_FOLD]} canonical-fold"
+ )
+ print(f"folds preserved {', '.join(f'{a} -> {t}' for a, t in payload['canonical'].items())}")
+ print()
+
+ if failures:
+ print(f"FAIL — {len(failures)} finding(s):")
+ for code, detail in failures:
+ print(f" [{code}] {detail}")
+ return 1
+ print("OK — the committed union is what the registry, the profiles and the canonical")
+ print(" folds generate, and the console re-exports it instead of redeclaring it.")
+ return 0
+
+
+# --------------------------------------------------------------------------
+# Self-test
+# --------------------------------------------------------------------------
+def self_test(root: Path) -> int:
+ """Inject drift in each direction and require --check to catch each one."""
+ try:
+ inputs = load(root)
+ except GateError as exc:
+ print(f"self-test: cannot read the tree: {exc}", file=sys.stderr)
+ return 2
+
+ payload = build_payload(**inputs)
+ good_ts, good_json = render_typescript(payload), render_json(payload)
+ consumer = _read(root / CONSUMER_REL) or ""
+
+ clean = evaluate(payload, good_ts, good_json, consumer)
+ if clean:
+ print("self-test: the unmodified tree already fails; fix that first", file=sys.stderr)
+ for code, detail in clean:
+ print(f" [{code}] {detail}", file=sys.stderr)
+ return 1
+
+ def drop_member(name: str) -> dict:
+ reduced = dict(payload.items())
+ reduced["members"] = [m for m in payload["members"] if m != name]
+ return reduced
+
+ hand_edited_ts = good_ts.replace(' "qualys",', "", 1)
+ a_connector, a_fold = next(iter(inputs["registry"])), next(iter(inputs["canonical"]))
+
+ cases: list[tuple[str, str, tuple]] = [
+ (
+ "a hand edit removes a union member from the generated TypeScript",
+ "output-drifted",
+ (payload, hand_edited_ts, good_json, consumer),
+ ),
+ (
+ "a hand edit changes the generated JSON",
+ "output-drifted",
+ (payload, good_ts, good_json.replace('"total"', '"totl"', 1), consumer),
+ ),
+ (
+ "the generated TypeScript is deleted",
+ "output-missing",
+ (payload, None, good_json, consumer),
+ ),
+ (
+ "the console goes back to declaring its own union",
+ "consumer-redeclares",
+ (payload, good_ts, good_json, consumer + '\nexport type ConnectorType = "a" | "b";\n'),
+ ),
+ (
+ "the console stops re-exporting the generated union",
+ "consumer-not-reexporting",
+ (payload, good_ts, good_json, _REEXPORT.sub("", consumer)),
+ ),
+ (
+ "a canonical fold is flattened out of the union",
+ "fold-source-dropped",
+ (drop_member(a_fold), good_ts, good_json, consumer),
+ ),
+ (
+ "a fold's target leaves the registry",
+ "fold-target-dropped",
+ (drop_member(inputs["canonical"][a_fold]), good_ts, good_json, consumer),
+ ),
+ (
+ "the fold map is dropped from the rendered TypeScript",
+ "fold-not-emitted",
+ (payload, re.sub(r'^\s+"[a-z_]+": "[a-z_]+",$', "", good_ts, flags=re.MULTILINE), good_json, consumer),
+ ),
+ ]
+
+ print(f"self-test against {root}")
+ print(f"clean tree: {payload['counts']['total']}-member union, 0 failures (the baseline every case below perturbs)")
+ print(f" a declared id is {a_connector!r}; a folded spelling is {a_fold!r}\n")
+ ok = True
+ for description, expected, call in cases:
+ codes = {code for code, _ in evaluate(*call)}
+ caught = expected in codes
+ ok &= caught
+ print(f" {'PASS' if caught else 'FAIL'} {description}")
+ print(f" expected [{expected}] got {sorted(codes) or 'nothing'}")
+
+ # The registry reader itself: identity must come from the class, because
+ # two connectors are declared under an id their filename does not spell.
+ by_filename = {info["module"][:-3] for info in inputs["registry"].values()}
+ mismatched = sorted(cid for cid, info in inputs["registry"].items() if info["module"][:-3] != cid)
+ filename_would_break = bool(mismatched) and set(inputs["registry"]) != by_filename
+ print(f" {'PASS' if filename_would_break else 'FAIL'} IDENTITY: connector_id is read from the class, not the filename")
+ print(f" {len(mismatched)} connector(s) would be misnamed by a filename slug: {', '.join(mismatched) or 'none'}")
+ ok &= filename_would_break
+
+ print()
+ if not ok:
+ print("self-test FAILED: the gate did not catch drift it claims to catch")
+ return 1
+ print(f"self-test OK: {len(cases)} injected defects, each caught by its own code")
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/services/ingest/internal/normalizer/normalizer.go b/services/ingest/internal/normalizer/normalizer.go
index e1df9628f..c550be40f 100644
--- a/services/ingest/internal/normalizer/normalizer.go
+++ b/services/ingest/internal/normalizer/normalizer.go
@@ -447,6 +447,25 @@ var connectorTypeAliases = map[string]string{
// "Syslog / CEF"). Resolution happens once, at the top of Normalize, so the
// profile lookup, the alias map, canonicalClassByConnector and the product
// identity on the canonical path all agree on one name.
+//
+// `palo_alto_cortex` was the one entry that named a vendor rather than a
+// product, and Palo Alto ships two the platform can ingest: `cortex_xdr` and
+// `cortex_xsiam`. It folds onto Cortex XDR, and the record is:
+//
+// - the spelling entered packages/types in the initial-release commit
+// (2026-05-01), before either connector existed — `cortex_xsiam` landed
+// 2026-05-07 and `cortex_xdr` 2026-05-08 — so it was aspirational vendor
+// vocabulary, not a reference to a connector somebody had wired;
+// - outside this map, the union and the changelog, the string appears
+// nowhere in the tree and never has: no console code, no saved connector
+// instance, no seed row, no API catalog entry, no fixture. Nothing
+// emitting it can be misrouted, because nothing emits it;
+// - where the console does name the product it says "Cortex XDR" (the
+// landing catalog lists it under EDR and lists no XSIAM), which matches
+// CortexXDRConnector's own description, "Palo Alto Cortex XDR incidents
+// via the public REST API";
+// - XSIAM is a SIEM reachable under its own id, so nothing is lost: a
+// deployment that means XSIAM says `cortex_xsiam` and always could.
var connectorTypeCanonical = map[string]string{
"google_chronicle": "chronicle",
"ibm_qradar": "qradar",
diff --git a/tests/test_connector_types_generated.py b/tests/test_connector_types_generated.py
new file mode 100644
index 000000000..d7c1fcdcf
--- /dev/null
+++ b/tests/test_connector_types_generated.py
@@ -0,0 +1,225 @@
+"""The console's ConnectorType union must be generated, not maintained.
+
+The union was hand-written, and ten of its members named nothing the platform
+could ingest. ``ibm_qradar`` was the expensive one: no connector declared it
+and no profile was keyed on it, so strict mode rejected those events and
+lenient mode minted a vendor called "ibm_qradar" — a second alert source for
+the QRadar deployment ``qradar`` already fed.
+
+PR #811 corrected the members and left the mechanism. These tests hold the
+three properties that make the mechanism safe to remove:
+
+ * the output is a pure function of the registry, so regeneration is
+ idempotent and reordering the registry cannot change a byte;
+ * a connector's identity comes from ``connector_id`` on the class, the way
+ ``_build_registry()`` resolves it — two connectors in this tree are
+ declared under an id their filename does not spell, so a filename slug
+ would silently misname them (the defect
+ ``scripts/generate_connector_docs.py`` shipped);
+ * the canonical folds survive generation, because the console still emits
+ ``ibm_qradar`` and the fold is what makes it arrive as ``qradar``.
+"""
+
+from __future__ import annotations
+
+import sys
+from pathlib import Path
+
+import pytest
+
+REPO_ROOT = Path(__file__).resolve().parent.parent
+sys.path.insert(0, str(REPO_ROOT / "scripts"))
+
+import generate_connector_types as gct # noqa: E402
+
+TS_OUT = REPO_ROOT / gct.TS_OUT_REL
+JSON_OUT = REPO_ROOT / gct.JSON_OUT_REL
+CONSUMER = REPO_ROOT / gct.CONSUMER_REL
+
+
[email protected](scope="module")
+def inputs() -> dict:
+ return gct.load(REPO_ROOT)
+
+
[email protected](scope="module")
+def payload(inputs: dict) -> dict:
+ return gct.build_payload(**inputs)
+
+
+# --------------------------------------------------------------------------
+# Committed output is current
+# --------------------------------------------------------------------------
+def test_committed_outputs_match_generation(payload: dict) -> None:
+ assert TS_OUT.read_text(encoding="utf-8") == gct.render_typescript(
+ payload
+ ), f"{gct.TS_OUT_REL} is stale. Run `python3 scripts/generate_connector_types.py` and commit the result."
+ assert JSON_OUT.read_text(encoding="utf-8") == gct.render_json(payload)
+
+
+def test_check_mode_passes_on_the_committed_tree() -> None:
+ assert gct.main(["--check", "--repo-root", str(REPO_ROOT)]) == 0
+
+
+def test_self_test_passes() -> None:
+ """The generator's own injected-drift suite, run as part of the test suite.
+
+ A `--self-test` nothing invokes is the same defect one level up.
+ """
+ assert gct.self_test(REPO_ROOT) == 0
+
+
+# --------------------------------------------------------------------------
+# Idempotence, and nothing positional
+# --------------------------------------------------------------------------
+def test_generation_is_idempotent(payload: dict, inputs: dict) -> None:
+ once = gct.render_typescript(payload)
+ twice = gct.render_typescript(gct.build_payload(**gct.load(REPO_ROOT)))
+ assert once == twice
+
+
+def test_registry_order_does_not_change_the_output(payload: dict, inputs: dict) -> None:
+ """Reordering _CONNECTOR_CLASSES must not move a single byte.
+
+ ``scripts/generate_detections.py`` assigned ``det-{category}-{idx:03d}``
+ by position, so inserting anywhere but the end renumbered every later
+ rule and it needed a committed lock file to become stable. This generator
+ assigns no identity — a member *is* its ``connector_id`` — so the same
+ class of bug cannot arise, and this is the assertion that says so.
+ """
+ reversed_registry = dict(reversed(list(inputs["registry"].items())))
+ shuffled = dict(inputs, registry=reversed_registry)
+ assert gct.render_typescript(gct.build_payload(**shuffled)) == gct.render_typescript(payload)
+ assert gct.render_json(gct.build_payload(**shuffled)) == gct.render_json(payload)
+
+
+# --------------------------------------------------------------------------
+# Identity
+# --------------------------------------------------------------------------
+def test_identity_comes_from_the_class_not_the_filename(inputs: dict) -> None:
+ """``jira_connector.py`` declares ``jira``; ``tenable.py`` declares ``tenable_io``.
+
+ A generator that slugged the filename would emit ``jira_connector`` and
+ ``tenable``, neither of which any profile, alias or fold resolves — the
+ exact shape of the ten dead union members, reintroduced mechanically.
+ """
+ registry = inputs["registry"]
+ mismatched = {cid: info["module"] for cid, info in registry.items() if info["module"][:-3] != cid}
+ assert mismatched, "expected at least one connector whose filename does not spell its id"
+ assert mismatched == {"jira": "jira_connector.py", "tenable_io": "tenable.py"}
+ for connector_id in mismatched:
+ assert connector_id in gct.build_payload(**inputs)["members"]
+
+
+def test_registry_matches_the_runtime_registry_key_set(inputs: dict) -> None:
+ """The generator's ids are the keys ``CONNECTOR_REGISTRY`` is built from.
+
+ Resolved independently here — walk the tuple, follow each import, read the
+ class attribute — so this fails if the generator ever starts inferring
+ identity some other way.
+ """
+ import ast
+
+ init = ast.parse((REPO_ROOT / gct.REGISTRY_REL).read_text(encoding="utf-8"))
+ modules = {
+ alias.asname or alias.name: node.module.split(".")[-1]
+ for node in ast.walk(init)
+ if isinstance(node, ast.ImportFrom) and node.module and node.module.startswith("app.connectors.")
+ for alias in node.names
+ }
+ expected = set()
+ for class_name in gct._registered_class_names(init):
+ tree = ast.parse((REPO_ROOT / gct.CONNECTORS_REL / f"{modules[class_name]}.py").read_text(encoding="utf-8"))
+ expected.add(gct._class_attribute(tree, class_name, "connector_id"))
+ assert set(inputs["registry"]) == expected
+
+
+def test_duplicate_connector_id_is_refused(tmp_path: Path) -> None:
+ """``_build_registry()`` raises on a duplicate id; so must the generator.
+
+ Silently collapsing two classes onto one id would shrink the union by one
+ member and give no reason.
+ """
+ pkg = tmp_path / "services/connectors/app/connectors"
+ pkg.mkdir(parents=True)
+ (pkg / "alpha.py").write_text('class AlphaConnector:\n connector_id = "same"\n')
+ (pkg / "beta.py").write_text('class BetaConnector:\n connector_id = "same"\n')
+ (pkg / "__init__.py").write_text(
+ "from app.connectors.alpha import AlphaConnector\n"
+ "from app.connectors.beta import BetaConnector\n"
+ "_CONNECTOR_CLASSES = (AlphaConnector, BetaConnector)\n"
+ )
+ with pytest.raises(gct.GateError, match="duplicate connector_id"):
+ gct.parse_registry(tmp_path)
+
+
+# --------------------------------------------------------------------------
+# The folds
+# --------------------------------------------------------------------------
+def test_canonical_folds_survive_generation(payload: dict, inputs: dict) -> None:
+ assert inputs["canonical"], "the normalizer declares no canonical folds; the parser has drifted"
+ rendered = gct.render_typescript(payload)
+ for alternate, target in inputs["canonical"].items():
+ assert alternate in payload["members"], f"{alternate!r} was flattened out of the union"
+ assert target in payload["members"], f"{alternate!r} folds onto {target!r}, which left the union"
+ assert f'"{alternate}": "{target}"' in rendered
+
+
+def test_the_fold_map_is_the_normalizers(payload: dict) -> None:
+ """The five folds PR #811 established, read back off the generated file.
+
+ Two products, one deployment: every source here must denote the same
+ vendor product as its target, which is what makes folding safe.
+ """
+ assert payload["canonical"] == {
+ "google_chronicle": "chronicle",
+ "ibm_qradar": "qradar",
+ "palo_alto_cortex": "cortex_xdr",
+ "slack": "slack_audit",
+ "syslog": "syslog_cef",
+ }
+
+
+# --------------------------------------------------------------------------
+# Drift detection
+# --------------------------------------------------------------------------
+def test_check_fails_on_a_hand_edit(payload: dict) -> None:
+ """A member deleted by hand must fail --check rather than be regenerated away."""
+ good = gct.render_typescript(payload)
+ edited = good.replace(' "qualys",', "", 1)
+ assert edited != good
+ codes = {code for code, _ in gct.evaluate(payload, edited, gct.render_json(payload), CONSUMER.read_text(encoding="utf-8"))}
+ assert "output-drifted" in codes
+
+
+def test_check_fails_when_the_console_redeclares_the_union(payload: dict) -> None:
+ """The generated file can be perfectly current and completely ignored.
+
+ A hand-written union in ``connector.ts`` is what TypeScript resolves, so
+ "the generated file exists" is not the property that matters.
+ """
+ consumer = CONSUMER.read_text(encoding="utf-8") + '\nexport type ConnectorType = "only_this";\n'
+ codes = {code for code, _ in gct.evaluate(payload, gct.render_typescript(payload), gct.render_json(payload), consumer)}
+ assert "consumer-redeclares" in codes
+
+
+def test_new_connector_shows_up_as_drift(payload: dict, inputs: dict) -> None:
+ """Adding a connector without regenerating must fail --check.
+
+ This is the direction things actually change: a connector is added far
+ more often than the union is edited.
+ """
+ grown = dict(inputs)
+ grown["registry"] = {**inputs["registry"], "acme_xdr_9000": {"class": "AcmeConnector", "module": "acme.py", "name": "Acme"}}
+ grown_payload = gct.build_payload(**grown)
+ assert "acme_xdr_9000" in grown_payload["members"]
+ codes = {
+ code
+ for code, _ in gct.evaluate(
+ grown_payload,
+ gct.render_typescript(payload), # the committed file, now stale
+ gct.render_json(payload),
+ CONSUMER.read_text(encoding="utf-8"),
+ )
+ }
+ assert "output-drifted" in codes
diff --git a/tests/test_gate_coverage_classifier.py b/tests/test_gate_coverage_classifier.py
new file mode 100644
index 000000000..bf3a15d07
--- /dev/null
+++ b/tests/test_gate_coverage_classifier.py
@@ -0,0 +1,235 @@
+"""A gate must be inventoried for what it does, not what it is called.
+
+``scripts/check_gate_coverage.py`` resolves the workflow-to-check graph so an
+orphaned gate cannot hide. It used to decide *which* scripts to resolve by
+filename — ``check_*``, ``validate_*``, ``_conformance.py``, plus a list of
+five exceptions — which is the same defect one level up: a gate named
+something unexpected was not inventoried, and an uninventoried gate is
+indistinguishable from one that does not exist.
+
+Nineteen were in exactly that state. Every one is a CI gate today, eleven of
+them run by a workflow with ``--check``; deleting any of those steps would
+have left the script reporting full coverage over a smaller tree.
+
+These tests pin the structural classifier, in both directions: a gate under a
+name no convention would reveal must be found, and a generator that aborts on
+an empty read must not be mistaken for one.
+"""
+
+from __future__ import annotations
+
+import sys
+from pathlib import Path
+
+import pytest
+
+REPO_ROOT = Path(__file__).resolve().parent.parent
+sys.path.insert(0, str(REPO_ROOT / "scripts"))
+
+import check_gate_coverage as cgc # noqa: E402
+
+#: What the filename test used to accept. Kept here, in the test rather than
+#: in the script, so "the classifier no longer depends on naming" is an
+#: assertion instead of a claim.
+_OLD_PREFIXES = ("check_", "validate_", "audit_", "lint_", "verify_")
+_OLD_SUFFIXES = ("_check.py", "_gates.py", "_conformance.py", "_audit.py")
+_OLD_EXTRA = {
+ "connector_conformance.py",
+ "detection_truth_table.py",
+ "openapi_diff.py",
+ "readme_gates.py",
+ "security_audit.py",
+}
+
+
+def _named_like_a_check(name: str) -> bool:
+ if name in _OLD_EXTRA or Path(name).match("sync_vendored_*.py"):
+ return True
+ return name.startswith(_OLD_PREFIXES) or name.endswith(_OLD_SUFFIXES)
+
+
[email protected](scope="module")
+def inventory() -> dict[str, dict[str, str]]:
+ return cgc.load(REPO_ROOT)["checks"]
+
+
+# --------------------------------------------------------------------------
+# A gate under an unconventional name
+# --------------------------------------------------------------------------
+UNCONVENTIONAL_GATE = '''\
+#!/usr/bin/env python3
+"""frobnicate_widgets.py — a gate whose name announces nothing."""
+
+import sys
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parent.parent
+
+
+def main() -> int:
+ offenders = []
+ for path in ROOT.glob("docs/**/*.md"):
+ if "\\t" in path.read_text(encoding="utf-8", errors="replace"):
+ offenders.append(str(path))
+ if offenders:
+ print(f"{len(offenders)} document(s) contain a literal tab:")
+ for path in offenders:
+ print(f" {path}")
+ return 1
+ print("OK")
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
+'''
+
+
+def test_a_gate_planted_under_an_unconventional_name_is_found() -> None:
+ """The case the old classifier could not see.
+
+ `frobnicate_widgets.py` matches no prefix, no suffix and no exception
+ list, and nothing invokes it — so only an intrinsic signal can find it.
+ """
+ assert not _named_like_a_check("frobnicate_widgets.py")
+ signals = cgc.classify_source(UNCONVENTIONAL_GATE)
+ assert signals, "a script that exits non-zero on findings it collected must be inventoried"
+ assert "findings-exit" in signals
+
+
+def test_the_planted_gate_is_reported_as_unreachable_when_nothing_runs_it(inventory) -> None:
+ """Being found is only half of it — the gate must then fail on it.
+
+ A classifier that inventories a script but whose rules never act on the
+ inventory is the vacuous case this whole script exists to disprove.
+ """
+ checks = {**inventory, "frobnicate_widgets.py": cgc.classify_source(UNCONVENTIONAL_GATE)}
+ routes = {name: ["workflow:ci.yml"] for name in inventory}
+ routes["frobnicate_widgets.py"] = []
+ codes = {code for code, _ in cgc.evaluate(checks, routes, [], {})}
+ assert "check-unreachable" in codes
+
+
+# --------------------------------------------------------------------------
+# The inverse: not everything that exits non-zero is a gate
+# --------------------------------------------------------------------------
+ABORTING_GENERATOR = '''\
+#!/usr/bin/env python3
+"""Renders a document. Exits non-zero when it has nothing to render."""
+
+import sys
+from pathlib import Path
+
+
+def main() -> int:
+ specs = sorted(Path("specs").glob("*.yaml"))
+ if not specs:
+ print("no specs found", file=sys.stderr)
+ return 1
+ Path("out.md").write_text("\\n".join(p.name for p in specs))
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
+'''
+
+NETWORK_TOOL = '''\
+#!/usr/bin/env python3
+"""Posts events to a running service. Not a verdict about the tree."""
+
+import sys
+import urllib.request
+
+EVENTS = [{"a": 1}, {"b": 2}]
+
+
+def main() -> int:
+ sent = 0
+ for event in EVENTS:
+ urllib.request.urlopen("http://localhost:8000", data=b"{}")
+ sent += 1
+ if sent < len(EVENTS):
+ return 1
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
+'''
+
+
+def test_a_generator_that_aborts_on_an_empty_read_is_not_a_check() -> None:
+ """Polarity is the whole distinction.
+
+ ``if not specs: return 1`` says "I could not do my job". ``if offenders:
+ return 1`` says "I found something". Only the second is a verdict.
+ """
+ assert cgc.classify_source(ABORTING_GENERATOR) == {}
+
+
+def test_a_script_that_only_talks_to_a_service_is_not_a_check() -> None:
+ assert cgc.classify_source(NETWORK_TOOL) == {}
+
+
+# --------------------------------------------------------------------------
+# What the change did to the inventory
+# --------------------------------------------------------------------------
+def test_structural_classification_is_a_superset_of_the_old_names(inventory) -> None:
+ """Nothing the filename test caught may be dropped.
+
+ A classifier that traded nineteen false negatives for one is not an
+ improvement, and a silently smaller inventory is how coverage is lost.
+ """
+ named = {p.name for p in (REPO_ROOT / "scripts").glob("*.py") if _named_like_a_check(p.name)}
+ dropped = named - set(inventory)
+ assert not dropped, f"structural classification lost {sorted(dropped)}"
+
+
+def test_the_inventory_grew_by_the_scripts_the_names_hid(inventory) -> None:
+ """The count moved, and this says by how much and why.
+
+ Every added script is classified by an intrinsic signal — what it does —
+ not by the fact that something happens to run it.
+ """
+ named = {p.name for p in (REPO_ROOT / "scripts").glob("*.py") if _named_like_a_check(p.name)}
+ added = set(inventory) - named
+ assert len(added) >= 15, f"expected the name-based test to have been hiding gates; found {sorted(added)}"
+ for name in added:
+ assert set(inventory[name]) - {"gates-a-workflow"}, f"{name} is inventoried only because something runs it"
+
+
+def test_every_verdict_flag_invocation_in_ci_is_inventoried() -> None:
+ """CLASSIFIER -> WORKFLOW, the other direction.
+
+ If a workflow runs a script with ``--check`` it is asking for a verdict.
+ A classifier that does not call that script a check has a blind spot, and
+ this is the assertion that finds it rather than a reader noticing.
+ """
+ data = cgc.load(REPO_ROOT)
+ missing = sorted(set(data["verdict_invocations"]) - set(data["checks"]))
+ assert not missing, f"CI asks these for a verdict but the classifier does not inventory them: {missing}"
+
+
+def test_blind_spot_rule_fires(inventory) -> None:
+ codes = {
+ code
+ for code, _ in cgc.evaluate(
+ inventory,
+ {name: ["workflow:ci.yml"] for name in inventory},
+ [],
+ {},
+ {"some_unclassified_thing.py": "ci.yml"},
+ )
+ }
+ assert "classifier-blind-spot" in codes
+
+
+def test_generated_connector_types_is_inventoried(inventory) -> None:
+ """The generator this change also added must itself be under the gate."""
+ assert "generate_connector_types.py" in inventory
+ assert "verdict-flag" in inventory["generate_connector_types.py"]
+
+
+def test_self_test_passes() -> None:
+ assert cgc.self_test(REPO_ROOT) == 0