Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
Merge remote-tracking branch 'origin/main' into fix/playbook-editor-step-vocabulary
Beenu Arorafdf53999ebe3
diff --cc CHANGELOG.mdindex 0254fb96b,57a1b82bc..baef29ac8--- a/CHANGELOG.md+++ b/CHANGELOG.md@@@ -7,89 -7,84 +7,163 @@@ and this project adheres to [Semantic V## [Unreleased]++### Added++++- **`scripts/generate_corpus_stats.py`** — generates++ `apps/web/src/data/corpus-stats.json` + `corpusStats.ts` from the compiled++ engine ruleset, the generated detection truth table, and the marketplace++ index, reconciling all three against each other and refusing to publish if++ they disagree. Every landing surface imports the constants, so none can++ carry its own literal. `--check` is wired into `ci.yml :: python-lint`;++ `--self-test` hand-edits the artefact and requires the drift to be caught.++ The artefact keeps `executable` and `onDisk`/`quarantined` as separate++ fields, and the UI leads with the executable count.++- **`scripts/check_alert_reduction_claims.py`** — prose cannot be generated++ the way a count can, so the retraction is gated instead. No published++ surface may assert the legacy harness runs the production grouping; any++ surface quoting 75.3 % must carry the retraction; the `alert_reduction`++ suite card may not be declared `kind: 'measurement'`; and every surface++ publishing the real figure must quote `PUBLISHED_REDUCTION_PCT`, a new++ constant in `services/fusion/tests/test_alert_reduction_real.py` that the++ test asserts against its own measurement — so the chain from measurement++ to published prose has no hand-copied link. A paragraph that dates or++ negates the claim is exempt, so the retraction can quote the wording it++ retracts.++++- **The playbook parity gate now reads every declaration of the step++ vocabulary, wherever it is.** `scripts/check_playbook_schema_parity.py`++ compared the engine against `packages/types/src/playbook.ts` and never++ opened `apps/web`; naming the editor's file here would have fixed that file++ and left the next one free. The TypeScript half is a scan: every `.ts` and++ `.tsx` file in the tree is parsed for literal collections of step-type++ names, and any collection overlapping the engine's vocabulary must either++ match it exactly or be a recorded subset with a reason, checked in both++ directions so an exemption that stops being needed fails the build. The++ editor's execution annotations are compared against the schema's++ `x-aisoc-execution` in both directions as well, so a surface cannot tell an++ author a step will run when the contract says it will not. `--list` prints++ what the scan credited, not only what it flagged, and the gate refuses to++ report agreement when it finds no declaration at all.++++ Against `origin/main` it reports 56 disagreements across all five++ vocabularies; against this tree it credits five declarations, each complete.++++ Two limits are stated rather than left implicit: a collection overlapping++ the vocabulary by fewer than two members is not treated as one, and a list++ derived at runtime is not a literal and is not seen — which is the shape the++ gate wants, because a derived list cannot drift. A file the reader cannot++ parse to the end is never silently skipped: it is checked for step-type++ names in its raw bytes first, and the gate refuses the tree if it finds any.++++- **A WCAG AA sweep over all twenty-two inspector forms.** The labels in++ `SchemaForm` and `StepInspector` sat beside their controls with no++ association, requiredness was an `aria-hidden` asterisk, help text was++ unreferenced, and the error summary named fields it could not be reached++ from. Controls now carry ids, labels `htmlFor`, `aria-required`,++ `aria-invalid` and `aria-describedby`; errors are reported per field as well++ as in the summary; the condition and params groups are fieldsets. The sweep++ also caught `<ul role="alert">`, which is not an allowed role for a list and++ left its items without a list parent.++++### Changed++++- **`readme_gates.py` covers the governance documents.** Its `FIGURE_DOCS`++ list named one compliance page, which is why `ROADMAP.md` drifted with CI++ green. `ROADMAP.md` and `RELEASES.md` are now on the list, the matrix++ **row total** is compared as well as the GATED/PARTIAL split, and a figure++ the prose explicitly dates ("the count at that time") is exempt so history++ need not be rewritten.++- **`check_scoreboard.py --check` verifies `agent_version` against++ `VERSION`.** `--refresh` already stamped it; nothing compared it, and the++ freshness gate reads only the date — which a refresh keeps current — so a++ row could be two days old and still labelled two majors behind.++### Fixed+ - **A retracted benchmark figure was still badged "Real measurement".**+ `apps/docs/docs/benchmark.md` withdrew the 75.3 % alert-reduction claim —+ the harness that produced it groups on four tiers of `(rule_id, host,+ user)` while the shipping `RawAlert.correlation_key()` groups on+ `{tenant}:{entity}:{tactic}`, so it does not merely re-implement fusion's+ grouping, it implements *different* grouping. The retraction reached one+ surface of five. `BenchmarkResults.tsx` rendered a green **"Real+ measurement"** badge on `0.753` with a blurb claiming the harness used the+ production rules, "same logic"; `benchmarks/alert-reduction.md`+ (`sidebar_position: 1`) called it a "faithful in-harness re-implementation"+ and headlined 75.3 %; `ComparisonTable.tsx` qualified it as "measured on+ fixed noisy stream". Two more were found while gating the invariant:+ `benchmark-methodology.md`, and `benchmark.md` itself, which re-asserted+ the claim in its own intro blockquote. All five now carry the wording+ `benchmark.md` already uses. The comparison table quotes **33.3 %**, the+ figure measured against the key the product actually runs.+ - **The landing page published three different wrong corpus counts.**+ "6,998 detections" in four places (the tree indexes 7,016, of which 5,937+ are quarantined and the engine loads **833**), "57 plugins" (77), "7,117+ community items" (7,155), and `218 rules across 5 categories` on the+ contributor leaderboard (833 across 6). The 6,998 figure also quoted the+ imported corpus as the detection capability, presenting quarantined rules+ as executable.+ - **Governance figures had gone stale.** `ROADMAP.md` published "136 rows —+ 128 GATED / 8 PARTIAL" against a matrix holding 139 / 131, in the same+ sentence that tells the reader to recount with the script "rather than+ trusting a figure quoted in prose — this line has gone stale before".+ `CLAIM_TO_GATE_MATRIX.md` carried a stale executable-rule figure (939) in+ a row note. The scoreboard's newest substrate row was labelled `v8.1.1`+ while `VERSION` read `10.0.0`; it is refreshed from a fresh deterministic+ run (0.97, unchanged) and now carries the tree's version. The scoreboard+ keeps its three rows, all `substrate: true`, and no live-LLM row was+ invented.+ - **The "Design partners" block on the landing page was removed** rather+ than updated. Four dashed "Partner A–D" chips under the caption+ "Reference partners onboarding through Q2 2026": placeholders rather than+ fabricated logos, but four of them assert a partner count nothing in the+ repository supports, and the window closed in June 2026 while still being+ advertised as upcoming.+-### Added+- **The playbook editor could author nine of the engine's twenty-two step+ types, and the build was green.** `apps/web/src/components/playbooks/types.ts`+ declared its own nine-member `StepType` union under a header comment+ claiming it mirrored `services/agents/app/playbook/models.py`, which+ declares twenty-two. The registry behind every form is keyed+ `Record<StepType, StepSchema>`, so exhaustiveness was satisfied against the+ *local* union and thirteen missing forms compiled cleanly; had it imported+ the union from `packages/types`, `Record` would have failed with thirteen+ missing keys. None of the fourteen files in the directory imported the+ published package. It now re-exports `StepType` from `@aisoc/types`, and the+ compiler named the gap.-- **`scripts/generate_corpus_stats.py`** — generates- `apps/web/src/data/corpus-stats.json` + `corpusStats.ts` from the compiled- engine ruleset, the generated detection truth table, and the marketplace- index, reconciling all three against each other and refusing to publish if- they disagree. Every landing surface imports the constants, so none can- carry its own literal. `--check` is wired into `ci.yml :: python-lint`;- `--self-test` hand-edits the artefact and requires the drift to be caught.- The artefact keeps `executable` and `onDisk`/`quarantined` as separate- fields, and the UI leads with the executable count.-- **`scripts/check_alert_reduction_claims.py`** — prose cannot be generated- the way a count can, so the retraction is gated instead. No published- surface may assert the legacy harness runs the production grouping; any- surface quoting 75.3 % must carry the retraction; the `alert_reduction`- suite card may not be declared `kind: 'measurement'`; and every surface- publishing the real figure must quote `PUBLISHED_REDUCTION_PCT`, a new- constant in `services/fusion/tests/test_alert_reduction_real.py` that the- test asserts against its own measurement — so the chain from measurement- to published prose has no hand-copied link. A paragraph that dates or- negates the claim is exempt, so the retraction can quote the wording it- retracts.+ The thirteen forms are built: `block_ioc`, `osquery_live_query`,+ `disable_user`, `reset_password`, `revoke_session`, `force_mfa`,+ `kill_process`, `quarantine_file`, `run_av_scan`, `run_script`,+ `search_siem`, `create_notable_event`, and `approval`. Each collects what+ its executor in `services/actions` reads, and each governed verb's target+ field is keyed on the name `engine._resolve_target` actually looks for, so a+ saved step carries its own target rather than depending on the alert context+ happening to hold one. Nothing collects a credential — those are resolved+ per tenant from the connector vault at dispatch — and nothing offers a+ `dry_run` control, because the bridge does not read one.-### Changed+ `approval` is presented, never offered. The engine records it in+ `_UNBRIDGEABLE` with a full reason, so the palette and the type dropdown+ filter on `execution !== 'unimplemented'` rather than on its name, an+ existing `approval` step still renders so an imported playbook can be read+ and fixed, and the inspector states that the engine fails the step closed+ and the run stops there.-- **`readme_gates.py` covers the governance documents.** Its `FIGURE_DOCS`- list named one compliance page, which is why `ROADMAP.md` drifted with CI- green. `ROADMAP.md` and `RELEASES.md` are now on the list, the matrix- **row total** is compared as well as the GATED/PARTIAL split, and a figure- the prose explicitly dates ("the count at that time") is exempt so history- need not be rewritten.-- **`check_scoreboard.py --check` verifies `agent_version` against- `VERSION`.** `--refresh` already stamped it; nothing compared it, and the- freshness gate reads only the date — which a refresh keeps current — so a- row could be two days old and still labelled two majors behind.+- **`osquery_live_query` reported SUCCESS while running nothing.** Its handler+ returned an error dict when the osquery backend clients could not be+ imported — which is always, in the shipped agents image, because those+ clients live in `services/actions` — and a returned dict leaves the step+ status at SUCCESS. Reachable from the console the moment the editor could+ author the type, so it now raises `PermanentStepFailure` and fails closed.++- **Three more partial vocabularies in the same directory.** `stepColors.ts`+ held a second nine-entry `Record<StepType, …>`; `StepInspector` and+ `PlaybookEditor` each hard-coded a nine-member array. All three are derived+ from the registry now. `packHelpers.ts` mapped step types to integration+ badges as a `Record<string, …>` covering eighteen of twenty-two, so a+ playbook built from the other four claimed to use no integrations at all.- ### Added-- - **The playbook parity gate now reads every declaration of the step- vocabulary, wherever it is.** `scripts/check_playbook_schema_parity.py`- compared the engine against `packages/types/src/playbook.ts` and never- opened `apps/web`; naming the editor's file here would have fixed that file- and left the next one free. The TypeScript half is a scan: every `.ts` and- `.tsx` file in the tree is parsed for literal collections of step-type- names, and any collection overlapping the engine's vocabulary must either- match it exactly or be a recorded subset with a reason, checked in both- directions so an exemption that stops being needed fails the build. The- editor's execution annotations are compared against the schema's- `x-aisoc-execution` in both directions as well, so a surface cannot tell an- author a step will run when the contract says it will not. `--list` prints- what the scan credited, not only what it flagged, and the gate refuses to- report agreement when it finds no declaration at all.-- Against `origin/main` it reports 56 disagreements across all five- vocabularies; against this tree it credits five declarations, each complete.-- Two limits are stated rather than left implicit: a collection overlapping- the vocabulary by fewer than two members is not treated as one, and a list- derived at runtime is not a literal and is not seen — which is the shape the- gate wants, because a derived list cannot drift. A file the reader cannot- parse to the end is never silently skipped: it is checked for step-type- names in its raw bytes first, and the gate refuses the tree if it finds any.-- - **A WCAG AA sweep over all twenty-two inspector forms.** The labels in- `SchemaForm` and `StepInspector` sat beside their controls with no- association, requiredness was an `aria-hidden` asterisk, help text was- unreferenced, and the error summary named fields it could not be reached- from. Controls now carry ids, labels `htmlFor`, `aria-required`,- `aria-invalid` and `aria-describedby`; errors are reported per field as well- as in the summary; the condition and params groups are fieldsets. The sweep- also caught `<ul role="alert">`, which is not an allowed role for a list and- left its items without a list parent.-## [10.0.0] — 2026-09-25**Two ways for a control to be absent: not written, or written and not