Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
docs: record the post-v8.0 wave, and fix three trackers that had gone stale (#720)
Beenu Arora2cb57d69bb85
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex b3a8dfe44..752e5968c 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -63,6 +63,91 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0credentials. Doing this before v8.0's degenerate-variance fix would have beenmisleading, since a constant stream collapses the standard deviation and everysuch principal would have read as permanently normal.+- **Four capability pillars, on a credibility floor+ ([#696](https://github.com/beenuar/AiSOC/pull/696),+ [#697](https://github.com/beenuar/AiSOC/pull/697),+ [#707](https://github.com/beenuar/AiSOC/pull/707)).** The floor came first,+ because a published claim that is false costs more than a missing feature.+ - **Context graph.** Neo4j schema v1.1 adds identity, asset, cloud, business+ and threat depth — Employee joined to Identity, Vulnerability, Application+ criticality and data classification, CloudAccount and Secret, and the+ IOC → Malware → Campaign → ThreatActor → Technique chain. One+ tenant-scoped traversal resolves an alert into all five behind+ `GET /api/v1/graph/incident-context/{alert_id}`, feeding the agent's+ context bundle, with `POST /graph/context/import` for directory and CMDB+ data. Neo4j had no migration mechanism at all, so any schema change+ landed only on fresh installs; real runners now exist for all three+ non-Postgres stores (`services/api/app/db/{graph,lake,vector}_migrations.py`)+ and the gate checks the half that matters — that each is *called* from a+ startup path. Qdrant's vector dimension is immutable, so an+ embedding-model change is a re-embed, surfaced by `pending_rebuilds()`+ rather than silently mixing incompatible vectors.+ - **Recursive investigation.** `run_with_tools` shipped working, guarded and+ instrumented, with zero production callers. It now drives+ `services/agents/app/investigator/deep_investigation.py` over ten+ strategies, with seven typed lake-backed pivots in+ `services/api/app/services/investigation_tools.py` behind+ `POST /investigate/query` — the model picks a tool and passes typed+ arguments, it never writes SQL. `scripts/check_investigation_depth.py`+ asserts the loop still has a caller, because a narrative reads equally+ plausible whether the agent pivoted five times or once.+ - **Action contract.** Risk, reversibility, verification and approval are+ declared per *capability* rather than per vendor, since the contract+ belongs to the verb. `approval_matrix.py` implements confidence × impact+ and `scripts/check_action_contract.py` enforces it.+ - **SOC-agent benchmark.** `packages/aisoc-benchmark` ships an adapter+ protocol plus an HTTP adapter, so a third party's agent can be graded+ against the same corpus, with hallucination rate and real-vs-synthetic+ provenance labels on every row.+- **A model matrix, so an accuracy number names its backend+ ([#715](https://github.com/beenuar/AiSOC/pull/715)).** The weekly eval runs+ one model, so its numbers describe the agent *on whatever the pins resolved+ to* and cannot separate a property of the agent from a property of the+ backend. `scripts/run_model_matrix.py` grades the same corpus across several+ models, one variable at a time. It is deliberately a thin wrapper that sets+ the pins and re-invokes `run_evals.py` rather than a second evaluator — two+ definitions of accuracy drift apart, which is the defect the alert-reduction+ suite already demonstrated — and a test asserts it defines no scoring of its+ own. With no funded key every row reads **not measured**, never `0.000`: a+ zero is a measurement, and "we did not run this" is not.+- **Dead letters can be read back+ ([#713](https://github.com/beenuar/AiSOC/pull/713)).** Three DLQ+ implementations existed — a log line, a Kafka topic with no consumer, and one+ that forgets on restart — and the worker defaulted to the first. An invisible+ drop is indistinguishable from an event that never arrived, and only one of+ those is an incident. `PostgresDLQ` plus migration `053_dead_letters.sql`+ persists them and `GET /api/v1/health/dead-letters` reads them back with a+ breakdown by reason. `/health/fleet` reports connector staleness alongside.+- **SLO alerts are generated from the objectives+ ([#713](https://github.com/beenuar/AiSOC/pull/713)).** `slos.yaml` declared+ targets for 17 services while the alert rules used thresholds corresponding+ to no objective in the file. `scripts/generate_slo_alerts.py` derives+ burn-rate alerts from it, covering the five services that actually expose+ `/metrics` and naming the twelve that do not, so no alert can be permanently+ dead.+- **The last two fidelity corpora are obtainable+ ([#717](https://github.com/beenuar/AiSOC/pull/717)).** Four datasets have+ loaders; only two shipped a downloader. `ait_lds_loader.py` and+ `mitre_engenuity_loader.py` could parse their full corpora from the day they+ landed with no way to fetch either, so their only numbers came from a+ ten-line micro fixture — which proves the loader parses and says nothing+ about the classifier at scale. The asymmetry was invisible because each half+ looked complete alone and nothing compared the two sets;+ `scripts/check_fidelity_datasets.py` does, and also enforces licence+ acceptance and citation across all four. New `*_full` floors are marked+ `measured: false`, lead with UNMEASURED, and are set at the micro-fixture+ level rather than guessed higher, because a floor invented above an+ unmeasured result is a gate that gets lowered rather than investigated.+- **Operator tooling for the paths that only matter under failure+ ([#696](https://github.com/beenuar/AiSOC/pull/696),+ [#697](https://github.com/beenuar/AiSOC/pull/697)).** AES-256-GCM backup+ encryption in `scripts/backup_crypt.py` (chunk-framed with a per-chunk nonce+ and AAD binding the chunk index, so a truncated or reordered archive fails to+ decrypt rather than restoring a smaller database, plus a SHA-256 manifest);+ scheduled Neo4j, Qdrant and Redis backups via a Helm CronJob;+ `scripts/support_bundle.py`; a per-investigation cost budget; and per-tenant+ ingest token buckets in `internal/inbox/ratelimit.go` (per-replica, and+ documented as such) on the one endpoint deliberately open to the internet.### Fixed@@ -103,6 +188,164 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0instructions (which documented the no-op YAML path that produced the 44orphans), and the orphaned `detections/splunk-imports/_migrated/` directoryof 16 files that no script, workflow or doc referenced.+- **Four documented security controls did not exist+ ([#696](https://github.com/beenuar/AiSOC/pull/696)).** AES-256-GCM backup+ encryption — the script gzipped and uploaded. Envelope encryption as the+ control mitigating a database dump — `EnvelopeCipher` had zero callers. One+ unbroken distributed trace — neither end of the Kafka spine emitted+ OpenTelemetry. Per-tenant retention marked GATED by a test asserting the+ purge SQL *parses* while nothing called it. Four more docs told operators to+ run `alembic downgrade` against a service with no Alembic, so the documented+ recovery path failed at the one moment it was needed. Each was corrected in+ the docs **and then implemented**, so the claim could return honestly:+ envelope encryption now carries `vault:v2:` tokens with `vault:v1:` read+ compatibility and fail-closed behaviour in both directions (the three+ vendored read-path copies *refuse* a `vault:v2:` token rather than returning+ ciphertext as plaintext); retention actually deletes; and tenant offboarding+ covers all five stores, with the Postgres table list discovered from+ `information_schema` rather than hardcoded — 14 of 72 tenant-scoped tables+ had no cascade, so deleting a tenant orphaned institutional memory and+ compliance evidence.+- **The prompt-injection guard missed every action-trigger payload+ ([#697](https://github.com/beenuar/AiSOC/pull/697)).** Adversarial recall was+ **0.22**, and the payloads that got through were the ones that produce a real+ isolate or disable — which turns the SOC into a denial-of-service aimed at+ its own estate. Eight new patterns take recall to **0.85** with zero false+ positives.+- **Nine capabilities declared a verification probe and three had one+ ([#697](https://github.com/beenuar/AiSOC/pull/697),+ [#698](https://github.com/beenuar/AiSOC/pull/698)).** Two probes were added+ and four declarations corrected to `false` with stated reasons, and+ "unverifiable means not autonomous" is now enforced: probes are required+ unconditionally for AUTOMATIC and for HIGH/SEVERE impact regardless of+ approval tier, and waived below MODERATE where the API response *is* the+ confirmation. `suspend_session` and `force_mfa` were demoted AUTOMATIC →+ ANALYST accordingly. Building the gate surfaced **eleven capabilities with a+ contract and no executor at all, including `unisolate_host`** — the rollback+ for the most disruptive action in the product resolved to nothing — fixed+ with a `KNOWN_ORPHANS` ratchet plus 17 new executors wired to client methods+ that already existed. Separately, `AzureEntraClient.get_user_enabled` called+ `_ensure_token()` with no argument and used the return value as a bearer+ token; the method takes a client and returns `None`, so every live call would+ have raised `TypeError` — invisible in simulation, which never constructs the+ client, and that method backs the `disable_user` probe, so a containment+ would have reported UNVERIFIED for an unrelated reason.+- **The sandbox's "deterministic" baseline moved every run+ ([#696](https://github.com/beenuar/AiSOC/pull/696)).** Risk scores used+ `abs(hash(str))`, with three comments calling it deterministic. CPython salts+ string hashing per process. The test runs the CLI under different+ `PYTHONHASHSEED` values, which is the only way to catch it.+- **The Helm chart could not be installed at all+ ([#696](https://github.com/beenuar/AiSOC/pull/696)).** Three templates+ referenced values absent from `values.yaml` — the UEBA deployment read+ `.Values.ueba.replicaCount` and there was no `ueba` key — so rendering+ failed before any cluster saw it.+- **Three shell and DR traps that only appear under failure+ ([#696](https://github.com/beenuar/AiSOC/pull/696)).** `pg_dump` stderr sent+ to `/dev/null` under `pipefail` handed an operator exit 1 with zero+ diagnostic; `fail() { ((ERRORS++)); … }` returns the *pre-increment* value,+ so under `set -e` the first failure killed the script before it printed why;+ and `grep -oP` made `restore --latest` impossible on macOS. `openssl enc`+ also refuses AEAD ciphers outright, which is why the AES-256-GCM path is a+ Python script on `cryptography` rather than a one-liner.+- **The one-directional gates+ ([#696](https://github.com/beenuar/AiSOC/pull/696),+ [#707](https://github.com/beenuar/AiSOC/pull/707)).** The graph-schema drift+ check compared the YAML vocabulary against Go and never the reverse, so it+ reported the schema consistent while Go declared 28 labels against the YAML's+ 17; it now parses the declared Go type (`NodeLabel` vs `RelType`) instead of+ classifying by string casing, which had misread `IOC`. In the same family:+ `neq` was never a matcher operator despite eleven rules using it, so+ `approver_role_neq: "codeowner"` was read as a field literally named+ `approver_role_neq`; every one of those rules' fixtures encoded the clause+ key verbatim because `build_positive()` synthesizes fixtures *from the rule*,+ so the durable fix is in the generator rather than the 22 files that+ regenerate from it; and `test_positive_fixtures_fire_and_negatives_do_not`+ only ever replayed the positives, leaving 100+ negative fixtures unrun under+ a test whose name says otherwise. Derived fields (`is_business_hours`,+ `<a>_eq_<b>` / `<a>_neq_<b>`) took the unreachable-rule ratchet from 138 to+ **133**, now reported by family — needs windowed engine, identity enrichment,+ allowlists, invented operands — so the number is actionable rather than bare.+- **A vendor's help text broke the entire docs deploy+ ([#708](https://github.com/beenuar/AiSOC/pull/708)).** Lacework's field help+ contains `https://<account>.lacework.net`, and Docusaurus parses `.md` under+ MDX, so `<account>` is an unclosed JSX element — one of which fails the whole+ build. The generator escapes angle brackets now, skipping code spans, with a+ gate over every connector page, because nothing about "MDX compilation+ failed" points at a vendor help string.+- **Six duplicate connector pages, with the gate reporting 100% coverage+ throughout ([#712](https://github.com/beenuar/AiSOC/pull/712)).** `_slug()`+ assumed hyphenated filenames; six pages predate that convention and use+ underscores, so the coverage check saw nothing at the hyphenated path and+ generated a thinner second page beside each hand-written original. The+ sidebar is generated from the same slug, so it listed only the thin copy and+ dropped the better half of each pair out of navigation. An existing page now+ wins over the convention, and a guard fails the gate when two pages differ+ only by separator.+- **Eight dead links survived in the published docs with a link checker running+ on every PR ([#709](https://github.com/beenuar/AiSOC/pull/709),+ [#718](https://github.com/beenuar/AiSOC/pull/718),+ [#719](https://github.com/beenuar/AiSOC/pull/719)).** A full QA pass over+ `https://beenuar.github.io/AiSOC/` — 167 sitemap pages, 170 internal and 405+ external link targets — took the health score from 81 to 99. Seven of the+ dead links pointed at three GitHub organisations that do not exist, and the+ wrong org was baked into the *generator* `scripts/curate_detections.py`, so+ fixing the page alone would have regenerated the 404. The link job could not+ catch this class: it runs lychee with `fail: false` and+ `--accept …403,429`, and GitHub rate-limits unauthenticated crawls hard, so a+ 403 from rate limiting is indistinguishable from the page existing.+ `scripts/check_repo_self_links.py` is the deterministic offline replacement.+ Two links into `docs/` (which the site does not serve) also took the whole+ deploy down, since Docusaurus treats a broken relative link as a build+ failure. Stale counts were re-read from the tree — 84 connectors, not 47;+ 833 executable rules, not "800 fixture-tested".+- **A hydration mismatch from adjacent JSX children+ ([#718](https://github.com/beenuar/AiSOC/pull/718)).** An SVG `<title>` built+ from several adjacent children gets `<!-- -->` separators injected during+ SSR that the client does not reproduce (React error #418). Build the whole+ string as one template literal so there is exactly one text node.+- **Sixty-six Dependabot alerts down to seven**+ ([#686](https://github.com/beenuar/AiSOC/pull/686),+ [#691](https://github.com/beenuar/AiSOC/pull/691),+ [#694](https://github.com/beenuar/AiSOC/pull/694),+ [#701](https://github.com/beenuar/AiSOC/pull/701),+ [#703](https://github.com/beenuar/AiSOC/pull/703),+ [#705](https://github.com/beenuar/AiSOC/pull/705),+ [#706](https://github.com/beenuar/AiSOC/pull/706),+ [#710](https://github.com/beenuar/AiSOC/pull/710), plus routine bumps in+ [#687](https://github.com/beenuar/AiSOC/pull/687)–[#714](https://github.com/beenuar/AiSOC/pull/714)).+ The three remaining highs are unpatchable (`ecdsa`, `image-size`). Two+ recurring shapes are worth remembering. A **caret pin that excludes the fix+ makes a security update unsolvable rather than pending**: the pytest advisory+ covers `< 9.0.3` and `^7.4.0` cannot reach it, so Dependabot had no version+ to propose and the failure surfaced as a red workflow rather than an open+ alert — ten services carried four different constraints for no reason, now+ uniformly `>=9.0.3,<10.0`. And **a range that permits the fix is not a lock+ that takes it**: the agents lock held langchain 1.0.2 against a 1.3.9 fix,+ all inside the existing caps. Transitive advisories need an override with a+ major ceiling, because without `<5` `@vitest/mocker` resolved to 5.0.1 —+ taking a major version of the test runner's mocking layer as a side effect of+ a security patch is how unrelated breakage gets blamed on security work.+ `dompurify` is the one worth naming: it is the sanitiser behind rendered+ report HTML, so a version below the fix was a live XSS surface in a product+ whose job is being trusted with hostile input.+- **The Fly demo outage finally has a stated cause+ ([#698](https://github.com/beenuar/AiSOC/pull/698)).** The self-provisioning+ deploy ran on merge and Fly answered: *"Your account has overdue invoices."*+ Both apps were reclaimed for non-payment, and the workflow's old silent+ `exit 0` when the app was missing hid that behind weeks of green runs. The+ error message used to guess at causes and got them wrong; it now prints+ Fly's own message verbatim. This is a billing action only the account owner+ can take, not a code fix.+- **Three CodeQL findings, at the root rather than around them+ ([#700](https://github.com/beenuar/AiSOC/pull/700),+ [#708](https://github.com/beenuar/AiSOC/pull/708)).** A previous attempt+ parenthesised an implicit concatenation without removing it, and added a+ top-level import alongside an existing from-import — turning one+ `py/import-and-import-from` alert into two. Fixed properly: setup prose built+ as named strings before the list, one import style per module, and four+ `py/unnecessary-lambda` findings cleared. Generated pages verified+ byte-identical.### Changed@@ -111,17 +354,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0its closing condition rather than as clean, because 138 rules still depend oncomputed fields and claiming otherwise would be the overstatement the matrixexists to prevent.+- The claim-to-gate matrix ends the wave at **102 rows: 93 GATED, 9 PARTIAL, 0+ NO GATE**, one row per capability added above, each backed by a test that+ fails against the old behaviour. The ratchet (`MAX_NO_GATE = 0`) still+ forbids a regression, and no `PARTIAL` row was relabelled without building+ the gate it names — relabelling would make the file a liability instead of a+ control.+- Of the twelve hardening phases, **Phase 4 is now the only one unchecked, and+ deliberately so**: what remains is a funded provider key for the live-agent+ eval, not code. Marking it done would be the exact failure the program exists+ to prevent.### Known-- `scripts/generate_detections.py` is not idempotent. Rule ids are positional- (`det-{category}-{NNN}`) and the committed YAML was produced from a different- spec ordering than the current code emits, so re-running it on a clean- checkout reassigns ids and trips the marketplace gate. `validate_detections.py`- does not compare the projection against the generator, so nothing catches it.- New rules must therefore be appended rather than inserted. Fixing it properly- means either content-derived ids or a one-time regenerate-and-commit plus a- `--check` gate.+- `scripts/generate_detections.py` still does not reproduce the committed+ detection pack, and the reason changed. Ids are no longer positional — #697+ pinned `(category, slug) → rule_id` in `detections/rule-ids.lock.json` — but+ **nothing reconciled the lock against the YAML already on disk**, and the+ `--check` mode that would have caught it was never written (nothing outside+ the generator references the lock). Running the generator on a clean checkout+ rewrites 57 files and moves 45 network rule ids.+ The consequence is worse than a regeneration hazard: the engine+ (`services/fusion/app/data/detection_ruleset.json`, which fires and stamps+ the id onto the alert) and the published projection **disagree about which+ rule owns those 45 ids**. `det-network-037` is "DNS TXT Response Over 250+ Bytes From Non-Resolver Host" to the engine and "DNS Tunnel Indicator: Long+ Hex Subdomain Sequence" in the YAML and `marketplace/index.json`, so an+ analyst who takes a rule id off an alert and looks it up in the catalogue+ reads a different rule's description, false-positive notes and playbook. This+ is precisely the harm the lock's own docstring warns about.## [8.0.0] — 2026-09-22diff --git a/RELEASES.md b/RELEASES.mdindex ca6dda67e..83419745c 100644--- a/RELEASES.md+++ b/RELEASES.md@@ -2,13 +2,33 @@This file mirrors what used to live in the "What's new" section of [`README.md`](README.md). The complete, machine-readable inventory (with file paths, env-var diffs, and per-release test counts) lives in [`CHANGELOG.md`](CHANGELOG.md).-> **TL;DR for first-time visitors:** AiSOC is on `v7.6.0`, released 2026-07-13 — the **Fully-Operational AI-SOC** release. It wires the three end-to-end paths that were previously unwired (the event lake is populated, the executable detection corpus fires on the live stream, every fused alert is auto-triaged, and approved SOAR actions execute against real connector credentials under an autonomy policy), and adds the competitive-parity differentiators plus nine new connectors. Every product claim is now backed by a failing CI test (claim-to-gate matrix: 33 GATED / 7 PARTIAL / 0 NO GATE). The latest GitHub release with notes and downloads: <https://github.com/beenuar/AiSOC/releases/latest>.+> **TL;DR for first-time visitors:** AiSOC is on `v8.0.0`, released 2026-09-22 — the **Close the loop** release. It connected capabilities the codebase already contained but never called: response actions are verified against the vendor rather than assumed from a 200, autonomy is governed by each tenant's own L0–L4 policy instead of one deployment-wide environment variable, repeat-alert suppression can fire at all, and analysts can hunt the events AiSOC itself ingested. Every product claim is backed by a failing CI test (claim-to-gate matrix: 93 GATED / 9 PARTIAL / 0 NO GATE). The latest GitHub release with notes and downloads: <https://github.com/beenuar/AiSOC/releases/latest>.---## What's new-`VERSION` is `7.6.0`. The **v7.6.0** release (2026-07-13) is the **Fully-Operational AI-SOC** release — it completes the A1–E1 roadmap that made the platform work end-to-end and pushed it to competitive parity + beyond.+`VERSION` is `8.0.0`. The **v8.0.0** release (2026-09-22) is the **Close the loop** release. v8.0 had been reserved for the package-publish milestone; nothing can publish without registry credentials, so the milestone was re-scoped and distribution/packaging moved out. What v8.0 does instead is close the gap between what the codebase contains and what it actually runs.++**v8.0.0 highlights (September 22, 2026)**+- **The finding worth remembering, because it repeated a dozen times:** the mechanism existed, was unit-tested, and had no caller on the path that needed it. A passing test on an uncalled function is indistinguishable from a working feature until someone traces the call graph. `evidence_fingerprint` promised volatile fields were excluded while its only caller hashed the alert row id plus the whole raw event, so no two alerts ever matched and v7.7's repeat-alert suppression could only ever report zero. `PostActionVerifier` had no caller, and its isolation probe returned `bool(device_id)` — it would have certified an uncontained host. The console wrote per-tenant L0–L4 autonomy tiers to Postgres while the dispatcher read one global environment variable. `get_entity_neighbors` accepted a `tenant_id` and never passed it to the driver.+- **Fabricated security data across 24 components.** The `AlertDetailView` catch block rendered a full invented verdict — named IP, C2 domain, "12 additional systems" — as `status: 'completed'`. `/hunt/search` always returned synthetic telemetry behind a green "Live backend" pill. All seeded and mock data is now gated behind demo mode, with honest empty, zero and error states otherwise, enforced by a new `check_mock_data_gated.py` gate.+- **Three services were fully unauthenticated** — `purple-team` (20 routes), `honeytokens` (9) and `ueba` (7) — and so was the live-action router. Default-deny landed on all four.+- **Securing the customer's AI estate** as the flagship capability: a new `ai` connector category, an `ai_gateway` connector, two ingest webhook templates mapping to OCSF `6003` and `2001`, a dependency-free SDK that hashes prompts by default while still reporting which secret shapes they contained, eight executable AI-runtime detections, and AiSOC's own MCP server as the first monitored asset.++The full inventory (every file, env-var, and test count) lives under `[8.0.0]` in [`CHANGELOG.md`](CHANGELOG.md).++---++## v7.7.0 (2026-08-04)++`VERSION` was `7.7.0`. Seven gap-closing waves, each its own PR: detection **backtesting** (`POST /rules/{id}/backtest` replays a candidate rule over real tenant-scoped lake events and reports honest `would_fire` / `hit_rate`); three **detection-authoring modes** (a Python `def rule(event)` framework with a fixture gate, an AI builder that turns natural language into Sigma plus auto-derived fixtures through the eval gate into a governed proposal, and a no-code builder); **invoking-identity least-privilege** scoping for response actions, so the authenticated principal replaces free-text `requested_by` and nobody approves their own action; self-service **data lifecycle** (per-tenant retention, a ReDoS-proof grok transform DSL, runtime custom parsers); an agentless **CSPM** scanner plus compliance auto-evidence and SSRF-guarded destinations; and a customisable **report builder**. Wave 1 wired components that existed but were never connected: auto-triage outcomes persist as per-signature institutional-memory priors, and a repeat alert matching a *trusted* prior benign disposition is auto-resolved without re-triage — human priors trusted immediately, AI priors needing corroboration, a prior true positive never auto-closing. Full inventory under `[7.7.0]` in [`CHANGELOG.md`](CHANGELOG.md).++---++## v7.6.0 (2026-07-13)++`VERSION` was `7.6.0`. The **v7.6.0** release (2026-07-13) is the **Fully-Operational AI-SOC** release — it completes the A1–E1 roadmap that made the platform work end-to-end and pushed it to competitive parity + beyond.**v7.6.0 highlights (July 13, 2026)**- **Phase A — the data spine flows.** A ClickHouse lake writer populates `aisoc.raw_events` from the stream (A1); a live detection-evaluation worker runs the 947-rule executable corpus against every event and emits alerts (A2); a cold `docker compose up` now ships connectors + graph-at-ingest by default, proven by an extended integration gate (A3); and the UEBA behavioral model is fused into alert scoring, making the three-model story real (A4).@@ -17,7 +37,7 @@ This file mirrors what used to live in the "What's new" section of [`README.md`]- **Phase D — breadth.** Eight new connectors — IBM QRadar, Exabeam, Securonix, Devo, Netskope, Windows/Sysmon (WEF), Zeek/Suricata NDR, and a generic syslog/CEF listener (D1); an AI/LLM-usage audit connector + eight `llm-*` detections + hot/cold ClickHouse lake tiering (D2); and a live-vendor mock-server smoke suite that exercises each connector's real HTTP client (D3).- **Phase E — prove it.** The public benchmark scoreboard is now CI-gated against a deterministic live-agent MITRE-accuracy run, closing the last `NO GATE` and ratcheting `MAX_NO_GATE` to 0 (E1).-The full inventory (every file, env-var, and test count) lives under `[7.6.0]` in [`CHANGELOG.md`](CHANGELOG.md).+The full inventory (every file, env-var, and test count) lives under `[7.6.0]` in [`CHANGELOG.md`](CHANGELOG.md). Note that the claim-to-gate figure quoted in the v7.6.0 announcement (33 GATED / 7 PARTIAL) was the count at that time; the matrix has grown to 102 rows since.---diff --git a/ROADMAP.md b/ROADMAP.mdindex fde259ea2..2abd6b884 100644--- a/ROADMAP.md+++ b/ROADMAP.md@@ -44,7 +44,7 @@ A proof-first, security-first program to make every README claim gate-backed, cl- **Phase D (breadth):** D1 eight connectors (QRadar/Exabeam/Securonix/Devo/Netskope/Windows-Sysmon/Zeek-Suricata/syslog-CEF) · D2 AI/LLM-usage audit connector + `llm-*` detections + hot/cold lake tiering · D3 live-vendor mock-server smoke.- **Phase E (prove it):** E1 CI-gated benchmark scoreboard tied to a deterministic live-agent MITRE-accuracy run.-The claim-to-gate matrix stands at **33 GATED / 7 PARTIAL / 0 NO GATE** — **every product claim is now backed by a failing test** and the ratchet (`MAX_NO_GATE=0`) forbids any regression. The 7 remaining PARTIAL rows are honest, named deferrals to phases outside the A–E scope. Per-session working detail is tracked locally in `docs/audit/PROGRESS.md`.+The claim-to-gate matrix stands at **102 rows — 93 GATED / 9 PARTIAL / 0 NO GATE** — **every product claim is backed by a failing test** and the ratchet (`MAX_NO_GATE=0`) forbids any regression. The 9 remaining PARTIAL rows are honest, named deferrals; none was relabelled without building the gate it names. Count the table rows with `python3 scripts/check_claim_gate_matrix.py` rather than trusting a figure quoted in prose — this line has gone stale before. Per-session working detail is tracked locally in `docs/audit/PROGRESS.md`.## v4.0 — Shipped@@ -565,19 +565,56 @@ Terraform skeleton equivalent to the existing AWS module.----## v8.0 — Planned+## v8.0 — Shipped (2026-09-22)++v8.0 shipped as the **close-the-loop** release rather than the feature list+below. It had been reserved for the package-publish milestone, but nothing can+publish without registry credentials, so packaging moved out (see v8.1) and the+release instead wired capabilities the codebase already contained and never+called. What v8.0 actually delivered is recorded under `[8.0.0]` in+[`CHANGELOG.md`](CHANGELOG.md) and summarised in [`RELEASES.md`](RELEASES.md).++Disposition of every item that had been listed against v8.0:-- Mobile responder console (React Native) — triage and acknowledge from phone-- Plugin publishing marketplace v3 (commercial plugins, revenue sharing)-- MSSP RBAC enforcement on `/api/v1/actors/*` endpoints (threat attribution)- ~~Automated IOC sharing to community MISP instances via STIX/TAXII push~~ → **shipped in v7.2.0**- ~~NL→query: "show me failed logins from new ASNs last 24h" → ES|QL / KQL~~ → **shipped in v7.2.0** (deterministic translator + 50-pair eval set)-- AI-generated threat intelligence briefings from public feeds-- Embedded red-team scoring (ATT&CK coverage %) as a live dashboard widget-- SLA breach predictor (ML model on historical MTTR data)-- Incident cost estimator (breach impact calculator)- ~~SOC-in-a-box one-click cloud deploy (Terraform module for AWS / GCP)~~ → **GCP module shipped in v7.2.0** (AWS already shipped)- ~~Automated retro/blameless post-mortem drafting from case timeline~~ → **shipped in v7.2.0** (ideas backlog item promoted)+- Mobile responder console (React Native) — **not started**, deferred; see the+ note under v7.0 above. No React Native code exists in the tree.+- Plugin publishing marketplace v3 (commercial plugins, revenue sharing) —+ **not started**, deferred. Revenue sharing is a commercial decision rather+ than an engineering one, and the free packaging path is itself still blocked+ on registry credentials.+- MSSP RBAC enforcement on `/api/v1/actors/*` (threat attribution) — **shipped+ in v7.5.0** as part of the threat-actor attribution RBAC + port fix.+- AI-generated threat intelligence briefings from public feeds — **open**, not+ scheduled.+- Embedded red-team scoring (ATT&CK coverage %) as a live dashboard widget —+ **open**, not scheduled. The underlying coverage heatmap shipped in v5.1; the+ dashboard widget did not.+- SLA breach predictor (ML model on historical MTTR data) — **open**, not+ scheduled.+- Incident cost estimator (breach impact calculator) — **open**, not scheduled.++---++## v8.1 — In progress++Wave-2 features plus release integrity. Tracked in+[`docs/roadmap/v8-progress.md`](docs/roadmap/v8-progress.md) and, for the+community-facing view, issue+[#362](https://github.com/beenuar/AiSOC/issues/362).++**Packaging is not in v8.1 either, and the reason is worth stating plainly:**+`release.yml` already builds, packs and would upload all eight packages — the+npm and PyPI jobs are written and run on every tag. The repository's only+secret is `FLY_API_TOKEN`. There is no `NPM_TOKEN` and no PyPI trusted+publisher, so the upload steps skip with a warning by design rather than+failing the release. This is an account action, not an engineering task, and+promising it in a release that cannot perform it is the kind of claim this+project's claim-to-gate matrix exists to prevent. Packaging is therefore named+against **v8.2**, and it becomes a re-tag the moment the credentials exist.---diff --git a/docs/roadmap/v8-progress.md b/docs/roadmap/v8-progress.mdindex 773e5104f..e5c846736 100644--- a/docs/roadmap/v8-progress.md+++ b/docs/roadmap/v8-progress.md@@ -1,147 +1,113 @@-# AiSOC v8.0 — North-Star Plan Progress+# AiSOC v8 — progress tracker-**Last updated:** 2026-06-27 (Saturday, Asia/Bangkok)-**Authored by:** AI assistant; verify by cross-referencing-[`plans/aisoc_v8.0_north_star_plan_1dee8c63.plan.md`](./plans/aisoc_v8.0_north_star_plan_1dee8c63.plan.md).+**Last updated:** 2026-09-23+**Current release:** `v8.0.0` (2026-09-22) · **In progress:** `v8.1`-This tracker exists so the remaining v8.0 work is visible at a glance,-including which acceptance criteria have been verified and which are-still outstanding. It is intentionally short and dated — when in-doubt, treat the linked v8.0 plan file as the source of truth and-update this tracker after each landed task.+This tracker is the at-a-glance view of what has landed across the v8 line and+what is still open. It is deliberately short and dated. When it disagrees with+the tree, the tree wins — and the tracker is the thing to fix.-> **History note.** A previous tracker existed at this path but was-> deleted prematurely in PR #232. This re-creation captures the-> current verified state of the four T-IDs the user re-prioritised on-> 2026-06-27.+> **A note on this file's history.** It sat unchanged from 2026-06-27 through+> the v8.0 release, describing four wave-1 tasks as "in flight" months after+> they merged and naming a loose end that had already landed. `README.md`+> linked here for "v8.1 packaging work in flight" and a reader found a June+> snapshot with no packaging in it. A tracker that is not updated is worse than+> no tracker, because it is read as current.----## Status snapshot+## v8.0 — shipped 2026-09-22-| T-ID | Title | Priority | Effort | Status | Notes |-|------|-------|---------:|-------:|--------|-------|-| T3.7 | NL → playbook generator | P1 | M | ✅ **Done** (2026-06-27) | Substrate + LLM paths, 47 backend tests, 8 frontend tests, dialog wired into `/playbooks`. |-| T3.8 | Design system v2 + Storybook | P1 | M | ✅ **Done** (2026-06-27) | Storybook 9 + Vite + Tailwind v4, 14 story files / 65 named stories, 4 new primitives, 33 visual-regression snapshots, CI gate. |-| T5.3 | AIT-LDS + MITRE Engenuity loaders | P1 | M | ✅ **Done** (2026-06-27) | Two new fidelity loaders + 16 tests + date-stamped `benchmark.md` section (29/29 fidelity tests). |-| T4 wave-3 | Last 5–6 connectors (Abnormal Security, Box, Datadog, Dropbox, Lacework, OCI, Sublime Security) | P1 each | S–M each | ✅ **Done** (2026-06-27) | 6 new plugin manifests + 6 new docs pages + 39 new tests + 6 connector resilience fixes. Full connector suite 633/633. |+Released as **close the loop**: connecting capabilities the codebase already+contained but never called. The recurring finding, which repeated a dozen+times, was *the mechanism exists, is unit-tested, and has no caller on the path+that needs it* — a passing test on an uncalled function is indistinguishable+from a working feature until someone traces the call graph.-All four T-IDs landed via dedicated PRs (merged 2026-06-27):+Full inventory: `[8.0.0]` in [`CHANGELOG.md`](../../CHANGELOG.md).-| T-ID | PR | Branch | Merged at (UTC) |-|------|----|--------|-----------------|-| T3.7 | [#330](https://github.com/beenuar/AiSOC/pull/330) | `feat/v8-t37-nl-playbook-drafter` | 08:55:22 |-| T3.8 | [#331](https://github.com/beenuar/AiSOC/pull/331) | `feat/v8-t38-storybook-design-system` | 08:57:36 |-| T5.3 | [#332](https://github.com/beenuar/AiSOC/pull/332) | `feat/v8-t53-aitlds-mitre-engenuity` | 08:53:48 |-| T4 wave-3 | [#333](https://github.com/beenuar/AiSOC/pull/333) | `feat/v8-t4-wave3-connector-scaffolding` | 08:55:17 |-| Tracker | [#334](https://github.com/beenuar/AiSOC/pull/334) | `docs/v8-progress-tracker` | 08:57:39 |+## Post-v8.0 — the credibility floor and four pillars-Follow-up after merge: [`feat/v8-storybook-draftdialog-story`](https://github.com/beenuar/AiSOC/tree/feat/v8-storybook-draftdialog-story) re-introduces the-`DraftFromPromptDialog` Storybook story that was split out of #331-because it imported a component that only existed on #330.+Landed on `main` between the v8.0 tag and the v8.1 cut+([#696](https://github.com/beenuar/AiSOC/pull/696),+[#697](https://github.com/beenuar/AiSOC/pull/697),+[#707](https://github.com/beenuar/AiSOC/pull/707),+[#713](https://github.com/beenuar/AiSOC/pull/713),+[#715](https://github.com/beenuar/AiSOC/pull/715),+[#717](https://github.com/beenuar/AiSOC/pull/717)).----+The floor came first because four docs described controls that did not exist.+Each was corrected **and then implemented**, so the claim could return+honestly. Then the four pillars: context graph (Neo4j schema v1.1 plus real+migration runners for all three non-Postgres stores), recursive investigation+(`run_with_tools` had zero production callers), the per-capability action+contract, and a SOC-agent benchmark a third party can run against their own+agent.++Claim-to-gate matrix: **102 rows — 93 GATED / 9 PARTIAL / 0 NO GATE**. Recount+with `python3 scripts/check_claim_gate_matrix.py`; do not quote a remembered+figure.-## What's verified per T-ID--### T3.7 — NL → playbook generator--- Backend module `services/agents/app/playbook/nl_drafter.py` implements- both the deterministic substrate path AND the LLM-assisted path- (with a clean fall-back when the LLM is unreachable, returns a- malformed payload, or fails Pydantic validation).-- Validates every draft against `schemas/playbook.schema.json`- (the file `scripts/lint_playbooks.py` consumes), with a step-type- collapse pass for Pydantic-only types so drafts never fail CI.-- HTTP surface: `POST /api/v1/playbooks/draft-from-nl` with 200 /- 400 / 422 covered by 9 integration tests.-- Frontend: `DraftFromPromptDialog` on `/playbooks` posts to the- endpoint, parks the draft in `sessionStorage["aisoc:nl-draft"]`,- routes to `/playbooks/new?nl=true`; the existing `PlaybookEditor`- hydrates from the seed via a one-shot effect.-- Test counts: 30 unit + 9 API + 8 frontend = **47 new tests**.-- Regressions: none — `237/237` playbook backend tests, `390/390`- apps/web tests green after the change.--### T3.8 — Design system v2 + Storybook--- Storybook 9.1 with the `@storybook/react-vite` framework wired up- in `apps/web/.storybook/{main.ts,preview.tsx}`. Tailwind v4 plugs in- via `@tailwindcss/vite` so utility classes work identically inside- Storybook and the running app.-- Four new canonical UI primitives in `apps/web/src/components/ui/`:- `Button` (5 × 4 variants/sizes), `Badge` (10 tones), `Card`- (+ header / body / footer), `StatusPill` (6 statuses).-- 14 story files × 65 named stories under `apps/web/stories/`,- grouped Foundations / Primitives / Composite.-- Visual-regression: 33 jsdom-DOM snapshots committed at- `apps/web/src/test/__snapshots__/storybook-snapshots.test.tsx.snap`- via Vitest.-- CI gate: new `storybook-build` job in `.github/workflows/ci.yml`- builds the static bundle and uploads as artifact (7-day retention).-- Pre-existing checks all green: type-check, lint (0 errors,- pre-existing warnings only), tests `390/390`.--### T5.3 — AIT-LDS + MITRE Engenuity loaders--- `services/agents/tests/fidelity/ait_lds_loader.py` parses Apache- CLF, joins labels from a sidecar CSV, normalises to OCSF Web- Resources Activity (class_uid 6002).-- `services/agents/tests/fidelity/mitre_engenuity_loader.py` parses- Round-7 procedure JSON, extracts techniques + tactics, normalises- to OCSF Detection Finding (class_uid 2004).-- Substrate runner extended (`runner.py`) with `_classify_ait_lds`- and `_classify_mitre_engenuity`; `_iter_dataset` + `_to_ocsf`- dispatch updated.-- Micro fixtures committed under `services/agents/tests/eval_data/`- (`access.log` force-added; `.gitignore` updated to unignore).-- 16 new tests in `test_fidelity_harness.py`; full suite **29/29**- fidelity tests green.-- `apps/docs/docs/benchmark.md` carries a new date-stamped- "Public-dataset fidelity (substrate)" section.--### T4 wave-3 — last 5–6 connectors--- Connector classes already existed; the gap was scaffolding around- them. Landed:- - 6 new `plugins/<connector>/plugin.yaml` (Abnormal Security, Box,- Datadog, Dropbox, OCI, Sublime Security; Lacework already- shipped).- - 6 new `apps/docs/docs/connectors/<connector>.md` pages.- - Comprehensive `services/connectors/tests/test_wave3_connectors.py`- with **39 new tests** covering schema, capabilities,- normalisation, connection probes, and error handling.-- 6 connector resilience fixes: every `fetch_alerts` (and Datadog's- `_fetch_logs` / `_fetch_events`) now wraps the `httpx` call in- `try / except (httpx.HTTPError, httpx.InvalidURL)` so network- errors log + break the pagination loop instead of crashing the- polling worker.-- Full connectors suite **633/633** tests green.+Of the twelve hardening phases, **Phase 4 is the only one unchecked, and+deliberately**: what remains is a funded provider key for the live-agent eval,+not code.----## What's still open--- **All five PRs are merged**, and the `DraftFromPromptDialog`- Storybook story that was split out of #331 has since landed too —- `apps/web/stories/composite/DraftFromPromptDialog.stories.tsx` is on- `main`. This tracker still described it as the outstanding loose end,- which had quietly become false.-- This tracker should be updated on every merge with the PR URL and- the merge timestamp.--## Out of scope for this pass--- T1–T2 fidelity work beyond the AIT-LDS + MITRE Engenuity micro- fixtures. The corpora are now *obtainable*:- `scripts/datasets/download_ait_lds.py` and- `download_mitre_engenuity.py` close the asymmetry where CICIDS and- CTU-13 each had a downloader and these two did not — so their- loaders could parse a full corpus nobody could fetch. Running them is- still a local, opt-in step, and the `*_full` floors are marked- `measured: false` until someone does.-- Storybook visual-regression in a real browser (pixel-diff) — the- current gate is jsdom DOM snapshots, which catch structural- regressions at sub-second cost. Real-browser regression is in the- Track-7 roadmap.+## v8.1 — in progress++### Wave-2 features (issue [#362](https://github.com/beenuar/AiSOC/issues/362))++Each item is audited against the tree before any code is written, because the+v8.0 lesson is that a capability is often already present and merely unwired.+Status is filled in from that audit rather than from the original ticket.++| T-ID | Item | Status |+|------|------|--------|+| T1.2 | Versioned config-snapshot writers (AWS / GitHub / Lacework / Okta) — Neo4j `:CONFIGURED_AS {ts}` edges so posture drift is queryable across time | see audit |+| T2.3 | `LLMInputContract` fail-closed validation across every sub-agent | see audit |+| T3.2 | Effective-permissions resolvers for Azure / GCP / Okta / Google Workspace | see audit |+| T3.3 | Attack-chain ranking + timeline UI hardening | see audit |+| T3.5 | Business-context rule engine | see audit |+| T3.6 | ChatOps coverage — Slack Block Kit approvals, Teams Adaptive Cards, signed email approval URLs | see audit |++### Release integrity++| Item | Status |+|------|--------|+| Changelog backfill for the post-v8.0 wave (#682–#719) | done |+| `RELEASES.md` brought up to v8.0.0 (it still announced v7.6.0) | done |+| `ROADMAP.md` — "v8.0 — Planned" section, stale matrix count | done |+| This tracker | done |+| Codespaces quickstart ([#716](https://github.com/beenuar/AiSOC/issues/716)) | open |+| Screencast + `hero.gif` assets | open |++---+## Not in v8.1: packaging++`release.yml` already builds, packs and would upload all eight packages on+every tag. The repository's only secret is `FLY_API_TOKEN` — there is no+`NPM_TOKEN` and no PyPI trusted publisher — so the upload steps skip with a+warning by design rather than reddening the release.++This is an account action, not an engineering task. Naming it against a release+that cannot perform it is the kind of claim the claim-to-gate matrix exists to+prevent, so packaging is named against **v8.2** and becomes a re-tag once the+credentials exist. Registry name state at the v8.1 cut: npm `aisoc`,+`@aisoc/sdk` and `@aisoc/mcp` are unclaimed; on PyPI `aisoc` is taken by an+unrelated project, so the Python side ships as `aisoc-sandbox`, and+`aisoc-sandbox`, `aisoc-cli`, `aisoc-sdk`, `aisoc-plugin-sdk` and+`aisoc-detections` are all free. One-time setup:+[`docs/operations/publishing.md`](../operations/publishing.md).++## Deferred by decision, not by omission++- **Mobile responder console (React Native).** Not started; no React Native+ code in the tree.+- **Plugin marketplace v3 (commercial plugins, revenue sharing).** Revenue+ sharing is a commercial decision rather than an engineering one, and the free+ packaging path is itself credential-blocked.++Both were labelled "deferred to v8.0" until that label went stale at the v8.0+tag. They are scope decisions, not gaps.