Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

chore(release): v10.0.0 (#869)

Beenu Aroraf7485be5ebd0
diff --git a/CHANGELOG.md b/CHANGELOG.md
index b4d35ed2f..ea749a2ae 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,576 +7,242 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
-### Added
-
-- **A gate that reads the one place a dead path could get in.** A comment is
- the only prose no check in this tree opens: `check_repo_self_links.py`
- reads markdown and matches `github.com` URLs, lychee resolves links, and
- neither looks inside source files — which is where most cross-file
- references in this repository actually live. That is how
- `docker-compose.yml` came to name
- `docs/architecture/decisions/0001-llm-gateway-in-core.md`, a path with no
- directory behind it, and survive every gate. `scripts/check_comment_paths.py`
- reads comments in fourteen languages plus Python docstrings and asserts
- that every repository path they name exists.
- The corpus was measured before the gate was written, because a naive
- version would have been mostly noise: of **3,092** path-shaped strings in
- non-markdown comments, most are MIME types, MCP method names
- (`tools/call`), Splunk REST routes (`services/search/jobs`), container
- images or URL routes. Three structural filters — a known file extension, a
- first segment that names something at the top of this repository, and not
- a URL / template / glob / gitignored artefact — cut that to **1,251**
- claims about this tree, resolved against the repository root *and* each
- ancestor of the referring file (a comment in `services/agents/tests/`
- saying `tests/conftest.py` means the one beside it).
- Extension-less directory references are **deliberately not checked** and
- the gate says so in its own output: 286 resolve and 52 do not, and
- hand-classifying the 52 put genuine rot at roughly one in five, which is
- the precision at which a gate gets ignored.
- Ten exceptions are recorded, each with a reason and each verified in both
- directions — an entry whose path now resolves, or which no longer appears
- in any comment, fails the build rather than sitting there.
-
-### Fixed
-
-- **A consumer that retried a permanent failure in silence, forever.** The
- `graph_ws` broadcaster in `services/ingest` could not die the way the UEBA
- consumer did — its loop `continue`s past an error — but it answered every
- error the same way: a flat 50ms sleep with the error discarded. Against a
- broker that was never coming back that is twenty reconnect attempts a
- second behind a container reporting `running`, restarts 0 and `/health`
- 200, with no line in any log and no counter anywhere. Not a silent death;
- a permanent failure wearing the costume of a transient one.
- Most of those errors never reached the loop at all. `kafka.NewReader`
- falls back to a **silent logger** when `ErrorLogger` is nil, and with a
- `GroupID` set the consumer group's dial, join and rebalance failures are
- reported *only* through it — `ReadMessage` stays blocked — so the single
- most likely permanent fault, an unreachable or misnamed broker, was
- discarded inside the library. `ErrorLogger` is now wired.
- Errors are classified into what the loop can do about them: **transient**
- (retry, with exponential backoff capped at 30s instead of a flat 50ms),
- **permanent** (the loop stops, because a loop over a fault no retry can
- clear turns a misconfiguration into indefinite churn — the same reasoning
- the UEBA consumer records), and **poison** (one undecodable envelope,
- counted and skipped without backing off a healthy subscription). A fourth
- state is a duration rather than a class: a `no such host` can be a startup
- race for a few seconds and a variable nobody set for ever, so it is
- reported as transient and then, after two minutes, as *not resolving* —
- the point at which the operator's next move stops being "wait".
- Permanence is decided by an enumerated set of protocol codes, not by
- negating `kafka.Error.Temporary()`: `REBALANCE_IN_PROGRESS` is
- non-retriable in the protocol's sense and happens on every deploy, so
- deriving the set that way would have stopped the consumer on a routine
- rebalance. A test walks the entire error table in both directions.
-- **Readiness that reported on the process and not on the subscription.**
- Ingest's `/readyz` now names every registered background subscription and
- its state, in the healthy case too, so a 200 says what was checked rather
- than only that nothing was wrong. The verdict stays keyed on the publish
- path deliberately: unlike the Python services, consuming is not ingest's
- job, and failing readiness for an opt-in WebSocket fan-out would pull the
- pipeline's front door out of the load balancer to fix a broadcast. The
- surface that depends on the subscription reports on it directly —
- `/v1/graph_ws/stream` answers 503 with the reason instead of completing a
- handshake onto a socket that will stay silent — and
- `aisoc_graph_ws_source_attached` / `_errors_total` / `_not_resolving` are
- the alertable signals.
-- **Two scheduler loops that logged that a tick failed and never what.**
- `retention_purge` and `hunt_scheduler` logged `err=%s` with
- `type(exc).__name__` and nothing else: `err=ProgrammingError` every thirty
- seconds says a tick failed, never why, and never whether waiting is the
- right response. Both now report the sanitised message and escalate from
- `warning` to `error` once the run of failures has outlived a transient
- explanation.
-- **Twenty-two dead paths in comments and docstrings**, found by the new
- gate on its first run. Among them: cost provenance pointing at migration
- `055` when the file is `063`; the Wazuh severity table pointing at
- `apps/docs/connectors/` instead of `apps/docs/docs/connectors/`; eight
- copies of the vendored `tenant_scope.py` pointing at a
- `check_vendored_tenant_scope.py` that is spelled `sync_`; eleven files
- pointing at a root-level `tests/test_security_defaults.py` that lives
- under `services/api/`; and three separate pointers at CI checks that had
- never been written, one of which `check_action_contract.py` already
- documented as fictional in its own docstring.
-- **A load generator that could tick forever and send nothing.**
- `services/demo-producer` silently `continue`d when `http.NewRequestWithContext`
- failed, which only happens for a malformed URL — the one error in that
- loop no retry can clear. It now says so and stops.
+## [10.0.0] — 2026-09-25
+
+**Two ways for a control to be absent: not written, or written and not
+reachable.** This release is mostly the second. Row-level security covered 92
+tables and filtered nothing, because every service connected to Postgres as a
+superuser. Fifty-eight routes across four services carried no authentication
+at all. UEBA consumed events, reported healthy, and could not write a baseline
+or an anomaly. AI triage was wired to a gateway that both of its resolvers had
+been deliberately written to ignore. In each case the mechanism was present
+and the path to it was not.
+
+Read the BREAKING section before upgrading. **Every deployment must act on one
+item**: services now connect as a DML-only `aisoc_app` role rather than as the
+schema owner, which is what makes those 92 policies filter. The bundled
+Postgres provisions it on a fresh volume; an existing volume or a managed
+database does not.
+
+Two published figures moved because they were measured rather than restated:
+CORE is **11 services** — the LLM gateway is in it now, so a provider key
+works with no profile change — and `full` is **21**, not the 30 previously
+published. The quick start also ends differently: `make up` creates an
+administrator and prints a generated password once, because the credential
+pair the documentation used to publish was wrong in three independent ways and
+nobody following the README could sign in.
-### Fixed
+### BREAKING
-- **The wave-1 service-test job installed twelve packages with no version
- bound, which is the outage that already happened, still loaded.** A CI job
- pip-installing a hand-curated list named a bare, unbounded `sqlalchemy` and
- passed for months only because a platform marker happened to pull
- `greenlet`; when 2.1.0 deleted that clause, every import of
- `sqlalchemy.ext.asyncio` began failing at collection. That one was bounded.
- The other twelve in the same list were not, and the job's own comment said
- so — *"Bounding the range is the second half: unbounded, any upstream
- release lands here before anyone reads it."* `pydantic`,
- `pydantic-settings`, `pytest`, `pytest-asyncio`, `structlog`, `redis`,
- `aioredis`, `httpx`, `aiokafka`, `asyncpg`, `clickhouse-driver` and `pyyaml`
- now carry the range the three services under test declare, so the job
- installs what they ship rather than whatever PyPI is serving this morning.
+- **The services no longer connect to Postgres as a superuser, so the 92
+ row-level-security policies added below now actually filter.** The previous
+ entry closed the coverage gap and measured, in the same breath, that none of
+ it did anything: `docker-compose.yml`, the CI service containers, the Helm
+ chart and the Terraform environment all ran every service as
+ `POSTGRES_USER=aisoc`, which the postgres image creates as a **superuser**,
+ and a superuser ignores policies *even under* `FORCE ROW LEVEL SECURITY` —
+ FORCE binds the table owner, not a superuser. Sixty-one new policies bought
+ nothing operationally.
- Two things fell out of doing it, both pre-existing and both invisible while
- the install was unbounded. `redis` was installed bare while
- `services/fusion` declares `redis[hiredis]` — the same dropped-extra shape
- as the `sqlalchemy` break, an install path quietly installing a smaller set
- than ships. And the three manifests **could not be satisfied at once**:
- `services/fusion` declared `httpx = "^0.26.0"` while `honeytokens` and
- `purple-team` declare `>=0.27.0`. The unbounded install resolved 0.28.1, so
- fusion was being tested on a version its own manifest forbade and shipping
- 0.26.0 — tested and shipped were different software, which is the exact
- thing `check_dependency_pins.py` exists to prevent. fusion moves to
- `>=0.27,<0.29`, matching api / actions / mesh / osquery-tls / slack-bot /
- teams-bot, and its lock re-resolves to 0.28.1 so the three now agree and the
- job tests what fusion ships. All three suites pass on the bounded set
- (fusion 308, honeytokens 66, purple-team 100, every coverage floor met).
+ `061_runtime_app_role.sql` splits the credential in two. `aisoc` owns the
+ schema and applies migrations; `aisoc_app` is what every service connects
+ as, holding `USAGE` on `public`, `SELECT / INSERT / UPDATE / DELETE` on
+ tables and views, `USAGE, SELECT` on sequences and `EXECUTE` on functions —
+ no `CREATE`, no `TRUNCATE` (`002_rls.sql` had granted `ALL`, which let one
+ statement delete every tenant's rows without a policy seeing a `WHERE`
+ clause), and ownership of nothing. Measured on `postgres:16` with two alerts
+ seeded one per tenant and the session bound to tenant A, reading `alerts`
+ with **no tenant predicate at all**: the old role saw 2, the new role sees
+ 1, and an insert for tenant B is refused by the policy.
- Still unbounded and reported rather than silently fixed: `services/connectors`
- and `services/threatintel` also declare `httpx = "^0.26.0"`, and neither is
- in this job.
+ Things this turned up along the way, each of which would have survived the
+ role switch and quietly undone it:
-### Changed
+ - **Two views read around every policy underneath them.** A view executes as
+ its *owner* unless declared `security_invoker`, and both views here are
+ owned by the role that ran the chain. Bound to tenant A,
+ `mssp_tenant_latest_metrics` returned 2 rows before and 1 after.
+ - **`SET LOCAL row_security = off` stops being a no-op and becomes a
+ crash.** For a role the policies apply to, Postgres refuses the query
+ rather than ignoring the policy. The retention purge, the hunt scheduler's
+ sweep and tenant deletion all used it; all three now rely on the
+ `OR current_tenant_id() IS NULL` arm and call `assert_cross_tenant_session()`
+ first, which raises if a tenant *is* bound — a sweep that sees one tenant
+ and reports success is worse than one that fails.
+ - **`CREATE TABLE IF NOT EXISTS` still needs `CREATE` on the schema when the
+ table already exists**, because Postgres checks the ACL before the
+ existence test. Two stores in `services/agents` opened their pool that way
+ and swallowed the failure into `logger.debug`, so the agents service would
+ have silently recorded no cost telemetry and no institutional memory at
+ all. Both tables are already in the migration chain; the bootstrap now
+ probes first and reports at `error` if it genuinely has to create one.
-- **`mypy` moved to 2.3.1 across all ten declarations, and the baseline it
- ratchets became reproducible outside CI.** The bump alone makes
- `scripts/check_mypy_baseline.py` exit 1 on 35 mismatched `(tree, file, code)`
- entries, so the re-record is the work — and a re-record is the moment a
- ratchet can quietly grow, so every entry was triaged before being banked.
+ Deployment surfaces updated: `docker-compose.yml`, the demo compose stack,
+ `integration.yml`, the Helm chart's `values.yaml`, the Terraform environment
+ (which now generates the runtime role's password rather than reusing the RDS
+ master user) and `.env.example`. `infra/postgres/initdb/zz_runtime_role_password.sh`
+ sets the credential on a fresh volume before the container reports healthy;
+ `app.scripts.run_migrations` applies it on every run, which covers an
+ upgrade in place. **Operators on a managed Postgres must act**: apply the
+ chain as the owner with `AISOC_APP_DB_PASSWORD` set, then point
+ `DATABASE_URL` at `aisoc_app` and `DATABASE_MIGRATION_URL` at the owner.
+ `002_rls.sql` created `aisoc_app` with the literal password `changeme`; 061
+ does not clear it (that would break an operator who had already set a real
+ one), so rotate it if none of the automatic paths applies to you.
- **Zero new findings.** All 35 are the same code in the same direction:
- `import-untyped` went 38 → 3 and nothing appeared, nothing grew. The cause
- is a behaviour change rather than a code improvement, and is recorded as
- such: mypy 2 extends `ignore_missing_imports = true` to silence
- `import-untyped`, which 1.x did not. Isolated on a two-line file — under
- 1.20.2 the finding is reported with the option either way; under 2.3.1 it is
- reported only with the option off. So the ten trees that set it lose the
- "this third-party import carries no type information" signal, every one of
- the 35 was `yaml`, and `packages/aisoc-cli` — strict, with no such option —
- still reports its three. `enable_error_code = import-untyped` does not
- bring it back; `ignore_missing_imports` wins. The alternative, dropping the
- option, would surface `import-not-found` for every uninstalled dependency in
- a deliberately dependency-free run, which is a statement about the runner
- rather than the tree. Banked at 955, with the reason on the record.
+ Two new gates, both with a self-test that runs before the scan and both
+ failing over an empty or absent tree. `scripts/check_runtime_db_role.py`
+ fails if a runtime DSN connects as the role that surface provisions as the
+ database superuser — structurally, by reading `POSTGRES_USER` /
+ `db_username` out of the tree rather than matching a name — and in
+ `--dsn` mode reads `pg_roles` and `pg_class` directly, catching
+ `rolsuper`, `rolbypassrls`, ownership, a view without `security_invoker`,
+ and a role still accepting `changeme`. `scripts/check_rls_policy_shape.py`
+ fails if a policy is written without the fail-open arm, replaying the chain
+ in order so a definition a later migration repaired is not reported. Both
+ run in `isolation.yml`; the live half runs in `integration.yml`.
- **The baseline was only reproducible on a CI runner, which is not
- reproducible.** Measured rather than assumed: the interpreter is not the
- variable people assume — Python 3.13 against 3.11 moves exactly 2 findings
- in 1 file (PEP 701 changed f-string sub-expression line attribution in 3.12,
- splitting findings 3.11 reports once) — while installing eight project
- dependencies moves 59. mypy reads type information out of installed
- packages, so a contributor with a service virtualenv active got mismatches
- in both directions and nothing saying the cause was their environment. A
- second cause surfaced while measuring the first: a `.mypy_cache` written
- under different conditions is reused by the next run, and the same tree
- answered 990 cold against 988 warm.
+ `tests/isolation/test_postgres_rls.py` now reads as `DATABASE_URL` — the
+ role the deployment ships — instead of a `NOSUPERUSER NOBYPASSRLS` role it
+ constructed for itself, so its 80-table two-tenant replay covers what ships.
+ `test_superuser_bypasses_rls_which_is_why_the_probe_role_exists` is replaced
+ by its inverse.
- Both are now closed at the source. The gate passes `--no-site-packages`
- (measured on `services/connectors`: 73 findings bare, 140 with eight
- dependencies installed, 73 either way with the flag — and a no-op in CI's
- own dependency-free environment, verified by re-recording under 1.20.2 and
- getting the committed file back byte for byte) and `--no-incremental`. It
- records the mypy version in an `(environment)` block and refuses to compare
- across majors, because a major *moves* findings and every difference would
- otherwise print as a file-level diff that describes the version. The
- interpreter difference is reported as a note rather than a refusal, so a
- contributor on 3.12 can still run the gate. Verified: the same 955 from a
- bare interpreter and from one with the project's dependencies installed.
+- **The four services that run their own alembic chain now migrate as the
+ owner and serve as the runtime role, and three of them were not reached by
+ the role switch at all.** `honeytokens`, `osquery-tls`, `purple-team` and
+ `ueba` manage their own schema, and each applied it as whatever DSN the
+ operator supplied — so their migration and runtime credentials were the same
+ one, and pointing such a service at the owner turned off row-level security
+ for the twelve tables those chains own with nothing objecting. Each
+ `env.py` now reads `<SERVICE>_DATABASE_MIGRATION_URL`, then
+ `DATABASE_MIGRATION_URL`, and only then falls back to the runtime DSN with a
+ warning on stderr naming what will break.
- `mypy` is now in `check_dependency_pins.py`'s `CRITICAL` set, so the ten
- declarations have to move together the way `ruff`'s fourteen do. Nothing
- enforced that before, which is how a single-path bump could be proposed at
- all. One real bug was caught by the re-recorded baseline while this was
- being written — a `used-before-def` introduced into the gate's own
- self-test — and fixed rather than banked.
+ Three things were found while wiring it, each measured rather than inferred:
-- **`ruff` moved to the 0.16 line across all fourteen declarations, the tree
- was reformatted under it, and the lint gate stopped ending at `services/`.**
- The bump alone reds every open pull request, because `ruff format --check`
- is a hard gate on the required `Python — Lint & Type-check` job and 0.16
- formats differently from 0.4.10. So the migration is the reformat. Measured
- under 0.16.9 (what the range resolves today, and byte-identical to 0.16.8 on
- both format and lint): **218 files reformatted**, 862 insertions against
- 1075 deletions, almost all of it collapsing calls that 0.4.10 wrapped and
- 0.16 fits inside the 140-column limit. It is its own commit with no other
- change in it.
+ - **`DATABASE_URL` was inert on three of the four.** `docker-compose.yml`
+ sets it on every service, but `honeytokens`, `purple-team` and
+ `osquery-tls` declare `env_prefix` in their settings, so the name they
+ read is `HONEYTOKEN_DATABASE_URL` and the compose entry did nothing: each
+ fell back to a default naming the **owner**. Both spellings now resolve,
+ unprefixed first, the convention `services/ueba` and `services/fusion`
+ already used. Operator note: on a deployment that sets both to *different*
+ databases, the unprefixed one now wins.
+ - **The four chains shared one `alembic_version` table** — they share one
+ database in the default deployment and number their revisions identically.
+ Following `apps/docs/docs/quickstart.md` against `postgres:16`: after
+ `ueba` reached `0002`, `honeytokens alembic upgrade head` ran **zero**
+ migrations and `purple-team` failed applying its RLS revision to tables
+ that had never been created, so two services had no tables and no policies
+ and every command reported success. Each chain now keeps its own version
+ table and adopts an existing deployment's recorded version on the next
+ upgrade — only when that chain's own tables are already present, so it
+ cannot claim a sibling's row.
+ - **The runtime role had no grant on those tables.**
+ `061_runtime_app_role.sql` grants over `ALL TABLES` as they stood and sets
+ `ALTER DEFAULT PRIVILEGES` for the role that issued it; nothing orders the
+ chains against it, and a deployment applying them under different owners
+ gets neither. Each chain now grants `SELECT, INSERT, UPDATE, DELETE` on its
+ own tables, guarded so it is a notice rather than a failure where the role
+ does not exist.
- All fourteen declarations now read `>=0.16.8,<0.17` — seven manifests
- (`services/{api,fusion,osquery-tls}`, `packages/{aisoc-cli,aisoc-detections,
- plugin-sdk-py,sdk-py}`), three workflows (`ci.yml`, `ai-sdk.yml`,
- `python-detections.yml`), `.devcontainer/Dockerfile`, and the three
- `poetry.lock` files that resolve it, now at 0.16.9.
+ Verified live on `postgres:16` with all four chains applied and two tenants
+ seeded: bound to one tenant the runtime role sees one row of two in
+ `ueba_entity_baselines`, `honeytokens` and `osquery_node`; unbound it sees
+ both, which is the fail-open arm the cross-tenant sweeps depend on; a
+ cross-tenant insert is refused by the policy; `CREATE TABLE` is refused with
+ `permission denied for schema public`; `ALTER TABLE … NO FORCE ROW LEVEL
+ SECURITY` with `must be owner of table`; and `SET LOCAL row_security = off`
+ raises rather than doing nothing. None of the four chains creates a view, so
+ the `security_invoker` hazard does not arise there, and none of the four
+ services issues DDL outside its chain.
- The three lint findings 0.16 reports on `services/` are fixed rather than
- suppressed: two `C420` dict comprehensions become `dict.fromkeys` (both
- values are string literals, so the shared-value hazard the rule exists
- around does not apply), and one `UP031` percent-format becomes an f-string
- in a chunker test that counts characters — all three rewrites asserted
- equal to the originals before the change was kept.
+- **The published `aisoc-web:latest` image shipped demo mode baked on**, and
+ it is the image `docker-compose.yml` pulls for `make up`. Next inlines
+ `NEXT_PUBLIC_*` at build time, so there was no runtime escape: a
+ self-hoster's console announced "Demo data resets daily at 00:00 UTC. All
+ write actions are disabled." over their own real alerts, disabled every
+ write control, and offered a "Self-host AiSOC" link inside an already
+ self-hosted install. The only remedy was to rebuild the image.
- **Scope.** The gate read `ruff check services/`, which left `scripts/` —
- every CI gate in this repository, the code the rest of its claims are
- verified by — linted by nothing at all. That is the same gap
- `check_mypy_baseline.py` closed with its unmanaged scope, and it is closed
- the same way here: lint now covers `services/ scripts/ tests/ tools/`, the
- trees the repo-root `ruff.toml` governs that nothing else lints, and
- `ruff format --check .` covers the whole repository because formatting has
- no semantic content and `.` needs no edit when a tree is added. That
- surfaced 17 real findings outside `services/`, each fixed at the source: two
- `B007` loops switched to `.values()`, a `B904` re-raise in
- `security_audit.py` now chains `from exc` so the original traceback
- survives, two `E402` imports that genuinely follow a `sys.path` mutation
- carry a reason next to the marker, and two over-length lines were wrapped.
+ The demo is now its own build under its own tag. `latest`, `main` and
+ `vX.Y.Z` are the product with demo mode off; `demo` and `vX.Y.Z-demo` carry
+ the demo bundle, and `infra/compose/docker-compose.demo.yml` pulls those.
+ Nothing has to choose between a working demo and a usable self-host image.
- Two exclusions, both named rather than implied. `plans/` is excluded in
- `ruff.toml`: it is the archived prototype subtree `codeql.yml` and
- `check_mypy_baseline.py` already exclude and the project rules forbid
- editing, and it holds 844 lint findings and ~280 unformatted files — they
- are excluded because they cannot be fixed, which is not the same as being
- clean. `E501` is ignored for `scripts/detection_specs.py` and
- `scripts/detection_specs_part2.py`: both open their rule table with
- `# fmt: off`, so the formatter is already opted out of that layout by a
- deliberate decision, and all 199 long lines there are single dict or string
- literals — 72 positive fixtures, 66 negative fixtures, 50 `match_when`
- clauses, 11 descriptions. Two named data modules, not `scripts/`; exactly
- one long line elsewhere under it needed wrapping.
+ Demo credentials are also no longer compiled into a non-demo bundle at all.
+ `apps/web/Dockerfile` defaulted the autologin address and password to real
+ values, and `login/page.tsx` and `DemoAutoLogin.tsx` each declared them as
+ module-level literals — gating the *render* on `isDemoMode()` hid the panel
+ but left the strings in every chunk the project ships. All three now read
+ them from the build environment, and only the demo build supplies them.
- `packages/` and `plugins/` stay outside the lint gate on purpose and the
- gap is a number rather than a shrug: 91 findings and 2 respectively. Two of
- the five packages are already linted by their own workflows under their own
- configs, and bringing the three published SDKs under this gate means
- behaviour-touching fixes — `subprocess.run` check semantics, blind
- `except Exception` handling, `zip(strict=)` — that belong in a change
- reviewed as such rather than in a formatter migration.
+- **Three MSSP response schemas describing fabricated data are removed:
+ `MSSPKpiOverview`, `ManagedTenantRow`, `CrossTenantIncident`.** They were
+ the shape of five hardcoded companies with invented alert counts, not the
+ shape of anything the platform measured. `/mssp/overview`,
+ `/mssp/tenants` and `/mssp/incidents` keep their paths and now return
+ `PortfolioSummaryOut`, `PortfolioTenantOut` and `PortfolioAlertOut`,
+ computed from real rows.
-- **`sqlglot` moved to the 30 line across all eight declarations, and the
- forward-compatibility matrix leg was rewritten so it can still reach past
- the pin.** The bump itself was never the work: a single-path proposal moved
- 1 of 8 declarations and left six workflows at `>=23,<27`, which
- `scripts/check_sqlglot_pin.py` fails closed on — correctly, since that gate
- exists because sqlglot 27 renamed the SELECT's FROM argument key and
- silently stripped the tenant predicate, the table allowlist and the `url()`
- exfiltration ban from every single-table query while reporting success. All
- eight now read `>=30,<31` (`services/api/pyproject.toml` plus `ci.yml`,
- `integration.yml` ×2, `isolation-live.yml`, `cross-tenant-rbac.yml`,
- `check-openapi.yml`, `reproducible-builds.yml`), and
- `services/api/poetry.lock` resolves 30.19.0 inside it.
+ What moved, and why it could not be preserved:
- One major wide rather than four. The previous range permitted 23–26 and
- exactly one of those was ever exercised, so three majors were installable
- and untested; 27, 28 and 29 are likewise unexercised, and a range states
- what may be installed rather than what upstream has released.
+ - `health_score` / `avg_health_score` are **gone**, not nulled. It was an
+ undefined composite with no formula anywhere in the tree; keeping the
+ field would promise a measurement that does not exist. (`MetricsOut` on
+ the separate `/mssp/metrics` route still carries a `health_score` and is
+ unchanged by this release.)
+ - `avg_mttr_minutes` → `mttr_minutes`, measured from cases a tenant
+ actually closed in the trailing 30 days, and **null** when it closed
+ none. The old value was the literal `23.4`. Present on both the per-tenant
+ row and the portfolio summary, where it averages only over tenants that
+ closed something rather than counting a null as a zero.
+ - `sla_breach_count` and `sla_breaches` → `sla_breached_cases`, counted
+ from `cases.sla_breached`.
+ - `connectors_online` / `connectors_degraded` / `connector_status` →
+ `total` / `healthy` / `stale` / `error`, derived from each connector's
+ `health_status` and `last_sync`. Nested under a `connectors` object on
+ the per-tenant row; flat `connectors_*` fields on the summary.
+ - `tenant_id` is now a real tenant UUID rather than a string like
+ `"t-acme"`.
+ - `assignee` is gone from the incident rows. It named invented analysts;
+ an alert's real owner is `case_id`, which is now returned instead.
+ - New: `synthetic_alerts`, so seeded demo rows are counted apart from a
+ tenant's real posture instead of inflating it.
- Measured before moving, on 64 adversarial cases written against the
- documented policy rather than reusing the module's fixtures — DDL, DML,
- admin verbs, multi-statement input, seven ClickHouse table functions
- including ones hidden in subqueries and CTE bodies, non-allowlisted tables
- reached through JOIN, subquery, UNION arm and CTE, and five forged
- `tenant_id` predicates. 26.33.0 and 30.19.0 return the same disposition on
- all 64 (23 rewritten, 33 `LakeSqlForbiddenError`, 8 `LakeSqlSyntaxError`)
- and the rendered SQL of all 23 rewritten cases is byte-identical. Two cases
- differ in wording only and both remain refusals: `DETACH TABLE` now reaches
- the statement-kind check instead of failing to parse, and `KILL`'s class
- moved to `sqlglot.expressions.ddl`, which the parse error quotes. The
- in-tree suites pass 58/58 on each major.
+ The routes also return `403` to a caller who belongs to no operator
+ organisation, where they previously returned an empty list to anyone
+ authenticated.
- The forward leg is the part that needed a decision. It read `>=27,<31`,
- which only reached past the pin while the pin sat below 27; there is no
- sqlglot 31, so with the pin on the 30 line every way of naming a ceiling is
- worse than leaving it off — `>=31` cannot be installed and reds the job
- forever, `>=30,<31` duplicates the shipped leg and reports two greens for
- one piece of evidence, and retargeting to `>=27,<30` tests majors below the
- pin, which is regression coverage for versions nothing installs and leaves
- nothing between the next major and production. The leg is now unbounded
- above, so it resolves whatever is newest and arms itself the day a higher
- major is published with no edit. Because that means it re-runs the shipped
- version today, a `Forward coverage` step emits a warning annotation saying
- so rather than letting a duplicate green pass for evidence, and
- `check_sqlglot_pin.py` gained the structural half in both directions: the
- forward leg must carry no upper bound and must not float below the shipped
- floor, and the step's `SHIPPED_CEILING` must equal the shipped range's `<N`
- bound so a stale copy cannot report the wrong verdict. Seven injected
- violations in `--self-test`, which runs before the gate. What is
- deliberately not asserted is that a higher major exists: upstream's release
- schedule is not this repository's to require, and a gate red until sqlglot
- ships 31 would be red for a reason no change here could fix.
+ Nothing in `apps/web` consumes these three routes; it calls
+ `/mssp/children`, which is unchanged.
-- **Four root-workspace dependency bumps landed as one lockfile change.**
- `cytoscape` 3.34.0 → 3.34.3, `lucide-react` 1.28.0 → 1.48.0,
- `@modelcontextprotocol/sdk` 1.29.0 → 1.30.1 and `tsx` 4.23.1 → 4.23.15 all
- resolve through the single root `pnpm-lock.yaml`, so merging them
- separately makes each one rebase the next until they all report
- `CONFLICTING`. Regenerated once from the four manifests together: 47
- insertions against 56 deletions, versus 1,130 insertions across the four
- individual proposals, because each of those re-churned the same peer keys.
- The `esbuild` resolutions are byte-identical before and after — `0.25.12`
- and the `vite>esbuild` / `tsup>esbuild` / `bundle-require>esbuild`
- override's `0.28.1` — so the `tsx` bump does not reach Turbopack's bundler,
- which is the reason those overrides are scoped to named parents rather than
- applied workspace-wide.
+- **Two `ActionType` members are removed: `add_ioc_to_blocklist` and
+ `run_playbook`.** Neither had an executor. `POST /actions` accepted both and
+ answered `No executor found for action type`, which reads as a broken
+ deployment rather than a verb nobody built; it now rejects them at
+ validation. A verb with no implementation path should leave the surface
+ rather than sit on it dead.
-- **Three more root-workspace bumps landed as one lockfile change.**
- `@testing-library/user-event` 14.6.4 → 14.6.7, `autoprefixer` 10.5.2 →
- 10.6.1 and `zustand` 5.0.14 → 5.0.15, regenerated once from the three
- manifests together: 19 insertions against 22 deletions, versus 636
- insertions across the three individual proposals. The only transitive
- movement is `caniuse-lite` 1.0.30001803 → 1.0.30001810, which
- `autoprefixer` carries as a data table. Verified on the merged result
- rather than on the three green ticks, because none of these three has a CI
- job that exercises what it changes beyond the shared web gates:
- `pnpm install --frozen-lockfile`, `eslint .` (0 errors), `tsc --noEmit`,
- `vitest run --coverage` (62.17% statements, gate green), `next build` and
- `storybook build` all pass.
+ Why neither could be preserved by implementing it:
-- **Five more root-workspace bumps landed as one lockfile change.**
- `@vitest/coverage-v8` 4.1.10 → 4.1.11, `axe-core` 4.11.4 → 4.13.0,
- `react-hot-toast` 2.6.0 → 2.6.1, `framer-motion` 12.43.0 → 13.4.0 and
- `zod` 4.4.3 → 4.6.5 (the last in `services/mcp`), regenerated once: 61
- insertions against 84 deletions, versus 1,040 insertions across the five
- individual proposals.
- The `@vitest/coverage-v8` bump is the first pull request the new `vitest`
- group produced, and it closes the exact-version peer drift that group was
- added for — `pnpm install` stops reporting
- `unmet peer @vitest/[email protected]: found 4.1.10`.
- `framer-motion` is a major and what installs is **13.4.3**, not the 13.4.0
- in the proposal's title, because `^13.4.0` resolves above it. Verified on
- 13.4.3 rather than on the proposed number: `tsc --noEmit` passes, which is
- the gate that would see a removed API, and this app's entire framer-motion
- surface is `motion`, `AnimatePresence` and `useReducedMotion` across 37
- import sites. v13 drops the optional `@emotion/is-prop-valid` peer that
- v12 declared, and this workspace does not use emotion. 615 tests pass with
- zero errors and coverage unchanged to the digit (62.17% / 57.71% / 54.77% /
- 64%), `next build` generates all 104 static pages, and `storybook build`
- succeeds.
+ - `add_ioc_to_blocklist` was a second name for `block_ioc`, which has a
+ Defender arm, a capability contract, a registered adapter and a place in
+ the vocabulary. Two names for one verb means half the callers reach the
+ dead one. Use `block_ioc`.
+ - `run_playbook` is the wrong shape for this registry rather than a missing
+ feature. Playbook execution lives in `services/agents` and always has, and
+ the contract in this service belongs to the *verb* — "run an arbitrary
+ bundle of verbs" has no verb-level impact, reversal or verification probe
+ to declare. Approving it once would execute whatever steps it contained
+ without each step meeting its own contract, which is precisely what the
+ per-capability contract exists to prevent. Playbooks already dispatch step
+ by step through this service, so every step is graded on the way past.
-### Fixed
-
-- **Dependabot proposed Expo SDK 57 packages for an SDK 54 app, from an entry
- that was never meant to see them.** `pnpm-workspace.yaml` excludes
- `apps/mobile` with a `!apps/mobile` negation and pnpm honours it — the root
- lockfile has no `apps/mobile` importer. Dependabot's manifest scan does
- not: it expands `apps/*`, finds `apps/mobile/package.json` and proposes
- updates for it from the root `/` entry, where none of the Expo holds
- written into the dedicated `/apps/mobile` entry apply. The result was
- `expo-constants 18.0.14 -> 57.0.19` — a release-train version, not a
- package version — carrying no lockfile change at all, since from the root
- entry's point of view the lockfile is the root one and the package is not
- in it. `pnpm install` resolves it to `expo-router 6.0.24 unmet peer
- expo-constants@^18.0.13`, while `tsc --noEmit` stays green, so the mobile
- job would have passed. Fixed with `exclude-paths: ["apps/mobile/**"]` on
- the root entry, which is version-updates-only and so leaves that
- directory's security alerts with the entry that owns them.
-
-- **The `services/realtime` Dependabot entry let the eslint family arrive one
- package at a time.** `@eslint/[email protected]` declares
- `peerDependencies: { eslint: "^10.0.0" }`, so bumping it alone against this
- service's `eslint@^9` installs a tree `npm ls` reports as `invalid` in nine
- places — and nothing else notices: `npm ci`, `tsc`, `eslint src test` and
- the test run are all green on it, and the effective rule set grows by three
- rules rather than shrinking. Unlike the root workspace this service does
- not use `eslint-plugin-react`, and `[email protected]` already
- accepts `eslint@^10`, so the family is grouped rather than held: it can
- move whenever all of it arrives in one reviewable pull request.
-
-- **The root entry's holds are written per dependency name, so a breaking
- major of anything not already on the list still arrives as an ordinary
- weekly bump.** `vitest` 4.1.11 → 5.0.1 was the demonstration. Every
- `ignore` in that entry names a specific package someone had already been
- burned by — `eslint`, `typescript`, `storybook`, `@storybook/*` — and
- nothing holds majors as a class, so "stop proposing breaking majors"
- described four names rather than a rule. Measured on the proposal: all 59
- test files and all 615 tests passed, then the run failed with 59 unhandled
- rejections reading `TypeError: Expected string coverage payload, received
- object` from `V8CoverageProvider.onAfterSuiteRun`, and a coverage report of
- `All files | 0 | 0 | 0 | 0`. vitest 5 changed the V8 coverage payload
- shape and `@vitest/coverage-v8` stayed on 4.1.10, because its own manifest
- range was still satisfied and it peer-requires `vitest` at an *exact*
- version rather than a range. Separately, the root `pnpm.overrides` pins
- `@vitest/mocker` to `>=4.1.11 <5` while `[email protected]` depends on
- `@vitest/[email protected]`, so the override wins silently and the proposal's
- lockfile carries a vitest 5 runtime with a vitest 4 mocker. Held at the
- major for `vitest` and `@vitest/*`, and grouped within the major so the
- exact-version peer pair stops drifting — `pnpm install` on the current
- lockfile already reports `unmet peer @vitest/[email protected]: found
- 4.1.10`, which is survivable inside a major and is why nobody had noticed.
-
-- **`update-types: ["version-update:semver-major"]` cannot hold a 0.x
- dependency, so the Storybook hold covered every member of the family
- except the one that needed it.** For a pre-1.0 package the breaking change
- arrives in the minor slot, and `@storybook/test-runner` is the only
- pre-1.0 member: 0.23.0 declares
- `peerDependencies: { storybook: "^0.0.0-0 || ^8.2.0 || ^9.0.0" }` and
- 0.24.5 declares `"^0.0.0-0 || ^10.0.0 || … || ^11.0.0-0"`. Its version line
- is independent of Storybook's the way the Expo packages' are, but its peer
- tracks the Storybook major exactly, so 0.24 is the Storybook 10 line of
- the package and installs against this workspace's `[email protected]` as an
- unsatisfied peer. Nothing in CI could catch it — no workflow runs
- `test-storybook`, and the only Storybook job runs `build-storybook`, which
- never loads the runner — so the proposal was green on all 22 required
- checks while migrating the whole jest tree 29 → 30 underneath it. The
- minor channel is now held for that package alongside the family's major.
-
-- **The `services/api` pip entry had no holds, against two ratchet gates.**
- `ruff` is declared in fourteen files (a count
- `scripts/check_dependency_pins.py` enforces agreement across) and `mypy` in
- six, plus `.github/workflows/ci.yml` for both, and Dependabot can only edit
- one of them.
- Measured rather than assumed: `ruff` 0.16.8 reformats 90 files and reports
- 3 lint errors where the pinned 0.4.10 reports `All checks passed!` and
- `1229 files already formatted`; `mypy` 2.3.1 makes
- `scripts/check_mypy_baseline.py` exit 1 with 35 `(tree, file, code)`
- entries no longer matching. Both failures land on the required
- `Python — Lint & Type-check` check, which is to say on every open pull
- request rather than on the bump's own. Held at the majors (and, for `ruff`,
- the minors, since 0.4 → 0.16 is a minor step under semver while 0.4.x
- patches still flow). `sqlglot` is deliberately *not* held: an `ignore` rule
- suppresses security updates as well as version updates, and a single-path
- sqlglot bump already cannot merge because `scripts/check_sqlglot_pin.py`
- fails closed when the seven install paths disagree.
-
-### Security
-
-- **`apps/mobile` resolved a vulnerable `image-size` that the workspace had
- already fixed.** Two high-severity advisories — GHSA-5p2g-fcmc-qvqq
- (CVE-2025-71329, JXL/HEIF parsers) and GHSA-w3rx-r6r6-pgpr
- (CVE-2025-71330, ICNS parser), both infinite-loop denial of service, both
- fixed in 2.0.3 — stayed open against `apps/mobile/pnpm-lock.yaml` for a
- structural reason rather than an unfixed one. The repository root pinned
- `image-size` to `>=2.0.4 <3` through a pnpm override, and `apps/mobile` is
- a deliberately independent install root with its own
- `pnpm-workspace.yaml` and lockfile, created that way so its install would
- stop rewriting the root lock. The override could not reach it and nothing
- compared the two. Resolved by declaring the same override in
- `apps/mobile/package.json`; the lockfile now resolves `[email protected]`
- and drops its `queue` dependency with it.
-
- Metro is the only consumer, and no version of Metro that depends on
- `image-size` permits 2.x — every one declares `^1.0.2`, which is why
- Dependabot recorded `security_update_not_possible` and failed its job
- instead of opening a pull request. Metro 0.83.3's two call sites go
- through `_interopRequireDefault(require("image-size")).default`, and
- `[email protected]`'s CommonJS build still exports a callable `default`, so
- `metro.getAssetSize` returns correct dimensions against it.
-
-### Added
-
-- **The Attack Graph says when it is showing a truncated graph.**
- `GET /api/v1/graph` bounds itself to 400 nodes and 900 edges and reports
- `truncated`; the console rendered the flag nowhere, so a graph cut at the
- ceiling and a graph that is genuinely that size looked identical. On an
- attack graph that difference changes a conclusion — a missing edge reads
- as a lateral path that does not exist. The view now shows a
- `role="status"` notice naming the counts, the depth and the ceiling that
- produced them, with a depth control that issues a new query rather than
- re-rendering the cached one. The three existing states are unchanged: an
- empty tenant still reads as empty, an unreachable backend still names the
- endpoint and the status, and neither is reported as a truncation.
-
-- `GraphOverviewResponse` gains `nodeLimit` and `edgeLimit`, read from
- `graph_service` rather than restated, so the ceiling a client reports is
- the ceiling the service applied. Additive and defaulted; no existing
- consumer changes.
-
-### Fixed
-
-- **`main` went red because an install list depended on an extra it never
- declared, and an upstream release stopped supplying it by accident.** The
- `Python — Service unit tests (fusion, honeytokens, purple-team)` job failed
- on `purple-team`'s `test_every_api_route_requires_auth` with
- `ModuleNotFoundError: No module named 'greenlet'` — 99 other tests in the
- job passed. Nothing about `purple-team` had changed: the test has imported
- `app.api.routes` since it was written, that module has always imported
- `sqlalchemy.ext.asyncio`, its `pyproject.toml` has always declared
- `sqlalchemy[asyncio]`, and its `poetry.lock` resolves `greenlet 3.5.6`. The
- declaration was right and the install path was wrong: this job consults
- neither the manifest nor the lock, it pip-installs a hand-curated list, and
- that list named a bare, unbounded `sqlalchemy`.
-
- It passed for months anyway. Through SQLAlchemy 2.0.x, `greenlet` was
- required *outside* the `asyncio` extra whenever `platform_machine` matched
- one of `aarch64 | ppc64le | x86_64 | amd64 | AMD64 | win32 | WIN32` — true
- on `ubuntu-latest` — so a bare `sqlalchemy` installed it incidentally and
- the extra was load-bearing and unnamed at the same time. SQLAlchemy 2.1.0
- removed that clause, leaving `greenlet>=1; extra == "asyncio"` as the only
- requirement. 2.1.0 was published at 20:12:49 UTC on 2026-09-24; the last
- green commit on `main` (`ba0c6429`) is timestamped eleven minutes before it
- and the first red one (`ad775e8d`, #829) ten minutes after. #829 was an
- LLM-routing change that touched no file under `services/purple-team` and
- nothing in that import chain — it is the commit whose run happened to
- re-resolve first, not the cause. So this was neither a dependency that went
- missing nor an import path newly reached; it was an unpinned install
- re-resolving across an upstream minor.
-
- The job now installs `"sqlalchemy[asyncio]>=2,<3"`, matching both the
- manifest and the wave-2 matrix job, which had it right all along.
- `greenlet` is deliberately *not* added as a top-level pin: the extra exists
- to pull it, and naming the transitive package instead would record the
- workaround rather than the dependency. `isolation.yml`, which had done
- exactly that — bare `sqlalchemy` plus an explicit `greenlet` — now declares
- the extra and drops the compensating entry.
-
-- **`services/connectors` reached `sqlalchemy.ext.asyncio` without declaring
- the extra that makes it importable.** `app/db/engine.py` calls
- `create_async_engine`, and the manifest declared `sqlalchemy = "^2.0.0"`.
- It resolved only because of the same pre-2.1.0 accident, which means the
- next SQLAlchemy bump would have dropped `greenlet` from the lock the image
- installs from and broken the poller in production rather than in CI. It was
- the only service in this position — every other service that imports the
- module already declared `[asyncio]`. Re-locking with the extra changes two
- lines and no resolved version.
-
-- **`scripts/check_dependency_pins.py` now compares extras, not just
- version ranges.** `sqlalchemy` and `sqlalchemy[asyncio]` are two different
- dependency sets, and the gate that exists to assert every install path
- agrees was reading the name and the range and discarding the extras — so it
- reported agreement between a manifest and a workflow installing strictly
- less software. Extras are now part of a declaration's identity in both
- syntaxes (PEP 621 brackets and Poetry's `extras = [...]` table, the latter
- being the one that was silently dropped), with three directions checked
- because the direction nobody aims a gate at is the one that rots:
- `manifest -> install path` (a workflow or image dropping an extra a
- manifest declares), `source -> manifest` (code importing the module an
- extra enables under a manifest that does not declare it, read out of the
- source so the manifest is not asked to vouch for itself), and
- `extra -> lock` (an extra declared while the lock resolved nothing it
- provides — "the extra is declared and the library is absent" is now a
- sentence this gate can say). Run against the pre-fix tree it names all
- three defects above and the files holding them; `--self-test` injects each
- direction separately, and six tests in
- `tests/test_dependency_pin_gate.py` pin the parsing and the directions.
-
-- **`scripts/check_toolchain_pins.py` compares every Node install root
- against every other.** There are four — the workspace root, `apps/mobile`,
- `services/realtime` and `services/mcp/cursor-extension` — and the gate read
- dependency resolution out of the first one only, so the `image-size` split
- above was invisible to it. It now discovers install roots structurally,
- reads both pnpm and npm override spellings and both lockfile formats, and
- checks in both directions: an override declared in one root against what
- every other root resolved, and each root's own lockfile against its own
- manifest. Exemptions are keyed on the exact versions they were verified
- against, so a bump re-opens the question rather than inheriting the
- clearance. Running it against the pre-fix tree names the defect and the
- file; running it over a lockfile it cannot parse, or a tree with no
- install root, fails rather than reporting a clean comparison it never
- performed. The gate found a third instance of the same class on first run
- (`ws` 6.2.6/7.5.13 in `apps/mobile`, verified clean against OSV and
- recorded).
-
-- The workspace-wide `esbuild` override ban now applies to every install
- root instead of the repository root alone. `apps/mobile` was the one place
- the mistake that broke Turbopack's font import map could have been
- reintroduced without anything noticing, because its bundler is Metro and
- the damage would surface in a different workspace.
-
-- `.github/dependabot.yml`'s `apps/mobile` entry gains the `typescript`
- semver-major hold that the `/` and `/services/realtime` entries already
- carry. Its CI gate is `tsc --noEmit`, so it was the only Node install root
- running `tsc` without the hold — the same shape as the `image-size`
- finding, a decision taken for the workspace that never reached the root
- installing separately.
-
-### BREAKING
+ The actions service's `ActionType` is not part of `docs/openapi.yaml`, so
+ the breaking-change gate does not see this; it is recorded here because a
+ dropped enum value is a break whether or not a workflow notices.
- **`CostTracker` no longer reports an estimated dollar figure as a measured
one, and `CostTracker.total_cost_usd` is gone.** It priced a call by looking
@@ -636,236 +302,57 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
left in place rather than deleted. Gated by
`scripts/check_cost_provenance.py`.
-- **Three MSSP response schemas describing fabricated data are removed:
- `MSSPKpiOverview`, `ManagedTenantRow`, `CrossTenantIncident`.** They were
- the shape of five hardcoded companies with invented alert counts, not the
- shape of anything the platform measured. `/mssp/overview`,
- `/mssp/tenants` and `/mssp/incidents` keep their paths and now return
- `PortfolioSummaryOut`, `PortfolioTenantOut` and `PortfolioAlertOut`,
- computed from real rows.
+### Added
- What moved, and why it could not be preserved:
+- **Two-way SIEM integration: an AiSOC verdict is written back onto the
+ finding that produced the alert.** The integration only ever ran inbound. A
+ Splunk notable became an alert, the agent triaged it, and the notable sat in
+ the Splunk queue untouched — so an analyst re-read a finding AiSOC had
+ already dismissed, and a finding AiSOC had confirmed waited its turn behind
+ them. Nothing on `main` wrote a disposition back to any SIEM in any form.
- - `health_score` / `avg_health_score` are **gone**, not nulled. It was an
- undefined composite with no formula anywhere in the tree; keeping the
- field would promise a measurement that does not exist. (`MetricsOut` on
- the separate `/mssp/metrics` route still carries a `health_score` and is
- unchanged by this release.)
- - `avg_mttr_minutes` → `mttr_minutes`, measured from cases a tenant
- actually closed in the trailing 30 days, and **null** when it closed
- none. The old value was the literal `23.4`. Present on both the per-tenant
- row and the portfolio summary, where it averages only over tenants that
- closed something rather than counting a null as a zero.
- - `sla_breach_count` and `sla_breaches` → `sla_breached_cases`, counted
- from `cases.sla_breached`.
- - `connectors_online` / `connectors_degraded` / `connector_status` →
- `total` / `healthy` / `stale` / `error`, derived from each connector's
- `health_status` and `last_sync`. Nested under a `connectors` object on
- the per-tenant row; flat `connectors_*` fields on the summary.
- - `tenant_id` is now a real tenant UUID rather than a string like
- `"t-acme"`.
- - `assignee` is gone from the incident rows. It named invented analysts;
- an alert's real owner is `case_id`, which is now returned instead.
- - New: `synthetic_alerts`, so seeded demo rows are counted apart from a
- tenant's real posture instead of inflating it.
-
- The routes also return `403` to a caller who belongs to no operator
- organisation, where they previously returned an empty list to anyone
- authenticated.
-
- Nothing in `apps/web` consumes these three routes; it calls
- `/mssp/children`, which is unchanged.
-
-- **Two `ActionType` members are removed: `add_ioc_to_blocklist` and
- `run_playbook`.** Neither had an executor. `POST /actions` accepted both and
- answered `No executor found for action type`, which reads as a broken
- deployment rather than a verb nobody built; it now rejects them at
- validation. A verb with no implementation path should leave the surface
- rather than sit on it dead.
-
- Why neither could be preserved by implementing it:
-
- - `add_ioc_to_blocklist` was a second name for `block_ioc`, which has a
- Defender arm, a capability contract, a registered adapter and a place in
- the vocabulary. Two names for one verb means half the callers reach the
- dead one. Use `block_ioc`.
- - `run_playbook` is the wrong shape for this registry rather than a missing
- feature. Playbook execution lives in `services/agents` and always has, and
- the contract in this service belongs to the *verb* — "run an arbitrary
- bundle of verbs" has no verb-level impact, reversal or verification probe
- to declare. Approving it once would execute whatever steps it contained
- without each step meeting its own contract, which is precisely what the
- per-capability contract exists to prevent. Playbooks already dispatch step
- by step through this service, so every step is graded on the way past.
-
- The actions service's `ActionType` is not part of `docs/openapi.yaml`, so
- the breaking-change gate does not see this; it is recorded here because a
- dropped enum value is a break whether or not a workflow notices.
-
-### Fixed
-
-- **The LLM gateway moves from the `full` profile into CORE, so the documented
- default install can actually do AI triage.** It was `full`-profile on the
- reasoning that the gateway is "only needed when a provider key is
- configured" — which skipped a step: every task role resolves to an
- `aisoc-<role>` alias, and an alias resolves at the gateway and nowhere else,
- so a CORE deployment could not use a key either. `make up`, the quickstart's
- Path B, and a plain `docker compose up -d` all now start it.
-
- With no provider key it boots, serves its seven aliases and answers
- `/health/liveliness` (verified against the bundled config with both provider
- keys empty); AiSOC makes no LLM call and the deterministic path is
- unchanged. CORE goes from 10 to 11 services and ~6 GB to ~6.5 GB — the image
- is 1.67 GB and idles at 451 MiB, measured, so "one lightweight container"
- was not true and the README's numbers moved rather than staying put. The
- gateway is also the only party that can report what a call cost, so CORE's
- cost figures were previously unmeasurable by construction. Reasoning and the
- two rejected alternatives: `docs/decisions/0006-llm-gateway-in-core.md`.
- While correcting that table, the `full` row's service count was checked and
- was wrong: `make up-full` starts **21** services, not 30 (30 is `full` plus
- the `monitoring`, `chatops`, `extras` and `osquery` profiles).
-
-- **`Event spine (real containers)` had a genuine race, not a flaky
- environment.** The graph-writer assertion was
- `docker compose logs … 2>/dev/null | grep -q "<early string>"` inside a step
- running under `set -euo pipefail`. `grep -q` exits the instant it matches —
- and that string is line 4 of the service's output — which closes the pipe
- while Compose is still writing; Compose exits 255 on the broken pipe and
- `pipefail` promotes that over grep's 0. The pipeline reported failure over a
- line that was present, which the failing run proves: the log dump its own
- error handler printed 115 ms later contains the exact string.
-
- Reproduced deterministically rather than inferred — with a large log the
- pipeline returns 255 every time, with a small one that fits the pipe buffer
- it returns 0 every time, and the match is in both. That size dependence is
- why it read as a flake. `2>/dev/null` made it worse by discarding the only
- message that distinguished "Compose failed" from "the line is absent", so
- the run reported the wrong cause. Fixed by capturing the logs once and
- grepping the capture, and by checking Compose's own exit status separately —
- not by a retry or a longer sleep, which would have converted a real race
- into a slower real race. `scripts/doctor.sh` has the same shape twice and is
- unaffected: it does not set `pipefail`.
-
-- **The dependency audit only ever looked at one of the repository's two pnpm
- install roots.** `run_pnpm_audit` ran `pnpm audit` in the repo root and
- nowhere else, so `apps/mobile` — a deliberately separate install root, with
- its own `pnpm-workspace.yaml` so its installs stop rewriting the root lock —
- was never scanned. Two high-severity advisories stood open there while the
- `security-audit` job reported a clean workspace, which is the same shape as
- the stale `poetry.lock` that once dropped `services/slack-bot` and hid eleven
- advisories. Install roots are now discovered from the tree (every directory
- holding a `pnpm-lock.yaml`), each is audited, and a finding names the root it
- came from rather than a generic "pnpm workspace". A tree with no lockfile
- anywhere is recorded as a coverage gap instead of reported as clean.
-
- The two advisories it surfaced (`GHSA-5p2g-fcmc-qvqq`, `GHSA-w3rx-r6r6-pgpr`
- in `image-size`) are suppressed with an expiry rather than fixed, because no
- fix is reachable: `apps/mobile` resolves `image-size 1.2.1` solely through
- `metro 0.83.3`, no patched 1.x exists, every metro release through 0.87.1
- still requires `^1.0.2`, and `image-size` 2.x drops the callable default
- export `metro/src/Assets.js` calls — so an override would break asset
- resolution at bundle time rather than bump a version. Dependabot's updater
- reaches the same verdict independently (`security_update_not_possible`).
- Both are denial-of-service only and build-time, reachable only from an image
- already committed to this repository.
-
-- **AI triage could not reach a model in the default deployment, and it was
- not a missing key.** `docker-compose.yml` set `LLM_GATEWAY_URL` on the `api`
- and `agents` services. Nothing read it. Both resolvers honoured only
- `OPENAI_BASE_URL`, which compose never set, and each carried a comment
- explaining that the refusal was deliberate — routing through the gateway
- should be an explicit choice so it is never ambiguous whether the bearer is
- the gateway master key or a provider key. The reasoning was sound and the
- result was that every `aisoc-<role>` alias went to `api.openai.com`, which
- cannot resolve one, and the caller's `except` rendered that as "no LLM
- available". `preflight_llm()` logged exactly this condition at boot with the
- remedy, so the diagnosis existed and the wiring did not.
-
- `LLM_GATEWAY_URL` is now the lowest-precedence base URL, and the ambiguity is
- resolved rather than avoided. **The model decides the route:** the gateway is
- adopted only for a gateway alias, which resolves nowhere else, while a
- concrete `AISOC_MODEL_PIN_<ROLE>` — the documented direct-to-provider escape
- hatch — is left pointing at its provider. **The route decides the bearer:**
- when AiSOC picks the gateway itself it sends `LITELLM_MASTER_KEY`, which
- compose now supplies to both services, and never a provider key. An explicit
- `OPENAI_BASE_URL` still outranks both. The rule lives once, in
- `services/agents/app/llm/routing.py`, shared with the BYOK resolver and
- mirrored in `services/api/app/services/model_aliases.py` for the service that
- cannot import it.
-
- Consumers audited and fixed alongside it: the API copilot and the NL-query
- translator (both already went through the shared helpers); the agents
- **contextual copilot**, which built `ChatOpenAI(model=…)` with no base URL at
- all and reported "LLM not configured" from a key check that could not see the
- gateway's; the **detection builder** (`/nl-detection`), the only consumer that
- could not be pointed anywhere — it POSTed to a hardcoded
- `https://api.openai.com/v1/chat/completions` with `OPENAI_MODEL`; the four API
- endpoints (translation, hunts, knowledge base, phishing) that read
- `LLM_BASE_URL` and defaulted to OpenAI without consulting `OPENAI_BASE_URL`;
- the BYOK / "explain this alert" resolvers in both services, which defaulted
- straight to `https://api.openai.com` and took an alias there; and
- `GET /llm/status`, so the indicator describes the route the explain path
- actually takes. The **MITRE RAG embedding** path is a deliberate exclusion —
- `infra/litellm/config.yaml` declares chat aliases only, so an embedding call
- routed there would 400 on every batch. It has its own
- `AISOC_EMBEDDING_BASE_URL` / `AISOC_EMBEDDING_MODEL` pair and the gate checks
- the exclusion in both directions.
-
-- **`OPENAI_MODEL` replaced the triage role's model on the highest-volume path
- in the product.** `.env.example` shipped `OPENAI_MODEL=gpt-4-turbo-preview`,
- a model no gateway config in this tree has ever defined. The auto-triage
- worker layers a tenant's BYOK configuration over the role pin — but the
- resolver's `LlmConfig` collapsed provenance into a single `source` word, so
- the worker could not tell a per-tenant override from the process-wide env
- baseline and bound all three fields unconditionally. Once traffic reached the
- gateway that would have produced `Invalid model name` on every triage call:
- the failure the commercial deployment already hit, where the copilot and every
- triage agent degraded to empty output.
-
- `LlmConfig` now carries `model_from_tenant` / `base_url_from_tenant` /
- `api_key_from_tenant`, computed from state the resolvers already tracked, and
- only a field the tenant actually set is treated as an override. A real BYOK
- model still wins. `.env.example` ships `OPENAI_MODEL=aisoc-summary` — an alias
- the bundled gateway defines — and the variable is documented as the BYOK /
- explain path only, never a task role.
+ New capability `update_alert_disposition` with five vendor arms (Splunk ES,
+ Elastic Security, Microsoft Sentinel, IBM QRadar, Microsoft Defender), two
+ new clients (`sentinel_client.py`, `qradar_client.py`), migration
+ `057_alert_source_links.sql`, and `POST /api/v1/alerts/{id}/source-writeback`.
+ The agents triage worker posts to that route after the verdict is durable and
+ fails soft — the verdict outranks its writeback, and an unreachable Splunk
+ must not dead-letter an alert that was triaged correctly. Docs:
+ [Integrations → SIEM writeback](apps/docs/docs/integrations/siem-writeback.md).
-- **A model with nowhere to go is now an error instead of a fallback.**
- `make_chat_model` and `chat_completions_url` raise `UnroutableModelError`,
- naming the alias and the remedy, when a gateway alias is requested with no
- gateway configured — rather than building a client destined to 404 and
- letting a caller read that as "the LLM is unavailable". `run_auto_triage`
- constructs its model inside its own `try` so the failure arrives as
- `AutoTriageError` like every other LLM failure; `auto_triage_node` catches
- that specifically, so constructing outside it would have failed a whole graph
- run over a configuration problem the deterministic path handles. The
- deterministic floor is unchanged and still reports `call_count=0 models=[]
- total_tokens=0` with no `recommended_actions`. `preflight_llm()` now reports
- both directions at boot: an alias with no gateway, and a concrete model
- pointed at the bundled gateway.
+ **The disposition mapping is the safety argument, not the approval tier.** A
+ confirmed true positive is *escalated and never closed*: it is the finding a
+ human most needs to see, and closing it because the platform is confident is
+ how an agent turns a real intrusion into a resolved ticket nobody read. An
+ unknown verdict is *refused, never guessed* — which matters because
+ `normalize_disposition` defaults an unrecognised string to `true_positive`,
+ so a mapper that normalised first would convert "I do not recognise this"
+ into a confident claim. Only benign and false-positive verdicts may close a
+ finding, and `benign_true_positive` is classified as a correct detection of
+ authorised activity rather than as a false positive, so it never inflates a
+ rule's own FP rate.
-### Added
+ **Governance ships dry-run by default.** `AISOC_SIEM_WRITEBACK_ENABLED`
+ defaults on and `AISOC_SIEM_WRITEBACK_EXECUTE` defaults **off**, so an
+ operator opts in to writing into their own SIEM. Anything that is not an
+ explicit yes — including a typo — is read as a dry run. `executed` is the
+ single field that means a vendor was touched; it is carried on the API
+ response, on the worker's return value, and as a no-default column on
+ `alert_source_links`, so a dry run, a refusal and a credential-less
+ simulation can never be read as a write that happened. Projecting a closing
+ verdict onto a linked Jira / ServiceNow ticket needs a third flag
+ (`AISOC_SIEM_WRITEBACK_CLOSE_CASE`, off) because the ITSM connectors project
+ a status transition rather than a note, so the only truthful way to reach the
+ ticket is to resolve the case for real.
-- **A model name must resolve, and the variable that routes it must be read —
- gated in both directions.** `scripts/check_llm_model_routing.py` reconciles
- `infra/litellm/config.yaml`, the role pins, the API-side mirror,
- `.env.example` and `docker-compose.yml`: every pin's alias must be defined by
- the gateway and every alias claimed by a pin; every model named in an env or
- compose file that wires the bundled gateway must be one the gateway defines;
- every variable compose points at the gateway must be read by *both* resolvers
- and every gateway variable the resolvers read must be set by compose; and a
- service handed the gateway URL must be handed the gateway key. Wired into
- `ci.yml :: python-lint`, with the self-test running first.
+- **The join key survives ingest.** `finding.uid` — the vendor's own id for a
+ finding — reached the OCSF envelope and was then discarded, so by the time a
+ row hit `alerts` the notable's rule UID, the Elastic signal id and the QRadar
+ offense id were gone and no verdict could be aimed at anything. Fusion now
+ carries it onto `alerts.external_id` and writes an `alert_source_links` row,
+ but only for a vendor with a writeback arm: a link to a system AiSOC cannot
+ write to would read as a two-way integration that is not one.
- Every parser is AST- or comment-aware, and the self-test proves each refusal
- rather than asserting it, because all three corpora contain text shaped
- exactly like what the gate looks for: the gateway config's commented
- Ollama/vLLM examples repeat `model_name: aisoc-triage` verbatim,
- `.env.example` documents the escape hatch as a commented
- `AISOC_MODEL_PIN_TRIAGE=gpt-4o-mini`, and `docker-compose.yml` explains the
- gateway in prose containing `OPENAI_BASE_URL=http://litellm:4000/v1` two lines
- above the key it describes. A line regex credits all three. 13 injected
- defects and 7 parser blind spots, each caught by its own code, plus the shared
- empty-tree refusal.
- **`GET /api/v1/graph` exists.** The console's Attack Graph view has always
called it and always got a 404, while `/graph/neighbors/…` and
`/graph/mitre-coverage` returned 200 from the same Neo4j instance holding
@@ -895,421 +382,451 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
the view holds no graph at **first paint**, so a future `fallbackData` (which
disables revalidation, making a mock permanent rather than provisional)
fails the build.
-- **`scripts/check_orm_migration_parity.py` — a column a model declares and
- no migration creates now fails the build.** Structural on both sides
- (Python `ast` over the models and the alembic revisions, plus the raw SQL
- those revisions and `services/api/migrations/*.sql` execute) and in both
- directions: `missing-in-migration`, `missing-in-model`,
- `type-narrower-in-migration`, `type-family-mismatch`, and
- `no-migration-source` for a mapped table no migration mentions at all —
- which is the finding that exists because pairing nothing produces zero
- findings, and zero findings over zero comparisons prints the same word as
- a clean result. `--credits` enumerates what each verdict was *based on*
- for the same reason.
-
- Scoped structurally rather than by a list: a service that calls
- `metadata.create_all` builds its tables from the models, so a column no
- migration creates is created anyway. `services/api` is the one such
- service and is reported as advisory with its count printed, not hidden.
- The four alembic-managed services have no such fallback, which is exactly
- why UEBA's missing column was fatal.
- Verified non-vacuous against the tree as it was: it reports all three of
- the real UEBA defects. The `type-family-mismatch` kind exists *because*
- an earlier version reported only two of them — a length comparison could
- not see `UUID` against `String(64)`, since neither side declared a length
- the other contradicted.
-
-- **`scripts/check_service_migration_bootstrap.py` — a migration chain
- nothing invokes is not a migration chain.** Asserts every service with an
- `alembic.ini` ships `app/_migrate.py`, keeps it byte-identical to the
- others, runs it from its `CMD`/`ENTRYPOINT` (matched on the command lines,
- so a mention in a comment does not count), and carries both an owner DSN
- and a compose healthcheck. The other direction too: a service shipping
- the module with no chain to apply would exit 1 on every start.
+- **The Attack Graph says when it is showing a truncated graph.**
+ `GET /api/v1/graph` bounds itself to 400 nodes and 900 edges and reports
+ `truncated`; the console rendered the flag nowhere, so a graph cut at the
+ ceiling and a graph that is genuinely that size looked identical. On an
+ attack graph that difference changes a conclusion — a missing edge reads
+ as a lateral path that does not exist. The view now shows a
+ `role="status"` notice naming the counts, the depth and the ceiling that
+ produced them, with a depth control that issues a new query rather than
+ re-rendering the cached one. The three existing states are unchanged: an
+ empty tenant still reads as empty, an unreachable backend still names the
+ endpoint and the status, and neither is reported as a truncation.
-- **`scripts/audit_health_probes.py` now checks the thirteen copies of
- `app/_health.py` are identical.** The module's own docstring claimed they
- were kept in sync and nothing verified it — they happened to be, so the
- claim was true by luck. It also pointed at `scripts/sync_health_module.py`,
- which does not exist in this tree.
+- `GraphOverviewResponse` gains `nodeLimit` and `edgeLimit`, read from
+ `graph_service` rather than restated, so the ceiling a client reports is
+ the ceiling the service applied. Additive and defaulted; no existing
+ consumer changes.
-- **Every test file in the tree is now executed by a workflow, and a gate
- holds it that way in both directions.** 63 were executed by nothing at all:
- 56 of the 82 under `services/agents`, because that invocation was a
- hand-maintained list of 26 file paths; five under the repository's own
- `tests/`, which was reached only as nineteen single-file invocations spread
- across eleven workflows; and both files under `scripts/tests/`, which no
- workflow named. A file nothing runs reports nothing, which is
- indistinguishable from a file that passed.
+- **`/graph` honours `?entity=` and selects the node it names.** Three callers
+ already built that URL — the Investigation Rail's entity chips, `HuntView`'s
+ "Pivot to graph" button, and now federated search — and nothing read it, so
+ every pivot navigated to the graph and dropped the entity, leaving the
+ analyst to find the node by eye. Both the typed form (`host:WIN-DC01`) and
+ the bare form `HuntView` emits are accepted, matching on node id then label.
- `scripts/check_test_discovery.py` derives the corpus from `git ls-files` and
- models reach the way pytest collects — directory arguments, `--ignore`,
- `conftest` `collect_ignore`, and `-m` marker deselection, since a file whose
- every test is deselected runs nothing however plainly it is named. An
- invocation naming a path that is not in the tree fails too. Quarantine is a
- shrink-only list where every entry carries a reason and an entry that has
- become reachable fails as stale, so "not run" can never again be silent.
+- **Federated SIEM search has a console surface.** `/api/v1/federated/backends`
+ and `/api/v1/federated/search` have fanned one query out to Splunk, Microsoft
+ Sentinel, Elastic and QRadar — in parallel, against each tenant's own
+ vault-encrypted credentials — for some time. `apps/web` had no client for
+ either, so the capability was reachable only from the SDK. `/federated-search`
+ now exposes it: pick which connected SIEMs to query, describe the search once
+ in free text plus optional `field operator value` filters, and read the merged
+ rows.
- Two blind spots in the gate's own parser were found by enumerating what it
- **credited** rather than what it flagged. A step that `cd`s into a service
- and then names `tests/` had its paths resolved against the repository root,
- which simultaneously reported all 82 agents files unreached and credited the
- root `tests/` tree with a package's invocation — the same defect in both
- directions at once. And `-m` was first treated as "a filter, not a
- collection rule", which would have credited a module in full under
- `-m "not integration"` while nothing in it ran.
+ The design constraint the page is built around is the endpoint's per-source
+ isolation. It deliberately never fails the whole call because one backend is
+ slow, 401s or 5xxs; it returns a verdict per backend instead. A UI that
+ renders only the merged rows discards that, and the analyst cannot tell
+ "Sentinel has nothing" from "Sentinel did not answer" — opposite conclusions
+ mid-incident. So the per-backend strip renders above the rows, always, with
+ each backend's own row count, latency and error message. A backend that
+ failed shows no row count, because "0 rows" beside a timeout reads as
+ "nothing matched". An empty result set is labelled "No matching events" when
+ a backend answered and matched nothing, and "No backend returned results"
+ when every backend failed.
- The invocations are directories now. `services/agents` runs `tests/`;
- `tests/` and `scripts/tests/` run as directories in `python-test`, with
- `tests/isolation/` left to `isolation.yml`, which owns its live stores.
+ Recognised entities in a row deep-link to `/graph?entity=<type>:<value>`.
+ Recognition is a fixed per-vendor field-name list rather than a heuristic:
+ `src_ip`, `source.ip` and `SourceIP` all resolve to the same IP pivot, while
+ `zip`, `recipient` and `description` resolve to none.
-- **`--suite <name>` in `scripts/run_evals.py` ran every suite.** Its `--help`
- said "runs that suite in isolation", the module docstring said "run a single
- suite by name", and `args.suite` reached nothing but the wording of the
- banner: the report dict called all eleven `_run_*` helpers inline. So
- `--suite mitre_accuracy` took the full runtime, graded eleven gates, and
- printed `PASS — mitre_accuracy green` — a verdict naming one suite and
- decided by eleven, with `--ci` able to fail an operator's single-suite
- bisection on an unrelated regression. The report now carries `suite_filter`
- and only the requested suites are called; the registry is asserted against
- `_SUITE_NAMES` so a name argparse accepts cannot be one no runner answers
- to. A substrate-import failure also exits **3**, which is what this file's
- own "Exit codes" block has always documented — it exited **2**, and 2 means
- "MITRE accuracy regressed against the baseline", so a fresh clone missing a
- dependency reported an accuracy regression, and the one actionable
- instruction (`pip install -e services/agents`) was not in the message.
- All three defects were pinned by tests in `scripts/tests/` that had been
- failing for as long as they existed, in the tree no workflow ran.
+- **A SOC operations dashboard at `/dashboards/operations`.** `/dashboard`
+ answers what is happening in the estate; this answers whether the machine
+ that reports it is working. The distinction matters because every failure
+ mode of a detection pipeline makes it *quieter* — a connector stops polling,
+ a schema changes and events bounce, a token expires — and an alert-centric
+ view reports all three as good news.
-- **No gate may report OK over a repository with no content, and it is a CI
- gate now rather than a probe somebody happened to run once.** Copying
- `scripts/` into an empty git repository and running every wired check there
- found five reporting OK over a tree holding nothing — the detection
- validator behind the rule count on the front page certifying zero rules as
- valid, a dashboard check that failed on an *empty* directory and passed on a
- *missing* one, `OK: 0 published Go module path(s)`, every self-link healthy
- over zero files, and a health-probe audit printing a table header and
- exiting 0. Those five were fixed; the probe that found them was a one-off,
- so the next gate written could reintroduce the defect freely.
+ Six panels, each backed by one endpoint and owning its own fetch, loading,
+ empty and error state: connector fleet staleness (`/health/fleet`), rejected
+ events by reason (`/health/dead-letters`), alert severity and disposition
+ (`/alerts/stats`), detection coverage counting only *enabled* rules
+ (`/detection/coverage`), agent runs/tokens/spend (`/costs/dashboard`), and
+ response actions paused for approval (`/approvals`). The first three had no
+ web client at all.
- `scripts/check_gate_contract.py` is that probe made permanent. It runs all
- 69 inventoried checks the way a workflow runs each one, inside a git
- repository holding only `scripts/`, and requires each to exit non-zero.
- Three things make the result mean something:
+ Fleet staleness is reported against each connector's own cadence — "3.2
+ intervals behind" rather than an absolute age, because a daily connector and
+ a five-minute connector are not late at the same wall-clock time. One dead
+ endpoint blanks one panel and names the failure; it does not take the page
+ down and does not substitute plausible figures.
- - **The inventory is not written here.** It comes from
- `check_gate_coverage.py`, which already resolves the workflow-to-check
- graph structurally. A second scanner would drift the first time either
- one learned something the other had not.
- - **A failure the empty tree did not cause is INCONCLUSIVE, not a pass.**
- An argparse usage error, or an import of a module the repository does not
- supply, means the gate was never exercised — crediting that as a refusal
- would be the same defect one level up. Whether a missing module is the
- repository's own is read from `git ls-files`, so `No module named 'app'`
- counts as the empty tree working and `No module named 'structlog'` does
- not.
- - **Every way CI runs a script is probed, and the worst result decides.**
- Four subcommands under one name are four gates; excusing the set because
- the first refused is how the others stay hidden. Only invocations
- carrying a declared verdict flag are probed, so the generator half of a
- script that is both generator and gate is not mistaken for one.
+- **A real operator organisation above tenants, so the MSSP console can be
+ backed by data instead of gated behind demo mode.** `/mssp/overview`,
+ `/mssp/tenants` and `/mssp/incidents` returned five hardcoded companies —
+ "Acme Corp, health 92.4, 12 open alerts", "Wayne Enterprises", invented
+ incidents with invented assignees — because the cross-tenant aggregation
+ query behind them was never written. Gating the sample behind demo mode
+ removed the lie but left the feature unbuilt: outside demo mode the
+ console showed zeros forever. Making it real was a schema change, not an
+ ungating.
- Exceptions are recorded with the disposition they are excused for and
- checked in both directions, so an entry naming a check that no longer
- exists fails, and so does one whose gate has started behaving differently.
- Four are recorded: `wet_eval_check.py` (reads two environment variables and
- no repository content), `security_audit.py`'s `validate-ignores` arm (its
- entire subject is a file inside `scripts/`, the one directory the scratch
- tree must keep), and `openapi_diff.py` / `wet_eval_update_benchmark.py`
- (both inputs named on the command line, one of them built outside the
- checkout).
+ Migration `058` adds `organizations`, `organization_members`,
+ `organization_tenants` and `organization_member_tenants`, and backfills
+ from `tenants.parent_tenant_id`, which keeps working. Three constraints
+ hold the boundary in the database rather than in whichever code path
+ writes a row: `organization_tenants` is unique on `tenant_id` so two
+ providers cannot claim one customer; `organization_member_tenants` has
+ composite foreign keys onto both the membership and the portfolio, so a
+ grant cannot name an unmanaged tenant and releasing a tenant revokes every
+ grant over it; and `organizations.home_tenant_id` cascades, so erasing a
+ provider's own tenant removes the organisation while leaving its customers
+ standing as unclaimed tenants.
-- **Every check resolves its repository root from git, and every check
- carries a `--self-test`.** Sixty of the sixty-nine derived a root from
- `Path(__file__).resolve().parent.parent` — whatever happens to sit two
- levels above the script — which is how a gate prints a confident OK about a
- tree it never opened, and only a deliberate run from another directory
- catches it. Fifty-four had no self-test at all. Both are now properties the
- contract gate enforces, with empty exception lists.
+ Roles carry two separate axes. `owner`/`admin` reach the whole portfolio;
+ `operator`/`viewer` reach only tenants granted to them, and **nothing**
+ when they have no grants — "no scope" degrading into "all scopes" is the
+ shape every cross-tenant leak in this codebase has had.
- `scripts/gate_toolkit.py` holds the single implementation of both. Five
- near-identical copies of the git resolution had already accumulated across
- `scripts/`, which is how the sixth gets written subtly differently.
+ Every cross-tenant read resolves its tenant list through
+ `resolve_portfolio_scope` and nowhere else, then passes it to
+ `require_scope`, which raises rather than letting an aggregate run
+ unfiltered; the list is bound as a query parameter. A `?tenant_id=` filter
+ is intersected with the portfolio, so naming an outside tenant narrows to
+ nothing instead of reaching out. An AST gate fails the build if a new
+ cross-tenant function is added that never calls `require_scope`, and
+ `test_mssp_portfolio_isolation.py` replays two organisations plus an
+ unmanaged tenant against live Postgres in `integration.yml`.
- Two things this turned up. `check_gate_coverage.py` decided whether a script
- inspects the repository partly by looking for `Path(__file__)` — the very
- idiom being migrated away from — so moving six gates onto the shared
- resolver silently dropped them out of the inventory, and deleting their
- workflow steps would then have gone unnoticed. And the first spelling of the
- root-resolution property matched `repo_root` anywhere in the file, which
- `check_gate_coverage.py` emits as a JSON key: a gate rooted at `__file__`
- read as compliant on the strength of a dictionary key in its own output.
- Both are read from the syntax tree now.
+ The sample rows are deleted rather than gated. `health_score` is gone
+ because it was an undefined composite; `mttr_minutes` is measured from
+ cases a tenant actually closed and is null when it closed none; seeded
+ rows are counted separately as `synthetic_alerts` and excluded from the
+ headline figures.
-- **The 153 Python files that belong to no manifest tree are type-checked, and
- five gates that reported OK over nothing now fail.** Giving all twenty trees
- a `[tool.mypy]` table still left `scripts/`, `tests/`, `tools/` and
- `plugins/` outside every config — which is to say the CI gates themselves
- were the only Python in the repository nothing type-checked, while every
- other claim here rests on them.
+- **Per-tenant limit headroom, so a cap cannot throttle a customer
+ silently.** A tenant that hits a ceiling raises no error anyone sees —
+ alerts keep arriving and stop being triaged, which reads to an evaluator
+ as "the AI doesn't work". `app/services/entitlements.py` measures
+ `connectors`, `seats`, `alerts_per_day` and `triages_per_month` from real
+ rows, reports `ok` / `warning` / `exhausted` per tenant across a
+ portfolio, and logs exhaustion at `warning`. AiSOC ships uncapped: a key
+ with no configured ceiling reports `unlimited` rather than a default
+ nobody set. Ceilings come from `tenants.limits` (per tenant, wins in
+ either direction) or the new `AISOC_DEFAULT_TENANT_LIMITS` setting —
+ declared as a real field, because an undeclared setting is dropped by
+ `extra="ignore"` and the operator who exports it gets no explanation.
- They are covered by a scoped invocation under a new root
- `mypy-unmanaged.toml`, not by a root `pyproject.toml`. Three things were
- measured rather than assumed: a root manifest carrying only `[tool.mypy]`
- makes `poetry check` answer *"The Poetry configuration is invalid"* from
- every directory without a manifest of its own, because poetry searches
- upward; pytest gains a `configfile` where it had none; and
- `check_dependency_pins.py`, `check_toolchain_pins.py` and
- `security_audit.py` all pass with one present only because each is scoped to
- `services/*` + `packages/*`, so they would miss it by accident of a glob and
- fail closed on it the day any of them is made structural.
+- **The Splunk warehouse driver executes.** It previously raised
+ `HuntNotConfigured("provider scaffolded but live SPL execution not yet
+ shipped")` on every call, so the SPL every hunt was translated into was
+ discarded. `app/services/spl_runner.py` carries the same three guards as the
+ ES|QL runner — per-tenant SSRF allow-list, air-gap policy, row cap — and
+ prefixes the translator's bare `index=…` expression with `search`, without
+ which Splunk's REST API answers 400 on every hunt.
- The scope is **computed, never listed** — `git ls-files '*.py'` minus every
- manifest tree — so a script added anywhere is checked with no edit, and
- deleting a tree's manifest moves its files into this scope rather than out
- of coverage. Sixteen `plugins/*/plugin.py` files share one module name in
- directories whose hyphens keep them from ever being packages, so the run is
- split into the minimum number of invocations with no collision, derived
- rather than configured. mypy's own `--linecount-report` must account for
- every module asked of it before the run may report clean, and a finding
- against a file outside the scope fails rather than being recorded.
+- **A playbook step that names a response verb now reaches governed
+ dispatch, and says honestly what happened to it.** Fifteen of the engine's
+ twenty-two step types name an action against somebody's estate. Three of
+ them — `block_ip`, `isolate_host`, `create_ticket` — returned
+ `{"simulated": true}` from inside the engine, reached no executor, and were
+ recorded `SUCCESS`; the other twelve had no handler at all. Meanwhile
+ `services/actions` held working executors for fourteen of the fifteen,
+ behind a contract declaring each verb's impact, reversibility, approval
+ requirement and whether a probe exists to confirm the effect landed. The
+ missing piece was never an executor. The comment in `action.py` asserting
+ that "playbooks dispatch step by step through this service, so every step
+ is graded on the way past" described something that did not happen.
- Five gates were found reporting OK over a repository containing nothing, by
- copying `scripts/` into an empty git repository and running all forty:
- `validate_detections.py` printed a warning and exited 0 over an empty
- detections corpus — the validator for the number the front page quotes,
- certifying zero rules as valid; `check_grafana_dashboards.py` failed on an
- empty dashboards directory and *passed* on a missing one, so the larger loss
- was the one it forgave; `check_go_module_paths.py` reported "OK: 0 published
- Go module path(s)"; `check_repo_self_links.py` reported every self-link
- healthy over zero files, having replaced a lychee run that could not tell a
- rate-limited 403 from a live link; and `audit_health_probes.py --check`
- printed a table header and exited 0 over zero services. Each now fails and
- says what it opened. `check_repo_self_links.py` also reported a file it
- could not decode as checked, and now names it.
+ The bridge is `services/agents/app/playbook/action_bridge.py` ->
+ `POST /api/v1/playbook-steps/dispatch` -> `live_actions.dispatch()`. It
+ lands in the API rather than going direct because that service holds the
+ credential vault and the tenant session, and `services/actions` is the only
+ place that may change a customer's estate — the same shape, and the same
+ reasoning, as `siem_writeback`. **One step, one request, one grading**: a
+ playbook is not approved as a unit, so authorising it cannot authorise
+ whatever its steps happen to contain.
- The ratchet in `scripts/check_mypy_baseline.py` gains a `file -> scope`
- direction and goes **919 → 994**: +89 newly visible, −14 fixed. Of the 89,
- 20 are `import-untyped` against the deliberate no-dependency environment the
- baseline is recorded in and the rest are annotation and narrowing findings.
- A separate `--warn-unreachable` pass over the same scope reported 8
- statements, all one shape: a defensive `isinstance` guard against untrusted
- YAML or JSON that the parameter's own annotation declares impossible. None
- is dead at runtime, so the annotations are what is wrong and the flag is
- deliberately not enabled — recording those 8 would invite someone to delete
- the guards to clear them.
+ `executed` is the single field that means a vendor was touched. A preview,
+ an approval queue, a blocked action, a tenant with no integration, a
+ credential-less simulation and a vendor failure are each named and each
+ `executed: false`, and a step that did not execute is recorded FAILED so
+ the run halts under the default `on_failure: abort`. `AWAITING_COMPLETION`
+ counts as executed and `PENDING_APPROVAL` does not: collapsing that pair
+ either loses an action in flight or invents one that never ran. Execution
+ is off by default (`AISOC_PLAYBOOK_ACTIONS_EXECUTE`), so out of the box a
+ response step previews and reports a preview.
-- **All twenty Python trees are type-checked, and the coverage is gated in both
- directions.** `#818` ran mypy for the first time, over the six trees that
- declared a `[tool.mypy]` table. Fourteen declared none, so the job named
- "Lint & Type-check" reported green over 787 Python files it never opened — a
- tool that appears to cover the repository while covering a third of it is
- indistinguishable from no tool at all, only more reassuring.
+ **`approval` is the one step type deliberately not bridged.** It is a
+ pause, and the engine is a single-threaded index walk with nothing to
+ suspend and nothing to wake. It is also no longer the mechanism: every
+ response step is now graded individually at dispatch and returns
+ `pending_approval` on its own when a human is required, so a gate in front
+ of one would gate a decision that is already gated. It fails closed with
+ that reason recorded rather than shipping a handler that pretends.
- Every tree now declares one, matching the two strictness shapes that already
- existed rather than inventing a third: the ten services take
- `python_version = "3.11"` / `strict = false` / `ignore_missing_imports = true`
- like `services/api` and `services/osquery-tls`; the four packages take
- `strict = true` / `python_version = "3.11"` like `packages/sdk-py`,
- `plugin-sdk-py` and `aisoc-cli`. No `ignore_errors`, no widened config, no
- `# type: ignore` added to reach a number.
+ **`run_playbook` as a nested step stays unimplemented, and the calculus was
+ re-examined rather than inherited.** Per-step grading was the missing piece
+ the previous decision named, and it now exists — but the objection it was
+ the answer to does not move: a nested playbook's steps are still not
+ visible where the parent declares its policy. What changed is that the
+ parent no longer *needs* to bound them, because each step is graded on
+ arrival wherever it came from. What has not changed is that the engine
+ cannot see a nested playbook's content before running it, so an author
+ cannot review what a run will do, and a cycle across two playbooks that
+ reference each other is unbounded by the per-step `visited` set. Recursion
+ depth and an ancestor set would answer the second; the first is a product
+ question about reviewability, not a governance gap, and it is the reason to
+ keep waiting.
- `scripts/check_mypy_baseline.py` gains the coverage check itself, rather than
- a second gate with its own idea of what a tree is. A Python tree on disk that
- declares no `[tool.mypy]` fails (`tree -> config`), and a configured or
- recorded tree that is no longer on disk fails (`config -> tree`) — the
- direction that rots quietly, because nothing ever fails when a stale entry is
- simply never consulted. Discovery is now structural, every directory holding a
- `pyproject.toml`, instead of `services/*` plus `packages/*`: the glob was a
- naming convention, and a Python tree added anywhere else would have satisfied
- a coverage gate written against that same glob while being checked by nothing.
- The gate resolves its root from `git rev-parse` rather than from its own file
- location, prints how many trees and files were in scope, and its `--self-test`
- injects coverage drift each way against a throwaway repository.
+ The schema's `x-aisoc-execution` map gains a `governed` class rather than
+ reusing `executed`, because "a handler ran and made an outbound call" and
+ "a vendor was touched" are different claims and the second is answered per
+ run. `simulated` stays in the vocabulary: it is the class for a handler
+ that answers from inside the engine, which is what these three did, and
+ deleting the word would make that state unspellable rather than absent.
+ `check_playbook_schema_parity.py` compares `governed` against
+ `engine.RESPONSE_STEP_TYPES` in both directions, so a verb wearing the
+ label while being answered locally fails the build.
-- **`scripts/check_logger_kwargs.py` — no stdlib logger may be called with a
- structlog keyword.** `logging.Logger.warning` accepts exactly `exc_info`,
- `stack_info`, `stacklevel` and `extra`; structlog's bound logger accepts any
- keyword and turns it into the event dict. Both libraries are used in this
- tree, so `logger.warning("x", reason=exc.reason)` is correct in one module and
- a `TypeError` in the next, and the two call sites are identical to read — only
- the binding decides.
+- **The published playbook schema now describes the engine that runs
+ playbooks, and a gate keeps it that way in both directions.**
+ `schemas/playbook.schema.json` is the contract authors are told to trust,
+ and it had drifted from `services/agents/app/playbook/` in every available
+ direction at once. Two schema files existed with different step
+ vocabularies — 15 types at the repo root, 9 under `schemas/` — against a
+ `StepType` enum of 22, and the NL drafter silently fell back from one to
+ the other if the primary was missing. Eleven step types were declared by a
+ schema and implemented nowhere (`trigger`, `action`, `loop`, `parallel`,
+ `human_approval`, `wait`, `isolate`, `block`, `create_case`,
+ `run_playbook`, `script`); thirteen were accepted by the engine and
+ declared by neither schema. Six step fields (`blast_radius`, `depends_on`,
+ `output_key`, `retry.max_attempts`, `retry.backoff_seconds`,
+ `retry.backoff_multiplier`) were declared and never read — `blast_radius`
+ carried the description "engine enforces analyst approval for destructive
+ steps", and the engine could not see the field at all.
- This needs a checker rather than a grep because of *where* it hides. All five
- instances `#818` fixed were inside `except` blocks, where the raised
- `TypeError` replaces the exception being handled and no sibling handler can
- catch it. A misused keyword on the happy path is found by the first person to
- run the code; the same keyword in a fallback runs only when something has
- already gone wrong. The report sorts by context and names how many findings
- sit on such a path.
+ Resolved by making `schemas/playbook.schema.json` the only schema, widened
+ to the full `StepType` range with the condition-as-string form the engine
+ already accepted, bounds matched to `bounds.py` (3600s / 25 retries, not
+ 600s / 5), and the two authored-but-inert playbook keys (`inputs`,
+ `dry_run_support`) declared as the documentation they are. The root
+ duplicate is deleted. The schema also carries `x-aisoc-execution`, a
+ machine-checked map recording whether each step type is `executed`,
+ `simulated`, or vocabulary with no handler — so an author can tell what
+ will happen before writing the playbook rather than after running it.
- The scanner resolves each module's logger flavour rather than assuming one:
- every binding, not just `logger`; `self._log` and class-body loggers reached
- through the instance; `from app.core.logging import get_logger` followed to
- the defining module, because both in-repo `get_logger` helpers return
- structlog while their name says nothing; `.bind()` / `.getChild()` chains;
- annotations, including parameters, for a logger that is handed in; and
- `logging.warning(...)` on the root logger. A `**kwargs` splat onto a stdlib
- logger cannot be decided statically and is reported separately rather than
- counted as clean, and a receiver it cannot classify is counted as
- `unresolved`, never as clean.
+ `run_playbook` is deliberately **not** implemented as a step, on its own
+ merits rather than by inheriting the argument that removed it as an action.
+ A nested playbook's steps are not visible where the parent declares its
+ step-level policy, so the parent cannot bound them; the engine reads no
+ step-level approval or blast-radius field today, so nesting would let one
+ ungated parent pull in an arbitrary tree; and twelve of the engine's own
+ step types have no handler, so a verb whose purpose is to execute more
+ steps would multiply that. It can return when playbook steps are graded
+ individually — recursion depth and an ancestor set are the easy part.
- It covers all 1,408 Python files including the ~150 under `scripts/`,
- `tests/`, `tools/` and `plugins/` that belong to no tree and so are checked by
- no `[tool.mypy]` at all. It classifies 514 stdlib and 970 structlog call sites
- with 18 unresolved receivers, and reports zero defects — and fails outright if
- it classifies no stdlib logger, because "found nothing" and "resolved nothing"
- otherwise print the same word. Pointed at the pre-`#818` files it reports all
- five, each labelled `[inside except]`.
+ `scripts/check_playbook_schema_parity.py` compares the schema enum, the
+ `StepType` enum, the engine's handler table, the execution map, the bounds
+ module and the pack validator's trigger list — every pair in both
+ directions, because the characteristic failure here is a check that asks
+ only whether the schema declares something the engine lacks and never the
+ reverse, which is the direction things actually drift. It carries a
+ `--self-test` that injects drift each way and fails if any goes undetected,
+ refuses to run at all on a tree missing its marker files rather than
+ printing OK about files it never opened, and names the root, schema, step
+ counts and playbook count it inspected.
- Three shapes were added after the first version reported a clean tree and the
- shapes were then found *in* that tree: an inline
- `logging.getLogger(__name__).warning(...)` with no binding to look up
- (`detection_proposals.py`), a class-body logger reached as `self.logger`
- (`core/config.py`), and an annotation with no factory call in sight. All three
- were invisible to a scanner that only read assignments of a name.
+- **Every `ActionType` now resolves a capability contract.** `notify_slack`
+ was the last one without, and it was a naming gap rather than a missing
+ capability: the verb is `notify`, it has had a contract throughout, and the
+ `SlackNotify` adapter already bridged the two names. Nothing connected a
+ lookup *by `ActionType` value* to that bridge, so `approval_gate` found no
+ contract and skipped the confidence matrix — a 10%-confidence
+ `notify_slack` was approved for auto-execution, and the verb most likely to
+ auto-execute was the one graded without reference to confidence. It now
+ requires an analyst under the default L1 tier.
-- **CI now runs the interpreter production runs, and `gofmt`, `services/realtime`
- and the published image list are gated.** Three loose ends `#817` named and
- did not close.
+ Closed with a one-entry alias (`ACTION_TYPE_CAPABILITY_ALIASES`) rather
+ than a rename, because `action_type` is persisted operator intent:
+ `remediation_whitelist` (migration 015) stores per-tenant pre-approvals
+ keyed `UNIQUE (tenant_id, action_type)`, so renaming the member silently
+ orphans every row an operator created for `notify_slack`. It is also a
+ documented request field and a member of the `ActionType` union in
+ `packages/types`. The retirement condition is recorded rather than left
+ open-ended: the map goes when `ActionType` does. `check_action_contract.py`
+ gains two directions — every `ActionType` must resolve a contract, and
+ every alias must name a real `ActionType`, point at a real contract, not
+ shadow a capability of the same name, and describe a bridge some adapter
+ actually implements.
- *Python.* Twenty-four workflows ran 3.12 while all thirteen service images
- ship 3.11. Every manifest declares `^3.11`, which permits both, so nothing
- written down was violated — which is exactly why it survived. Standardised on
- **3.11**, in 41 replacements across those 24 workflows, because 3.11 was
- already the answer everywhere except CI: the images ship it, the devcontainer
- installs it, `ruff.toml` targets `py311`, five of six `[tool.mypy]` tables set
- `python_version = "3.11"`, and all twenty-two manifests floor at 3.11 or
- below. Moving the images to 3.12 instead would have meant changing all of
- those *and* raising the published floor for seven installable packages — a
- breaking change for downstream consumers, to fix a CI hygiene problem. Testing
- at the declared floor is also the stronger guarantee. Nothing broke: 2726 API
- tests, eleven service suites at their coverage floors and three Python SDKs all
- pass on 3.11, and the mypy baseline is byte-identical (because those five
- `python_version` pins meant mypy was already checking 3.11 semantics while
- running on 3.12). `PYTHON_INTERPRETER_SPLIT` is deleted rather than emptied: a
- split that can be recorded is a split that can grow.
+- **`scripts/upgrade_playbooks.py`, which `docs/upgrade/MIGRATION.md` has told
+ operators to run since v4.** It did not exist, so the one command in the
+ upgrade path that touches a customer's own content failed at exactly the
+ moment it was needed. The field table beside it was worse: four of the five
+ spellings in its "v4" column are rejected by the schema
+ (`on_failure: {policy}`, `retry: {max_attempts, backoff}`,
+ `condition: {expr, language}`, and `type: "action"`, which is not a step
+ type at all), so following the doc by hand produced playbooks that would
+ not validate either. Both are corrected.
- *gofmt.* CI ran `go vet` and `go build` and never `gofmt`; 28 files across four
- modules had drifted. The new job walks every `.go` file rather than iterating a
- matrix, because a matrix is a list somebody has to remember to add to — which
- is how `packages/sdk-go` came to be compiled by nothing. Formatting landed as
- its own commit; nine files also had reStructuredText ``code`` markers in Go doc
- comments, which have no inline code span, so gofmt was rewriting them to two
- *left* curly quotes. The 58 marker pairs are gone, which leaves gofmt stable.
+ The script reports before it writes, validates its own output against the
+ real `schemas/playbook.schema.json` before touching anything, and leaves
+ alone any file whose upgrade would not validate. `loop`, `parallel`,
+ `wait`, `run_playbook`, `action` and `trigger` are reported and refused
+ rather than mapped onto a nearest neighbour — that rewrite is precisely the
+ defect removed from the NL drafter, where a `disable_user` step shipped as
+ `investigate`. An empty scan is a failure, not a clean bill of health.
- *`services/realtime`.* Published to GHCR on every release and built by nothing
- — no build, test, lint or type-check job anywhere — despite being one of the
- two ends of the Kafka spine, carrying the OpenTelemetry instrumentation that
- keeps the distributed trace continuous, and holding the TypeScript CORS guard.
- It now has all four, installing with `npm ci` from its committed lock.
- `@types/node` moves `^20` → `^22` to match the Node 22 runtime, and a
- `tsconfig.check.json` type-checks the test file, which `rootDir: src` had left
- checked by nothing.
+- **Competitor product names removed from the docs portal, the benchmark page
+ and the archived plan subtree, and a CI gate added to keep them out.** AiSOC
+ names no competitor product, but two published comparison tables and the
+ competitive-landscape section of the archived plan named eleven vendors
+ outright. The analytical content is preserved everywhere: every matrix cell
+ survives unchanged and every gap keeps its capability, direction and
+ magnitude — only the vendor's identity is gone. The comparison columns on
+ `apps/docs/src/pages/index.tsx` and `apps/docs/docs/benchmark.md` now read
+ "Open-source SIEM/HIDS" and "Commercial SIEM platform"; the plan's competitor
+ profiles and its five-column capability matrix now carry category labels
+ ("Autonomous triage", "Hyperautomation SOAR", "Hyperscaler bundle",
+ "Vendor-stack XDR", "First-line responder", "No-code automation").
- Three new bidirectional pairs in `check_toolchain_pins.py` — `module -> format`
- / `format -> module`, `image -> CI` / `CI -> image`, `target -> ship` /
- `ship -> target` — each of which rediscovers its defect when run against the
- previous commit. Five parser blind spots were found and fixed in the gate
- itself, every one present in this tree: matrix legs read file-wide (so
- `ci.yml`'s two `service:` matrices merged), the block-list matrix form
- unreadable, `working-directory` truncated at the space inside a matrix
- expansion, `working-directory` unreachable when it is a step's first key, and
- `gofmt` matched inside step names and quoted `echo` strings — the last of which
- also let a quoted path in a `compose-smoke.yml` shell array count as CI
- coverage for eleven services, so the check would have reported OK about the
- very gap it exists to find. Each published image now prints the step that
- exercises it instead of contributing to a tally, and the gate asks
- `git rev-parse` for its root instead of inferring it from its own file
- location. 35 self-test cases, 49 unit tests.
+ - `scripts/check_competitor_names.py` + `scripts/competitor_names.toml` drive
+ the gate from an **explicit list of competitor product names**, never a
+ heuristic, and allow-list the integration surfaces **by path** — connector
+ modules, plugin manifests, connector docs, the connector registry, fixtures
+ and tests. This matters because a vendor name is usually correct here:
+ `Torq` is simultaneously a first-party SOAR connector and a name that
+ appeared in a comparison matrix, so a global replace would have broken
+ working connector code. Its connector, manifest, docs page and tests are
+ untouched; only the competitive framing changed.
+ - Both lists are checked **in both directions**. A pattern that matches
+ nothing is dead and fails; an allow entry whose files no longer contain any
+ of the names it excuses is stale and fails, so an exemption cannot outlive
+ the code it excused; a mistyped name in an allow entry is rejected at load
+ rather than silently excusing nothing.
+ - A published comparison table is the one place where *any* vendor name is a
+ violation, including one AiSOC integrates with, because the table's job is
+ to position AiSOC against it. Those two regions are pinned by literal
+ start/end markers and checked against a wider vendor list; a marker that
+ drifts fails rather than scanning an empty slice and reporting clean.
+ - The scan root comes from the working directory or `--root`, never from the
+ script's own location, and the run prints the absolute path it walked plus
+ the file count — a gate that resolves its own repository can print a
+ confident OK about a tree it never inspected.
+ - `scripts/check_competitor_names.py --self-test` proves the gate detects a
+ known-bad sample and passes a known-good one in both directions, and fails
+ if a declared competitor has no fixture. `tests/test_competitor_names_gate.py`
+ adds 42 cases covering the stale-allow-list, marker-drift and dual-role-name
+ properties. Wired into `.github/workflows/competitor-names.yml`, which runs
+ on pull requests *and* pushes to `main` with no paths filter.
-- **`scripts/check_toolchain_pins.py` — the Go, Node and toolchain half of
- reproducible builds.** `check_dependency_pins.py` made every *package*
- install path agree and stopped at the Python services. Go has `go.sum` and
- pnpm has a frozen lockfile, so the raw material was already there — but
- nothing compared the paths that use it, and nothing audited the *runtime*
- those paths run on. Measured across 111 install paths, and every
- disagreement found:
+- **Tool attribution is now prevented at commit time and blocked in CI.** AiSOC
+ does not attribute work to a development tool or AI assistant. An audit found
+ the rule was being broken automatically: a `Co-authored-by:` trailer naming an
+ editor appeared in 214 commits on `main`, and 280 of 732 pull request bodies
+ carried a "Made with" or "Generated with" footer. Nothing in the repository
+ caused it — no `commit.template`, no `core.hooksPath`, nothing in
+ `.git/hooks/` — the editor appended it at commit time, and it survived
+ `git commit --amend`.
- | What | Before |
- |---|---|
- | `apps/web/Dockerfile` | `pnpm install --no-frozen-lockfile` — the production web bundle was the one install path in the repository free to resolve its own dependency set, while all 13 workflows installing the same workspace used `--frozen-lockfile` |
- | `services/realtime/Dockerfile` | `npm install` twice, and `package-lock.json` never copied into the build context, so the committed lockfile was inert and every image build re-resolved |
- | `install.sh` | `pnpm install --no-frozen-lockfile`, so a self-hoster's install could differ from everything CI tested |
- | `.devcontainer/devcontainer.json` | `pnpm install --frozen-lockfile=false` — an opt-out a substring test for the flag reads as *enabling* it |
- | `deploy-docs.yml` | `pnpm --filter @aisoc/docs install`, unlocked; the published docs site could build from a tree no other job resolved |
- | `apps/web/Dockerfile` | `npm install -g pnpm@8` against `packageManager: [email protected]` — two resolvers, the `poetry 1.7.1 vs 1.8.2` finding again |
- | Node | 20 in both images, the devcontainer, `deploy-docs.yml` and both installers; 22 in twelve workflows. Node 20 left security support in April 2026 |
- | `services/enrichment/Dockerfile` | `COPY go.mod go.sum*` — the glob makes the checksum file optional, so deleting it downgrades the build to an unverified resolve without failing |
- | `ci.yml` | `cache-dependency-path: packages/plugin-sdk-go/go.sum`, a file that does not exist. `setup-go` reports that as a **warning**, so the cache had been silently off while the job stayed green |
- | `packages/sdk-go` | compiled by no workflow at all |
+ - `.githooks/commit-msg` strips the line before it reaches a commit. It is
+ wired through `core.hooksPath`, so it is version-controlled and shared with
+ every clone rather than living in an untracked `.git/hooks/`;
+ `scripts/setup_hooks.sh` installs it and runs automatically as the `prepare`
+ script on `pnpm install`. The hook rewrites and never rejects — a hook that
+ can block is a hook that can halt someone's work on a bad pattern, so the
+ blocking job belongs in CI where it is visible.
+ - This is load-bearing rather than cosmetic because the repository
+ squash-merges with `squash_merge_commit_message=COMMIT_MESSAGES`: GitHub
+ composes the squash commit body from the branch commits, so a trailer on any
+ branch commit is copied onto `main` at merge time.
+ - `scripts/check_attribution.py` fails CI when attribution appears in a
+ commit message, a changed file or the PR body. A human `Co-authored-by:`
+ line is never flagged — a pattern only fires when the trailer names a known
+ tool — and `dependabot[bot]` is allowlisted.
+ - The gate runs on `push` to `main` as well as on `pull_request`. A PR-only
+ check is one-directional: it inspects what contributors propose but never
+ what actually lands, so anything introduced by the merge itself would pass
+ it while the gate stayed green.
+ - `--self-test` runs on every CI invocation and checks four directions against
+ a shared fixture corpus: the gate detects every known-bad sample (so it
+ cannot pass vacuously), flags no known-good sample (so it cannot pass
+ direction one by flagging everything), and the hook strips exactly what the
+ gate flags in both directions (so the two implementations cannot drift).
+ Both read their patterns from the same `.githooks/attribution-patterns.txt`.
+ The self-test earned its place immediately by catching a real hole in the
+ patterns — `Built with GitHub Copilot` slipped through, because a vendor
+ word sat between the verb and the tool name.
- Everything above is now one version and one lockfile discipline, proved
- rather than asserted: two `--no-cache` builds of each Node image resolve
- byte-identically (web 1,172 packages, realtime 187, matching sha256 both
- times).
+- **A model name must resolve, and the variable that routes it must be read —
+ gated in both directions.** `scripts/check_llm_model_routing.py` reconciles
+ `infra/litellm/config.yaml`, the role pins, the API-side mirror,
+ `.env.example` and `docker-compose.yml`: every pin's alias must be defined by
+ the gateway and every alias claimed by a pin; every model named in an env or
+ compose file that wires the bundled gateway must be one the gateway defines;
+ every variable compose points at the gateway must be read by *both* resolvers
+ and every gateway variable the resolvers read must be set by compose; and a
+ service handed the gateway URL must be handed the gateway key. Wired into
+ `ci.yml :: python-lint`, with the self-test running first.
- Six directions, each with a `--self-test` injection that fails if it goes
- undetected — 22 cases. Beyond agreement: *ship → test* (a version an image
- ships that no workflow exercises) and *test → ship* (a version CI uses that
- no image ships, which is the direction versions actually travel); *module →
- sum* and *sum → module*; *module → CI* and *CI → module*; unlocked install
- and its reverse, a lockfile nothing can consume; and both coverage
- directions — a file declaring a toolchain that the gate never opened, and a
- file it *did* open whose declaration the parser could not read.
+ Every parser is AST- or comment-aware, and the self-test proves each refusal
+ rather than asserting it, because all three corpora contain text shaped
+ exactly like what the gate looks for: the gateway config's commented
+ Ollama/vLLM examples repeat `model_name: aisoc-triage` verbatim,
+ `.env.example` documents the escape hatch as a commented
+ `AISOC_MODEL_PIN_TRIAGE=gpt-4o-mini`, and `docker-compose.yml` explains the
+ gateway in prose containing `OPENAI_BASE_URL=http://litellm:4000/v1` two lines
+ above the key it describes. A line regex credits all three. 13 injected
+ defects and 7 parser blind spots, each caught by its own code, plus the shared
+ empty-tree refusal.
- That last direction earned its place immediately. Hunting for this parser's
- equivalent of the blind spot `check_dependency_pins` found in itself, five
- turned up: `cd services/${{ matrix.service }}` resolved to nothing, so five
- of six Go modules looked ungated; `working-directory:` was not read at all,
- so `services/osquery-extensions` looked uncompiled while the workflow that
- compiles it sat two directories away; the npm pattern had no left word
- boundary and matched inside every `pnpm install`; quoted shell text was read
- as commands, so `die "pnpm install failed."` counted as an install path; and
- the `node_modules` skip was a prefix rather than a path segment, so on a
- checkout with dependencies installed the gate scanned 223 vendored manifests
- and reported Node floors of `0.10` from other people's packages — an answer
- that depended on whether someone had run `pnpm install`.
+- **`scripts/check_orm_migration_parity.py` — a column a model declares and
+ no migration creates now fails the build.** Structural on both sides
+ (Python `ast` over the models and the alembic revisions, plus the raw SQL
+ those revisions and `services/api/migrations/*.sql` execute) and in both
+ directions: `missing-in-migration`, `missing-in-model`,
+ `type-narrower-in-migration`, `type-family-mismatch`, and
+ `no-migration-source` for a mapped table no migration mentions at all —
+ which is the finding that exists because pairing nothing produces zero
+ findings, and zero findings over zero comparisons prints the same word as
+ a clean result. `--credits` enumerates what each verdict was *based on*
+ for the same reason.
- `EXPECTED_ESBUILD` pins the **resolved** esbuild set (0.25.12 bundled by
- Next, 0.28.1 from the scoped overrides). The overrides are deliberately
- per-parent because forcing esbuild workspace-wide broke Turbopack's font
- import map, and that scoping means a `vite` bump can pull a different
- esbuild with no esbuild line in the diff.
+ Scoped structurally rather than by a list: a service that calls
+ `metadata.create_all` builds its tables from the models, so a column no
+ migration creates is created anyway. `services/api` is the one such
+ service and is reported as advisory with its count printed, not hidden.
+ The four alembic-managed services have no such fallback, which is exactly
+ why UEBA's missing column was fatal.
-- **`scripts/check_mypy_baseline.py` — the type check the CI job is named
- after.** `ci.yml`'s **Python — Lint & Type-check** installed mypy and never
- invoked it. Six manifests declare `[tool.mypy]` and three set
- `strict = true`, so three authors had asked for a check that had never once
- run. It runs now. The 690 findings are recorded in
- `scripts/mypy_baseline.json` exactly as mypy reports them, keyed on
- `(tree, file, error-code)` so an error cannot be introduced under cover of
- fixing an unrelated one — no `ignore_errors`, no widened config, no excluded
- tree. Both directions: a finding above baseline fails, and a *fixed* finding
- still in the baseline fails too, so freed headroom must be banked rather
- than left to absorb the next regression.
+ Verified non-vacuous against the tree as it was: it reports all three of
+ the real UEBA defects. The `type-family-mismatch` kind exists *because*
+ an earlier version reported only two of them — a length comparison could
+ not see `UUID` against `String(64)`, since neither side declared a length
+ the other contradicted.
- Its first run against a rebased tree earned the gate: 22 findings — 14
- `union-attr`, 8 `arg-type` — in `services/api/app/services/playbook_step_dispatch.py`,
- a file merged hours earlier that nothing had ever type-checked. They are
- recorded rather than fixed because that file belongs to concurrent work;
- what matters is that they are now visible and cannot grow. Five other
- findings were real null-safety bugs in `services/fusion` — an
- `IsolationForest` and an `LGBMRanker` used before training, and the Kafka
- consumer and producer used before `start()` assigns them — and those are
- fixed, taking fusion from 7 to 2.
+- **`scripts/check_service_migration_bootstrap.py` — a migration chain
+ nothing invokes is not a migration chain.** Asserts every service with an
+ `alembic.ini` ships `app/_migrate.py`, keeps it byte-identical to the
+ others, runs it from its `CMD`/`ENTRYPOINT` (matched on the command lines,
+ so a mention in a comment does not count), and carries both an owner DSN
+ and a compose healthcheck. The other direction too: a service shipping
+ the module with no chain to apply would exit 1 on every start.
-- **`Reproducible builds` now proves the property for all thirteen Python
- services, not one.** The `twice` job asserted byte-identical resolution for
- `api` alone; the other twelve used the same mechanism and nothing checked
- it. A nightly matrix (03:17 UTC) double-builds all twelve services that have
- a Dockerfile and double-installs the thirteenth, `teams-bot`, which has a
- lock but no image. The matrix is derived from the tree rather than
- hand-listed, and a `coverage` job fails when the number of services proved
- is not the number that exist — a scheduled run that silently covers less
- than it claims is the `wet-eval` failure, eight green weekly runs with every
- real step skipped.
+- **`scripts/check_connector_profiles.py` — a connector-type drift gate that
+ reads in both directions.** Nothing compared the profile keys in
+ `services/ingest/internal/normalizer/normalizer.go` against the identifiers
+ the connectors service declares, and they had drifted apart in both
+ directions at once. The gate enumerates four name spaces — the Go profiles
+ and type aliases, the 84 declared `connector_id`s, the `ConnectorType`
+ union, and every `connector_type` appearing in a documented example — and
+ fails when a name in any of them resolves to nothing.
+
+ It also checks that a documented example reaches a *profile*, not merely a
+ declared connector: a reader pastes a flat payload, which never carries the
+ canonical envelope, so only a profile or alias can give the event a vendor
+ identity. That is the specific check that catches the README defect.
+
+ `--self-test` injects twelve defects, one per direction and per failure
+ code, and requires the gate to catch each; it runs in CI on the same tree
+ immediately before the gate itself. The resolved repo root, every file read
+ and every count are printed before the verdict, and an input that is missing
+ or parses empty is a hard error rather than a quiet pass — a gate that
+ reports OK about a tree it never opened is worse than no gate.
- **The console's `ConnectorType` union is generated from the connector
registry.** Its members were corrected in the previous release; the
@@ -1360,17 +877,43 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
no XSIAM, and XSIAM is reachable under `cortex_xsiam` for any deployment
that means it. The evidence is recorded beside the map in `normalizer.go`.
-- **`scripts/check_gate_coverage.py` — the gate on the gates.** The most
- expensive recurring defect in this repository is a mechanism that exists, is
- tested, and has no caller on the path that needs it. A gate is the worst case
- of that shape, because a gate *is* its caller: a conformance script no
- workflow runs cannot fail, which is indistinguishable from no gate at all,
- while its presence in `scripts/` advertises coverage to everyone who reads
- the tree. Finding them had meant tracing the call graph by hand.
-
- This resolves the graph mechanically instead of reading workflow names,
- because a check can be reached four different ways and only one of them is
- obvious: a workflow can run it directly, through a `make` recipe, through
+- **A gate that fails when a route lets its caller name the tenant.**
+ `scripts/check_route_tenant_scope.py` is an AST pass over every route
+ decorator and signature under `services/` — a structural property needs a
+ structural check, and the thing being looked for (a parameter's name, a
+ decorator's `dependencies` list, the router object the decorator hangs off)
+ survives renames a regex would miss. It fails in **both** directions: a
+ route that takes a tenant identifier with no auth dependency, and a route
+ that accepts one without intersecting it with the caller's scope. It reads
+ request-body models too, because `POST {"tenant_id": "<someone else's>"}` is
+ the same hole as `?tenant_id=` and the mutating routes carry it on the body.
+
+ `--self-test` injects a violation of each kind and asserts both are caught
+ while two clean controls pass, so the gate cannot quietly stop detecting
+ things. Exemptions each carry a written reason and are as narrow as the
+ facts allow: `services/mesh` is public by design (Ed25519 + k-anonymity, a
+ bearer token would break federation rather than secure it); three MSSP
+ routes *define* a scope rather than read within one, so intersecting would
+ make them impossible; and `osquery-tls` `/enroll` authenticates with a
+ per-tenant enroll secret because osqueryd has no session yet. That last
+ exemption is **conditional** — the self-test strips the verifier it names
+ and asserts the route is then reported, so deleting the check and keeping
+ the entry fails the build. The gate resolves its repository root from `git
+ rev-parse` rather than its own file location, and prints how many routes
+ across how many files it opened, so an OK can be distinguished from a scan
+ that never happened. `--inventory` prints the per-service table.
+
+- **`scripts/check_gate_coverage.py` — the gate on the gates.** The most
+ expensive recurring defect in this repository is a mechanism that exists, is
+ tested, and has no caller on the path that needs it. A gate is the worst case
+ of that shape, because a gate *is* its caller: a conformance script no
+ workflow runs cannot fail, which is indistinguishable from no gate at all,
+ while its presence in `scripts/` advertises coverage to everyone who reads
+ the tree. Finding them had meant tracing the call graph by hand.
+
+ This resolves the graph mechanically instead of reading workflow names,
+ because a check can be reached four different ways and only one of them is
+ obvious: a workflow can run it directly, through a `make` recipe, through
another script it already runs, or through a pytest suite it collects.
`connector_conformance.py` is reached *only* by that last route — the
connectors matrix in `ci.yml` collects `test_conformance.py`, which imports
@@ -1403,6 +946,119 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
vendored mirrors with no drift gate. A redactor copy that drifts strips a
different set on one side of the wire than the other.
+- **No gate may report OK over a repository with no content, and it is a CI
+ gate now rather than a probe somebody happened to run once.** Copying
+ `scripts/` into an empty git repository and running every wired check there
+ found five reporting OK over a tree holding nothing — the detection
+ validator behind the rule count on the front page certifying zero rules as
+ valid, a dashboard check that failed on an *empty* directory and passed on a
+ *missing* one, `OK: 0 published Go module path(s)`, every self-link healthy
+ over zero files, and a health-probe audit printing a table header and
+ exiting 0. Those five were fixed; the probe that found them was a one-off,
+ so the next gate written could reintroduce the defect freely.
+
+ `scripts/check_gate_contract.py` is that probe made permanent. It runs all
+ 69 inventoried checks the way a workflow runs each one, inside a git
+ repository holding only `scripts/`, and requires each to exit non-zero.
+ Three things make the result mean something:
+
+ - **The inventory is not written here.** It comes from
+ `check_gate_coverage.py`, which already resolves the workflow-to-check
+ graph structurally. A second scanner would drift the first time either
+ one learned something the other had not.
+ - **A failure the empty tree did not cause is INCONCLUSIVE, not a pass.**
+ An argparse usage error, or an import of a module the repository does not
+ supply, means the gate was never exercised — crediting that as a refusal
+ would be the same defect one level up. Whether a missing module is the
+ repository's own is read from `git ls-files`, so `No module named 'app'`
+ counts as the empty tree working and `No module named 'structlog'` does
+ not.
+ - **Every way CI runs a script is probed, and the worst result decides.**
+ Four subcommands under one name are four gates; excusing the set because
+ the first refused is how the others stay hidden. Only invocations
+ carrying a declared verdict flag are probed, so the generator half of a
+ script that is both generator and gate is not mistaken for one.
+
+ Exceptions are recorded with the disposition they are excused for and
+ checked in both directions, so an entry naming a check that no longer
+ exists fails, and so does one whose gate has started behaving differently.
+ Four are recorded: `wet_eval_check.py` (reads two environment variables and
+ no repository content), `security_audit.py`'s `validate-ignores` arm (its
+ entire subject is a file inside `scripts/`, the one directory the scratch
+ tree must keep), and `openapi_diff.py` / `wet_eval_update_benchmark.py`
+ (both inputs named on the command line, one of them built outside the
+ checkout).
+
+- **Every check resolves its repository root from git, and every check
+ carries a `--self-test`.** Sixty of the sixty-nine derived a root from
+ `Path(__file__).resolve().parent.parent` — whatever happens to sit two
+ levels above the script — which is how a gate prints a confident OK about a
+ tree it never opened, and only a deliberate run from another directory
+ catches it. Fifty-four had no self-test at all. Both are now properties the
+ contract gate enforces, with empty exception lists.
+
+ `scripts/gate_toolkit.py` holds the single implementation of both. Five
+ near-identical copies of the git resolution had already accumulated across
+ `scripts/`, which is how the sixth gets written subtly differently.
+
+ Two things this turned up. `check_gate_coverage.py` decided whether a script
+ inspects the repository partly by looking for `Path(__file__)` — the very
+ idiom being migrated away from — so moving six gates onto the shared
+ resolver silently dropped them out of the inventory, and deleting their
+ workflow steps would then have gone unnoticed. And the first spelling of the
+ root-resolution property matched `repo_root` anywhere in the file, which
+ `check_gate_coverage.py` emits as a JSON key: a gate rooted at `__file__`
+ read as compliant on the strength of a dictionary key in its own output.
+ Both are read from the syntax tree now.
+
+- **Every test file in the tree is now executed by a workflow, and a gate
+ holds it that way in both directions.** 63 were executed by nothing at all:
+ 56 of the 82 under `services/agents`, because that invocation was a
+ hand-maintained list of 26 file paths; five under the repository's own
+ `tests/`, which was reached only as nineteen single-file invocations spread
+ across eleven workflows; and both files under `scripts/tests/`, which no
+ workflow named. A file nothing runs reports nothing, which is
+ indistinguishable from a file that passed.
+
+ `scripts/check_test_discovery.py` derives the corpus from `git ls-files` and
+ models reach the way pytest collects — directory arguments, `--ignore`,
+ `conftest` `collect_ignore`, and `-m` marker deselection, since a file whose
+ every test is deselected runs nothing however plainly it is named. An
+ invocation naming a path that is not in the tree fails too. Quarantine is a
+ shrink-only list where every entry carries a reason and an entry that has
+ become reachable fails as stale, so "not run" can never again be silent.
+
+ Two blind spots in the gate's own parser were found by enumerating what it
+ **credited** rather than what it flagged. A step that `cd`s into a service
+ and then names `tests/` had its paths resolved against the repository root,
+ which simultaneously reported all 82 agents files unreached and credited the
+ root `tests/` tree with a package's invocation — the same defect in both
+ directions at once. And `-m` was first treated as "a filter, not a
+ collection rule", which would have credited a module in full under
+ `-m "not integration"` while nothing in it ran.
+
+ The invocations are directories now. `services/agents` runs `tests/`;
+ `tests/` and `scripts/tests/` run as directories in `python-test`, with
+ `tests/isolation/` left to `isolation.yml`, which owns its live stores.
+
+- **`--suite <name>` in `scripts/run_evals.py` ran every suite.** Its `--help`
+ said "runs that suite in isolation", the module docstring said "run a single
+ suite by name", and `args.suite` reached nothing but the wording of the
+ banner: the report dict called all eleven `_run_*` helpers inline. So
+ `--suite mitre_accuracy` took the full runtime, graded eleven gates, and
+ printed `PASS — mitre_accuracy green` — a verdict naming one suite and
+ decided by eleven, with `--ci` able to fail an operator's single-suite
+ bisection on an unrelated regression. The report now carries `suite_filter`
+ and only the requested suites are called; the registry is asserted against
+ `_SUITE_NAMES` so a name argparse accepts cannot be one no runner answers
+ to. A substrate-import failure also exits **3**, which is what this file's
+ own "Exit codes" block has always documented — it exited **2**, and 2 means
+ "MITRE accuracy regressed against the baseline", so a fresh clone missing a
+ dependency reported an accuracy regression, and the one actionable
+ instruction (`pip install -e services/agents`) was not in the message.
+ All three defects were pinned by tests in `scripts/tests/` that had been
+ failing for as long as they existed, in the tree no workflow ran.
+
- **`scripts/check_dependency_pins.py` — fails when any two install paths for
one package disagree.** A package is installed from more than one place: a
manifest, a lock, a Dockerfile, and whichever workflows pip-install a
@@ -1434,10 +1090,59 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
asserts the gate refuses a directory that is not the repository, rather
than printing a confident OK about a tree it never opened.
-- **`.github/workflows/reproducible-builds.yml`.** Four jobs: the pin gate
- and its self-test; `poetry check --lock` across all thirteen Python
- services; the API imported on both ends of the declared FastAPI range; and
- two `--no-cache` builds of the same commit compared package by package.
+- **`scripts/check_toolchain_pins.py` — the Go, Node and toolchain half of
+ reproducible builds.** `check_dependency_pins.py` made every *package*
+ install path agree and stopped at the Python services. Go has `go.sum` and
+ pnpm has a frozen lockfile, so the raw material was already there — but
+ nothing compared the paths that use it, and nothing audited the *runtime*
+ those paths run on. Measured across 111 install paths, and every
+ disagreement found:
+
+ | What | Before |
+ |---|---|
+ | `apps/web/Dockerfile` | `pnpm install --no-frozen-lockfile` — the production web bundle was the one install path in the repository free to resolve its own dependency set, while all 13 workflows installing the same workspace used `--frozen-lockfile` |
+ | `services/realtime/Dockerfile` | `npm install` twice, and `package-lock.json` never copied into the build context, so the committed lockfile was inert and every image build re-resolved |
+ | `install.sh` | `pnpm install --no-frozen-lockfile`, so a self-hoster's install could differ from everything CI tested |
+ | `.devcontainer/devcontainer.json` | `pnpm install --frozen-lockfile=false` — an opt-out a substring test for the flag reads as *enabling* it |
+ | `deploy-docs.yml` | `pnpm --filter @aisoc/docs install`, unlocked; the published docs site could build from a tree no other job resolved |
+ | `apps/web/Dockerfile` | `npm install -g pnpm@8` against `packageManager: [email protected]` — two resolvers, the `poetry 1.7.1 vs 1.8.2` finding again |
+ | Node | 20 in both images, the devcontainer, `deploy-docs.yml` and both installers; 22 in twelve workflows. Node 20 left security support in April 2026 |
+ | `services/enrichment/Dockerfile` | `COPY go.mod go.sum*` — the glob makes the checksum file optional, so deleting it downgrades the build to an unverified resolve without failing |
+ | `ci.yml` | `cache-dependency-path: packages/plugin-sdk-go/go.sum`, a file that does not exist. `setup-go` reports that as a **warning**, so the cache had been silently off while the job stayed green |
+ | `packages/sdk-go` | compiled by no workflow at all |
+
+ Everything above is now one version and one lockfile discipline, proved
+ rather than asserted: two `--no-cache` builds of each Node image resolve
+ byte-identically (web 1,172 packages, realtime 187, matching sha256 both
+ times).
+
+ Six directions, each with a `--self-test` injection that fails if it goes
+ undetected — 22 cases. Beyond agreement: *ship → test* (a version an image
+ ships that no workflow exercises) and *test → ship* (a version CI uses that
+ no image ships, which is the direction versions actually travel); *module →
+ sum* and *sum → module*; *module → CI* and *CI → module*; unlocked install
+ and its reverse, a lockfile nothing can consume; and both coverage
+ directions — a file declaring a toolchain that the gate never opened, and a
+ file it *did* open whose declaration the parser could not read.
+
+ That last direction earned its place immediately. Hunting for this parser's
+ equivalent of the blind spot `check_dependency_pins` found in itself, five
+ turned up: `cd services/${{ matrix.service }}` resolved to nothing, so five
+ of six Go modules looked ungated; `working-directory:` was not read at all,
+ so `services/osquery-extensions` looked uncompiled while the workflow that
+ compiles it sat two directories away; the npm pattern had no left word
+ boundary and matched inside every `pnpm install`; quoted shell text was read
+ as commands, so `die "pnpm install failed."` counted as an install path; and
+ the `node_modules` skip was a prefix rather than a path segment, so on a
+ checkout with dependencies installed the gate scanned 223 vendored manifests
+ and reported Node floors of `0.10` from other people's packages — an answer
+ that depended on whether someone had run `pnpm install`.
+
+ `EXPECTED_ESBUILD` pins the **resolved** esbuild set (0.25.12 bundled by
+ Next, 0.28.1 from the scoped overrides). The overrides are deliberately
+ per-parent because forcing esbuild workspace-wide broke Turbopack's font
+ import map, and that scoping means a `vite` bump can pull a different
+ esbuild with no esbuild line in the diff.
- **`scripts/check_sqlglot_pin.py` now checks the lock, not just the ranges.**
`services/api/Dockerfile` left the list of files declaring sqlglot when its
@@ -1447,289 +1152,294 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
only to each other would have left the gate agreeing about a bound that
nothing installs.
-- **A playbook step that names a response verb now reaches governed
- dispatch, and says honestly what happened to it.** Fifteen of the engine's
- twenty-two step types name an action against somebody's estate. Three of
- them — `block_ip`, `isolate_host`, `create_ticket` — returned
- `{"simulated": true}` from inside the engine, reached no executor, and were
- recorded `SUCCESS`; the other twelve had no handler at all. Meanwhile
- `services/actions` held working executors for fourteen of the fifteen,
- behind a contract declaring each verb's impact, reversibility, approval
- requirement and whether a probe exists to confirm the effect landed. The
- missing piece was never an executor. The comment in `action.py` asserting
- that "playbooks dispatch step by step through this service, so every step
- is graded on the way past" described something that did not happen.
-
- The bridge is `services/agents/app/playbook/action_bridge.py` ->
- `POST /api/v1/playbook-steps/dispatch` -> `live_actions.dispatch()`. It
- lands in the API rather than going direct because that service holds the
- credential vault and the tenant session, and `services/actions` is the only
- place that may change a customer's estate — the same shape, and the same
- reasoning, as `siem_writeback`. **One step, one request, one grading**: a
- playbook is not approved as a unit, so authorising it cannot authorise
- whatever its steps happen to contain.
+- **`.github/workflows/reproducible-builds.yml`.** Four jobs: the pin gate
+ and its self-test; `poetry check --lock` across all thirteen Python
+ services; the API imported on both ends of the declared FastAPI range; and
+ two `--no-cache` builds of the same commit compared package by package.
- `executed` is the single field that means a vendor was touched. A preview,
- an approval queue, a blocked action, a tenant with no integration, a
- credential-less simulation and a vendor failure are each named and each
- `executed: false`, and a step that did not execute is recorded FAILED so
- the run halts under the default `on_failure: abort`. `AWAITING_COMPLETION`
- counts as executed and `PENDING_APPROVAL` does not: collapsing that pair
- either loses an action in flight or invents one that never ran. Execution
- is off by default (`AISOC_PLAYBOOK_ACTIONS_EXECUTE`), so out of the box a
- response step previews and reports a preview.
+- **`Reproducible builds` now proves the property for all thirteen Python
+ services, not one.** The `twice` job asserted byte-identical resolution for
+ `api` alone; the other twelve used the same mechanism and nothing checked
+ it. A nightly matrix (03:17 UTC) double-builds all twelve services that have
+ a Dockerfile and double-installs the thirteenth, `teams-bot`, which has a
+ lock but no image. The matrix is derived from the tree rather than
+ hand-listed, and a `coverage` job fails when the number of services proved
+ is not the number that exist — a scheduled run that silently covers less
+ than it claims is the `wet-eval` failure, eight green weekly runs with every
+ real step skipped.
- **`approval` is the one step type deliberately not bridged.** It is a
- pause, and the engine is a single-threaded index walk with nothing to
- suspend and nothing to wake. It is also no longer the mechanism: every
- response step is now graded individually at dispatch and returns
- `pending_approval` on its own when a human is required, so a gate in front
- of one would gate a decision that is already gated. It fails closed with
- that reason recorded rather than shipping a handler that pretends.
+- **`scripts/check_mypy_baseline.py` — the type check the CI job is named
+ after.** `ci.yml`'s **Python — Lint & Type-check** installed mypy and never
+ invoked it. Six manifests declare `[tool.mypy]` and three set
+ `strict = true`, so three authors had asked for a check that had never once
+ run. It runs now. The 690 findings are recorded in
+ `scripts/mypy_baseline.json` exactly as mypy reports them, keyed on
+ `(tree, file, error-code)` so an error cannot be introduced under cover of
+ fixing an unrelated one — no `ignore_errors`, no widened config, no excluded
+ tree. Both directions: a finding above baseline fails, and a *fixed* finding
+ still in the baseline fails too, so freed headroom must be banked rather
+ than left to absorb the next regression.
- **`run_playbook` as a nested step stays unimplemented, and the calculus was
- re-examined rather than inherited.** Per-step grading was the missing piece
- the previous decision named, and it now exists — but the objection it was
- the answer to does not move: a nested playbook's steps are still not
- visible where the parent declares its policy. What changed is that the
- parent no longer *needs* to bound them, because each step is graded on
- arrival wherever it came from. What has not changed is that the engine
- cannot see a nested playbook's content before running it, so an author
- cannot review what a run will do, and a cycle across two playbooks that
- reference each other is unbounded by the per-step `visited` set. Recursion
- depth and an ancestor set would answer the second; the first is a product
- question about reviewability, not a governance gap, and it is the reason to
- keep waiting.
+ Its first run against a rebased tree earned the gate: 22 findings — 14
+ `union-attr`, 8 `arg-type` — in `services/api/app/services/playbook_step_dispatch.py`,
+ a file merged hours earlier that nothing had ever type-checked. They are
+ recorded rather than fixed because that file belongs to concurrent work;
+ what matters is that they are now visible and cannot grow. Five other
+ findings were real null-safety bugs in `services/fusion` — an
+ `IsolationForest` and an `LGBMRanker` used before training, and the Kafka
+ consumer and producer used before `start()` assigns them — and those are
+ fixed, taking fusion from 7 to 2.
- The schema's `x-aisoc-execution` map gains a `governed` class rather than
- reusing `executed`, because "a handler ran and made an outbound call" and
- "a vendor was touched" are different claims and the second is answered per
- run. `simulated` stays in the vocabulary: it is the class for a handler
- that answers from inside the engine, which is what these three did, and
- deleting the word would make that state unspellable rather than absent.
- `check_playbook_schema_parity.py` compares `governed` against
- `engine.RESPONSE_STEP_TYPES` in both directions, so a verb wearing the
- label while being answered locally fails the build.
+- **All twenty Python trees are type-checked, and the coverage is gated in both
+ directions.** `#818` ran mypy for the first time, over the six trees that
+ declared a `[tool.mypy]` table. Fourteen declared none, so the job named
+ "Lint & Type-check" reported green over 787 Python files it never opened — a
+ tool that appears to cover the repository while covering a third of it is
+ indistinguishable from no tool at all, only more reassuring.
-- **`scripts/upgrade_playbooks.py`, which `docs/upgrade/MIGRATION.md` has told
- operators to run since v4.** It did not exist, so the one command in the
- upgrade path that touches a customer's own content failed at exactly the
- moment it was needed. The field table beside it was worse: four of the five
- spellings in its "v4" column are rejected by the schema
- (`on_failure: {policy}`, `retry: {max_attempts, backoff}`,
- `condition: {expr, language}`, and `type: "action"`, which is not a step
- type at all), so following the doc by hand produced playbooks that would
- not validate either. Both are corrected.
+ Every tree now declares one, matching the two strictness shapes that already
+ existed rather than inventing a third: the ten services take
+ `python_version = "3.11"` / `strict = false` / `ignore_missing_imports = true`
+ like `services/api` and `services/osquery-tls`; the four packages take
+ `strict = true` / `python_version = "3.11"` like `packages/sdk-py`,
+ `plugin-sdk-py` and `aisoc-cli`. No `ignore_errors`, no widened config, no
+ `# type: ignore` added to reach a number.
- The script reports before it writes, validates its own output against the
- real `schemas/playbook.schema.json` before touching anything, and leaves
- alone any file whose upgrade would not validate. `loop`, `parallel`,
- `wait`, `run_playbook`, `action` and `trigger` are reported and refused
- rather than mapped onto a nearest neighbour — that rewrite is precisely the
- defect removed from the NL drafter, where a `disable_user` step shipped as
- `investigate`. An empty scan is a failure, not a clean bill of health.
+ `scripts/check_mypy_baseline.py` gains the coverage check itself, rather than
+ a second gate with its own idea of what a tree is. A Python tree on disk that
+ declares no `[tool.mypy]` fails (`tree -> config`), and a configured or
+ recorded tree that is no longer on disk fails (`config -> tree`) — the
+ direction that rots quietly, because nothing ever fails when a stale entry is
+ simply never consulted. Discovery is now structural, every directory holding a
+ `pyproject.toml`, instead of `services/*` plus `packages/*`: the glob was a
+ naming convention, and a Python tree added anywhere else would have satisfied
+ a coverage gate written against that same glob while being checked by nothing.
+ The gate resolves its root from `git rev-parse` rather than from its own file
+ location, prints how many trees and files were in scope, and its `--self-test`
+ injects coverage drift each way against a throwaway repository.
-- **`scripts/check_connector_profiles.py` — a connector-type drift gate that
- reads in both directions.** Nothing compared the profile keys in
- `services/ingest/internal/normalizer/normalizer.go` against the identifiers
- the connectors service declares, and they had drifted apart in both
- directions at once. The gate enumerates four name spaces — the Go profiles
- and type aliases, the 84 declared `connector_id`s, the `ConnectorType`
- union, and every `connector_type` appearing in a documented example — and
- fails when a name in any of them resolves to nothing.
+- **The 153 Python files that belong to no manifest tree are type-checked, and
+ five gates that reported OK over nothing now fail.** Giving all twenty trees
+ a `[tool.mypy]` table still left `scripts/`, `tests/`, `tools/` and
+ `plugins/` outside every config — which is to say the CI gates themselves
+ were the only Python in the repository nothing type-checked, while every
+ other claim here rests on them.
- It also checks that a documented example reaches a *profile*, not merely a
- declared connector: a reader pastes a flat payload, which never carries the
- canonical envelope, so only a profile or alias can give the event a vendor
- identity. That is the specific check that catches the README defect.
+ They are covered by a scoped invocation under a new root
+ `mypy-unmanaged.toml`, not by a root `pyproject.toml`. Three things were
+ measured rather than assumed: a root manifest carrying only `[tool.mypy]`
+ makes `poetry check` answer *"The Poetry configuration is invalid"* from
+ every directory without a manifest of its own, because poetry searches
+ upward; pytest gains a `configfile` where it had none; and
+ `check_dependency_pins.py`, `check_toolchain_pins.py` and
+ `security_audit.py` all pass with one present only because each is scoped to
+ `services/*` + `packages/*`, so they would miss it by accident of a glob and
+ fail closed on it the day any of them is made structural.
- `--self-test` injects twelve defects, one per direction and per failure
- code, and requires the gate to catch each; it runs in CI on the same tree
- immediately before the gate itself. The resolved repo root, every file read
- and every count are printed before the verdict, and an input that is missing
- or parses empty is a hard error rather than a quiet pass — a gate that
- reports OK about a tree it never opened is worse than no gate.
+ The scope is **computed, never listed** — `git ls-files '*.py'` minus every
+ manifest tree — so a script added anywhere is checked with no edit, and
+ deleting a tree's manifest moves its files into this scope rather than out
+ of coverage. Sixteen `plugins/*/plugin.py` files share one module name in
+ directories whose hyphens keep them from ever being packages, so the run is
+ split into the minimum number of invocations with no collision, derived
+ rather than configured. mypy's own `--linecount-report` must account for
+ every module asked of it before the run may report clean, and a finding
+ against a file outside the scope fails rather than being recorded.
-- **The published playbook schema now describes the engine that runs
- playbooks, and a gate keeps it that way in both directions.**
- `schemas/playbook.schema.json` is the contract authors are told to trust,
- and it had drifted from `services/agents/app/playbook/` in every available
- direction at once. Two schema files existed with different step
- vocabularies — 15 types at the repo root, 9 under `schemas/` — against a
- `StepType` enum of 22, and the NL drafter silently fell back from one to
- the other if the primary was missing. Eleven step types were declared by a
- schema and implemented nowhere (`trigger`, `action`, `loop`, `parallel`,
- `human_approval`, `wait`, `isolate`, `block`, `create_case`,
- `run_playbook`, `script`); thirteen were accepted by the engine and
- declared by neither schema. Six step fields (`blast_radius`, `depends_on`,
- `output_key`, `retry.max_attempts`, `retry.backoff_seconds`,
- `retry.backoff_multiplier`) were declared and never read — `blast_radius`
- carried the description "engine enforces analyst approval for destructive
- steps", and the engine could not see the field at all.
+ Five gates were found reporting OK over a repository containing nothing, by
+ copying `scripts/` into an empty git repository and running all forty:
+ `validate_detections.py` printed a warning and exited 0 over an empty
+ detections corpus — the validator for the number the front page quotes,
+ certifying zero rules as valid; `check_grafana_dashboards.py` failed on an
+ empty dashboards directory and *passed* on a missing one, so the larger loss
+ was the one it forgave; `check_go_module_paths.py` reported "OK: 0 published
+ Go module path(s)"; `check_repo_self_links.py` reported every self-link
+ healthy over zero files, having replaced a lychee run that could not tell a
+ rate-limited 403 from a live link; and `audit_health_probes.py --check`
+ printed a table header and exited 0 over zero services. Each now fails and
+ says what it opened. `check_repo_self_links.py` also reported a file it
+ could not decode as checked, and now names it.
- Resolved by making `schemas/playbook.schema.json` the only schema, widened
- to the full `StepType` range with the condition-as-string form the engine
- already accepted, bounds matched to `bounds.py` (3600s / 25 retries, not
- 600s / 5), and the two authored-but-inert playbook keys (`inputs`,
- `dry_run_support`) declared as the documentation they are. The root
- duplicate is deleted. The schema also carries `x-aisoc-execution`, a
- machine-checked map recording whether each step type is `executed`,
- `simulated`, or vocabulary with no handler — so an author can tell what
- will happen before writing the playbook rather than after running it.
+ The ratchet in `scripts/check_mypy_baseline.py` gains a `file -> scope`
+ direction and goes **919 → 994**: +89 newly visible, −14 fixed. Of the 89,
+ 20 are `import-untyped` against the deliberate no-dependency environment the
+ baseline is recorded in and the rest are annotation and narrowing findings.
+ A separate `--warn-unreachable` pass over the same scope reported 8
+ statements, all one shape: a defensive `isinstance` guard against untrusted
+ YAML or JSON that the parameter's own annotation declares impossible. None
+ is dead at runtime, so the annotations are what is wrong and the flag is
+ deliberately not enabled — recording those 8 would invite someone to delete
+ the guards to clear them.
- `run_playbook` is deliberately **not** implemented as a step, on its own
- merits rather than by inheriting the argument that removed it as an action.
- A nested playbook's steps are not visible where the parent declares its
- step-level policy, so the parent cannot bound them; the engine reads no
- step-level approval or blast-radius field today, so nesting would let one
- ungated parent pull in an arbitrary tree; and twelve of the engine's own
- step types have no handler, so a verb whose purpose is to execute more
- steps would multiply that. It can return when playbook steps are graded
- individually — recursion depth and an ancestor set are the easy part.
+- **`scripts/check_logger_kwargs.py` — no stdlib logger may be called with a
+ structlog keyword.** `logging.Logger.warning` accepts exactly `exc_info`,
+ `stack_info`, `stacklevel` and `extra`; structlog's bound logger accepts any
+ keyword and turns it into the event dict. Both libraries are used in this
+ tree, so `logger.warning("x", reason=exc.reason)` is correct in one module and
+ a `TypeError` in the next, and the two call sites are identical to read — only
+ the binding decides.
- `scripts/check_playbook_schema_parity.py` compares the schema enum, the
- `StepType` enum, the engine's handler table, the execution map, the bounds
- module and the pack validator's trigger list — every pair in both
- directions, because the characteristic failure here is a check that asks
- only whether the schema declares something the engine lacks and never the
- reverse, which is the direction things actually drift. It carries a
- `--self-test` that injects drift each way and fails if any goes undetected,
- refuses to run at all on a tree missing its marker files rather than
- printing OK about files it never opened, and names the root, schema, step
- counts and playbook count it inspected.
+ This needs a checker rather than a grep because of *where* it hides. All five
+ instances `#818` fixed were inside `except` blocks, where the raised
+ `TypeError` replaces the exception being handled and no sibling handler can
+ catch it. A misused keyword on the happy path is found by the first person to
+ run the code; the same keyword in a fallback runs only when something has
+ already gone wrong. The report sorts by context and names how many findings
+ sit on such a path.
-- **Every `ActionType` now resolves a capability contract.** `notify_slack`
- was the last one without, and it was a naming gap rather than a missing
- capability: the verb is `notify`, it has had a contract throughout, and the
- `SlackNotify` adapter already bridged the two names. Nothing connected a
- lookup *by `ActionType` value* to that bridge, so `approval_gate` found no
- contract and skipped the confidence matrix — a 10%-confidence
- `notify_slack` was approved for auto-execution, and the verb most likely to
- auto-execute was the one graded without reference to confidence. It now
- requires an analyst under the default L1 tier.
+ The scanner resolves each module's logger flavour rather than assuming one:
+ every binding, not just `logger`; `self._log` and class-body loggers reached
+ through the instance; `from app.core.logging import get_logger` followed to
+ the defining module, because both in-repo `get_logger` helpers return
+ structlog while their name says nothing; `.bind()` / `.getChild()` chains;
+ annotations, including parameters, for a logger that is handed in; and
+ `logging.warning(...)` on the root logger. A `**kwargs` splat onto a stdlib
+ logger cannot be decided statically and is reported separately rather than
+ counted as clean, and a receiver it cannot classify is counted as
+ `unresolved`, never as clean.
- Closed with a one-entry alias (`ACTION_TYPE_CAPABILITY_ALIASES`) rather
- than a rename, because `action_type` is persisted operator intent:
- `remediation_whitelist` (migration 015) stores per-tenant pre-approvals
- keyed `UNIQUE (tenant_id, action_type)`, so renaming the member silently
- orphans every row an operator created for `notify_slack`. It is also a
- documented request field and a member of the `ActionType` union in
- `packages/types`. The retirement condition is recorded rather than left
- open-ended: the map goes when `ActionType` does. `check_action_contract.py`
- gains two directions — every `ActionType` must resolve a contract, and
- every alias must name a real `ActionType`, point at a real contract, not
- shadow a capability of the same name, and describe a bridge some adapter
- actually implements.
+ It covers all 1,408 Python files including the ~150 under `scripts/`,
+ `tests/`, `tools/` and `plugins/` that belong to no tree and so are checked by
+ no `[tool.mypy]` at all. It classifies 514 stdlib and 970 structlog call sites
+ with 18 unresolved receivers, and reports zero defects — and fails outright if
+ it classifies no stdlib logger, because "found nothing" and "resolved nothing"
+ otherwise print the same word. Pointed at the pre-`#818` files it reports all
+ five, each labelled `[inside except]`.
-- **Competitor product names removed from the docs portal, the benchmark page
- and the archived plan subtree, and a CI gate added to keep them out.** AiSOC
- names no competitor product, but two published comparison tables and the
- competitive-landscape section of the archived plan named eleven vendors
- outright. The analytical content is preserved everywhere: every matrix cell
- survives unchanged and every gap keeps its capability, direction and
- magnitude — only the vendor's identity is gone. The comparison columns on
- `apps/docs/src/pages/index.tsx` and `apps/docs/docs/benchmark.md` now read
- "Open-source SIEM/HIDS" and "Commercial SIEM platform"; the plan's competitor
- profiles and its five-column capability matrix now carry category labels
- ("Autonomous triage", "Hyperautomation SOAR", "Hyperscaler bundle",
- "Vendor-stack XDR", "First-line responder", "No-code automation").
+ Three shapes were added after the first version reported a clean tree and the
+ shapes were then found *in* that tree: an inline
+ `logging.getLogger(__name__).warning(...)` with no binding to look up
+ (`detection_proposals.py`), a class-body logger reached as `self.logger`
+ (`core/config.py`), and an annotation with no factory call in sight. All three
+ were invisible to a scanner that only read assignments of a name.
- - `scripts/check_competitor_names.py` + `scripts/competitor_names.toml` drive
- the gate from an **explicit list of competitor product names**, never a
- heuristic, and allow-list the integration surfaces **by path** — connector
- modules, plugin manifests, connector docs, the connector registry, fixtures
- and tests. This matters because a vendor name is usually correct here:
- `Torq` is simultaneously a first-party SOAR connector and a name that
- appeared in a comparison matrix, so a global replace would have broken
- working connector code. Its connector, manifest, docs page and tests are
- untouched; only the competitive framing changed.
- - Both lists are checked **in both directions**. A pattern that matches
- nothing is dead and fails; an allow entry whose files no longer contain any
- of the names it excuses is stale and fails, so an exemption cannot outlive
- the code it excused; a mistyped name in an allow entry is rejected at load
- rather than silently excusing nothing.
- - A published comparison table is the one place where *any* vendor name is a
- violation, including one AiSOC integrates with, because the table's job is
- to position AiSOC against it. Those two regions are pinned by literal
- start/end markers and checked against a wider vendor list; a marker that
- drifts fails rather than scanning an empty slice and reporting clean.
- - The scan root comes from the working directory or `--root`, never from the
- script's own location, and the run prints the absolute path it walked plus
- the file count — a gate that resolves its own repository can print a
- confident OK about a tree it never inspected.
- - `scripts/check_competitor_names.py --self-test` proves the gate detects a
- known-bad sample and passes a known-good one in both directions, and fails
- if a declared competitor has no fixture. `tests/test_competitor_names_gate.py`
- adds 42 cases covering the stale-allow-list, marker-drift and dual-role-name
- properties. Wired into `.github/workflows/competitor-names.yml`, which runs
- on pull requests *and* pushes to `main` with no paths filter.
+- **CI now runs the interpreter production runs, and `gofmt`, `services/realtime`
+ and the published image list are gated.** Three loose ends `#817` named and
+ did not close.
-- **A gate that fails when a route lets its caller name the tenant.**
- `scripts/check_route_tenant_scope.py` is an AST pass over every route
- decorator and signature under `services/` — a structural property needs a
- structural check, and the thing being looked for (a parameter's name, a
- decorator's `dependencies` list, the router object the decorator hangs off)
- survives renames a regex would miss. It fails in **both** directions: a
- route that takes a tenant identifier with no auth dependency, and a route
- that accepts one without intersecting it with the caller's scope. It reads
- request-body models too, because `POST {"tenant_id": "<someone else's>"}` is
- the same hole as `?tenant_id=` and the mutating routes carry it on the body.
+ *Python.* Twenty-four workflows ran 3.12 while all thirteen service images
+ ship 3.11. Every manifest declares `^3.11`, which permits both, so nothing
+ written down was violated — which is exactly why it survived. Standardised on
+ **3.11**, in 41 replacements across those 24 workflows, because 3.11 was
+ already the answer everywhere except CI: the images ship it, the devcontainer
+ installs it, `ruff.toml` targets `py311`, five of six `[tool.mypy]` tables set
+ `python_version = "3.11"`, and all twenty-two manifests floor at 3.11 or
+ below. Moving the images to 3.12 instead would have meant changing all of
+ those *and* raising the published floor for seven installable packages — a
+ breaking change for downstream consumers, to fix a CI hygiene problem. Testing
+ at the declared floor is also the stronger guarantee. Nothing broke: 2726 API
+ tests, eleven service suites at their coverage floors and three Python SDKs all
+ pass on 3.11, and the mypy baseline is byte-identical (because those five
+ `python_version` pins meant mypy was already checking 3.11 semantics while
+ running on 3.12). `PYTHON_INTERPRETER_SPLIT` is deleted rather than emptied: a
+ split that can be recorded is a split that can grow.
- `--self-test` injects a violation of each kind and asserts both are caught
- while two clean controls pass, so the gate cannot quietly stop detecting
- things. Exemptions each carry a written reason and are as narrow as the
- facts allow: `services/mesh` is public by design (Ed25519 + k-anonymity, a
- bearer token would break federation rather than secure it); three MSSP
- routes *define* a scope rather than read within one, so intersecting would
- make them impossible; and `osquery-tls` `/enroll` authenticates with a
- per-tenant enroll secret because osqueryd has no session yet. That last
- exemption is **conditional** — the self-test strips the verifier it names
- and asserts the route is then reported, so deleting the check and keeping
- the entry fails the build. The gate resolves its repository root from `git
- rev-parse` rather than its own file location, and prints how many routes
- across how many files it opened, so an OK can be distinguished from a scan
- that never happened. `--inventory` prints the per-service table.
+ *gofmt.* CI ran `go vet` and `go build` and never `gofmt`; 28 files across four
+ modules had drifted. The new job walks every `.go` file rather than iterating a
+ matrix, because a matrix is a list somebody has to remember to add to — which
+ is how `packages/sdk-go` came to be compiled by nothing. Formatting landed as
+ its own commit; nine files also had reStructuredText ``code`` markers in Go doc
+ comments, which have no inline code span, so gofmt was rewriting them to two
+ *left* curly quotes. The 58 marker pairs are gone, which leaves gofmt stable.
-- **Tool attribution is now prevented at commit time and blocked in CI.** AiSOC
- does not attribute work to a development tool or AI assistant. An audit found
- the rule was being broken automatically: a `Co-authored-by:` trailer naming an
- editor appeared in 214 commits on `main`, and 280 of 732 pull request bodies
- carried a "Made with" or "Generated with" footer. Nothing in the repository
- caused it — no `commit.template`, no `core.hooksPath`, nothing in
- `.git/hooks/` — the editor appended it at commit time, and it survived
- `git commit --amend`.
+ *`services/realtime`.* Published to GHCR on every release and built by nothing
+ — no build, test, lint or type-check job anywhere — despite being one of the
+ two ends of the Kafka spine, carrying the OpenTelemetry instrumentation that
+ keeps the distributed trace continuous, and holding the TypeScript CORS guard.
+ It now has all four, installing with `npm ci` from its committed lock.
+ `@types/node` moves `^20` → `^22` to match the Node 22 runtime, and a
+ `tsconfig.check.json` type-checks the test file, which `rootDir: src` had left
+ checked by nothing.
- - `.githooks/commit-msg` strips the line before it reaches a commit. It is
- wired through `core.hooksPath`, so it is version-controlled and shared with
- every clone rather than living in an untracked `.git/hooks/`;
- `scripts/setup_hooks.sh` installs it and runs automatically as the `prepare`
- script on `pnpm install`. The hook rewrites and never rejects — a hook that
- can block is a hook that can halt someone's work on a bad pattern, so the
- blocking job belongs in CI where it is visible.
- - This is load-bearing rather than cosmetic because the repository
- squash-merges with `squash_merge_commit_message=COMMIT_MESSAGES`: GitHub
- composes the squash commit body from the branch commits, so a trailer on any
- branch commit is copied onto `main` at merge time.
- - `scripts/check_attribution.py` fails CI when attribution appears in a
- commit message, a changed file or the PR body. A human `Co-authored-by:`
- line is never flagged — a pattern only fires when the trailer names a known
- tool — and `dependabot[bot]` is allowlisted.
- - The gate runs on `push` to `main` as well as on `pull_request`. A PR-only
- check is one-directional: it inspects what contributors propose but never
- what actually lands, so anything introduced by the merge itself would pass
- it while the gate stayed green.
- - `--self-test` runs on every CI invocation and checks four directions against
- a shared fixture corpus: the gate detects every known-bad sample (so it
- cannot pass vacuously), flags no known-good sample (so it cannot pass
- direction one by flagging everything), and the hook strips exactly what the
- gate flags in both directions (so the two implementations cannot drift).
- Both read their patterns from the same `.githooks/attribution-patterns.txt`.
- The self-test earned its place immediately by catching a real hole in the
- patterns — `Built with GitHub Copilot` slipped through, because a vendor
- word sat between the verb and the tool name.
+ Three new bidirectional pairs in `check_toolchain_pins.py` — `module -> format`
+ / `format -> module`, `image -> CI` / `CI -> image`, `target -> ship` /
+ `ship -> target` — each of which rediscovers its defect when run against the
+ previous commit. Five parser blind spots were found and fixed in the gate
+ itself, every one present in this tree: matrix legs read file-wide (so
+ `ci.yml`'s two `service:` matrices merged), the block-list matrix form
+ unreadable, `working-directory` truncated at the space inside a matrix
+ expansion, `working-directory` unreachable when it is a step's first key, and
+ `gofmt` matched inside step names and quoted `echo` strings — the last of which
+ also let a quoted path in a `compose-smoke.yml` shell array count as CI
+ coverage for eleven services, so the check would have reported OK about the
+ very gap it exists to find. Each published image now prints the step that
+ exercises it instead of contributing to a tally, and the gate asks
+ `git rev-parse` for its root instead of inferring it from its own file
+ location. 35 self-test cases, 49 unit tests.
+
+- **`scripts/audit_health_probes.py` now checks the thirteen copies of
+ `app/_health.py` are identical.** The module's own docstring claimed they
+ were kept in sync and nothing verified it — they happened to be, so the
+ claim was true by luck. It also pointed at `scripts/sync_health_module.py`,
+ which does not exist in this tree.
+
+- **A gate that reads the one place a dead path could get in.** A comment is
+ the only prose no check in this tree opens: `check_repo_self_links.py`
+ reads markdown and matches `github.com` URLs, lychee resolves links, and
+ neither looks inside source files — which is where most cross-file
+ references in this repository actually live. That is how
+ `docker-compose.yml` came to name
+ `docs/architecture/decisions/0001-llm-gateway-in-core.md`, a path with no
+ directory behind it, and survive every gate. `scripts/check_comment_paths.py`
+ reads comments in fourteen languages plus Python docstrings and asserts
+ that every repository path they name exists.
+ The corpus was measured before the gate was written, because a naive
+ version would have been mostly noise: of **3,092** path-shaped strings in
+ non-markdown comments, most are MIME types, MCP method names
+ (`tools/call`), Splunk REST routes (`services/search/jobs`), container
+ images or URL routes. Three structural filters — a known file extension, a
+ first segment that names something at the top of this repository, and not
+ a URL / template / glob / gitignored artefact — cut that to **1,251**
+ claims about this tree, resolved against the repository root *and* each
+ ancestor of the referring file (a comment in `services/agents/tests/`
+ saying `tests/conftest.py` means the one beside it).
+ Extension-less directory references are **deliberately not checked** and
+ the gate says so in its own output: 286 resolve and 52 do not, and
+ hand-classifying the 52 put genuine rot at roughly one in five, which is
+ the precision at which a gate gets ignored.
+ Ten exceptions are recorded, each with a reason and each verified in both
+ directions — an entry whose path now resolves, or which no longer appears
+ in any comment, fails the build rather than sitting there.
### Changed
+- **A hosted deployment's hostname no longer appears in self-hosted docs as the
+ reader's own URL.** Seven files under `apps/docs/` pointed at the managed
+ instance: two `curl` examples told a self-hoster to push their SIEM data to
+ somebody else's ingest endpoint, a sample address appeared in a console
+ illustration, three white-paper links and a benchmark-scoreboard link sent
+ open-source readers to the hosted console, and a JSON Schema `$id` claimed the
+ hosted docs domain. These now use `example.com`, in-repo GitHub links, or the
+ project's own documentation URL. Pages genuinely *about* the managed offering
+ keep naming it, as does the docs build configuration.
+
+- **The Slack bot no longer deep-links an unconfigured deployment to somebody
+ else's console.** `AISOC_WEB_BASE_URL` defaulted to a hosted hostname, so a
+ self-hoster who deployed the bot without setting it got case cards pointing at
+ another instance. It now defaults to `http://localhost:3000`, matching the
+ compose default it had silently disagreed with, and both documented defaults
+ were corrected with it.
+
+- **A permanent playbook-step failure is no longer retried as if transient.**
+ `dispatch_step` raises before any I/O when the run context has no tenant,
+ and the engine retried it with exponential backoff: 2s, 4s, then 8s, to
+ arrive at the message it already had on the first attempt. The cost is not
+ the fourteen seconds — it is that a permanent misconfiguration presents to
+ an operator mid-incident as flakiness, so their next move looks like "wait"
+ when it is "go and set the variable".
+
+ `BridgeUnavailable` stays the base class every caller catches, and the
+ permanent half is now `BridgeMisconfigured`, marked with a new
+ `PermanentStepFailure` that any handler can raise and the engine honours.
+ The line: **permanent** when the cause is this deployment's configuration or
+ a violation of the API's own contract — the enable switch, the service
+ token, the missing tenant, a non-retryable 4xx, a JSON body with no
+ `executed` field; **transient** when the cause is reachability — a transport
+ error, any 5xx, `408`/`425`/`429`, and a body that did not parse as JSON at
+ all, which is overwhelmingly an ingress error page rather than the API. A
+ failed step records `permanent` and `attempts`, so the run says why it was
+ tried once and not four times.
+
- **`Backup → destroy → restore` and `docker compose up — full stack` report
a verdict on every pull request, so both can be required checks.** Both
workflows were path-filtered at the workflow level, and a workflow that does
@@ -1768,26 +1478,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
that. The derived set matches the old list exactly today, so nothing about
today's behaviour changes.
-- **A permanent playbook-step failure is no longer retried as if transient.**
- `dispatch_step` raises before any I/O when the run context has no tenant,
- and the engine retried it with exponential backoff: 2s, 4s, then 8s, to
- arrive at the message it already had on the first attempt. The cost is not
- the fourteen seconds — it is that a permanent misconfiguration presents to
- an operator mid-incident as flakiness, so their next move looks like "wait"
- when it is "go and set the variable".
-
- `BridgeUnavailable` stays the base class every caller catches, and the
- permanent half is now `BridgeMisconfigured`, marked with a new
- `PermanentStepFailure` that any handler can raise and the engine honours.
- The line: **permanent** when the cause is this deployment's configuration or
- a violation of the API's own contract — the enable switch, the service
- token, the missing tenant, a non-retryable 4xx, a JSON body with no
- `executed` field; **transient** when the cause is reachability — a transport
- error, any 5xx, `408`/`425`/`429`, and a body that did not parse as JSON at
- all, which is overwhelmingly an ingress error page rather than the API. A
- failed step records `permanent` and `attempts`, so the run says why it was
- tried once and not four times.
-
- **The tenant-predicate gate can now tell a guard from a log line, and the
ratchet shrank from 34 to 32.** It credited any query addressed by a key
passed to a call that also received the caller's tenant — which
@@ -1865,584 +1555,394 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
stayed unwired. It is wired now, and passes: all 13 tested Python services
are gated.
-- **A hosted deployment's hostname no longer appears in self-hosted docs as the
- reader's own URL.** Seven files under `apps/docs/` pointed at the managed
- instance: two `curl` examples told a self-hoster to push their SIEM data to
- somebody else's ingest endpoint, a sample address appeared in a console
- illustration, three white-paper links and a benchmark-scoreboard link sent
- open-source readers to the hosted console, and a JSON Schema `$id` claimed the
- hosted docs domain. These now use `example.com`, in-repo GitHub links, or the
- project's own documentation URL. Pages genuinely *about* the managed offering
- keep naming it, as does the docs build configuration.
-- **The Slack bot no longer deep-links an unconfigured deployment to somebody
- else's console.** `AISOC_WEB_BASE_URL` defaulted to a hosted hostname, so a
- self-hoster who deployed the bot without setting it got case cards pointing at
- another instance. It now defaults to `http://localhost:3000`, matching the
- compose default it had silently disagreed with, and both documented defaults
- were corrected with it.
-
-### Security
-
-- **The four services that run their own alembic chain now migrate as the
- owner and serve as the runtime role, and three of them were not reached by
- the role switch at all.** `honeytokens`, `osquery-tls`, `purple-team` and
- `ueba` manage their own schema, and each applied it as whatever DSN the
- operator supplied — so their migration and runtime credentials were the same
- one, and pointing such a service at the owner turned off row-level security
- for the twelve tables those chains own with nothing objecting. Each
- `env.py` now reads `<SERVICE>_DATABASE_MIGRATION_URL`, then
- `DATABASE_MIGRATION_URL`, and only then falls back to the runtime DSN with a
- warning on stderr naming what will break.
-
- Three things were found while wiring it, each measured rather than inferred:
-
- - **`DATABASE_URL` was inert on three of the four.** `docker-compose.yml`
- sets it on every service, but `honeytokens`, `purple-team` and
- `osquery-tls` declare `env_prefix` in their settings, so the name they
- read is `HONEYTOKEN_DATABASE_URL` and the compose entry did nothing: each
- fell back to a default naming the **owner**. Both spellings now resolve,
- unprefixed first, the convention `services/ueba` and `services/fusion`
- already used. Operator note: on a deployment that sets both to *different*
- databases, the unprefixed one now wins.
- - **The four chains shared one `alembic_version` table** — they share one
- database in the default deployment and number their revisions identically.
- Following `apps/docs/docs/quickstart.md` against `postgres:16`: after
- `ueba` reached `0002`, `honeytokens alembic upgrade head` ran **zero**
- migrations and `purple-team` failed applying its RLS revision to tables
- that had never been created, so two services had no tables and no policies
- and every command reported success. Each chain now keeps its own version
- table and adopts an existing deployment's recorded version on the next
- upgrade — only when that chain's own tables are already present, so it
- cannot claim a sibling's row.
- - **The runtime role had no grant on those tables.**
- `061_runtime_app_role.sql` grants over `ALL TABLES` as they stood and sets
- `ALTER DEFAULT PRIVILEGES` for the role that issued it; nothing orders the
- chains against it, and a deployment applying them under different owners
- gets neither. Each chain now grants `SELECT, INSERT, UPDATE, DELETE` on its
- own tables, guarded so it is a notice rather than a failure where the role
- does not exist.
+- **`ruff` moved to the 0.16 line across all fourteen declarations, the tree
+ was reformatted under it, and the lint gate stopped ending at `services/`.**
+ The bump alone reds every open pull request, because `ruff format --check`
+ is a hard gate on the required `Python — Lint & Type-check` job and 0.16
+ formats differently from 0.4.10. So the migration is the reformat. Measured
+ under 0.16.9 (what the range resolves today, and byte-identical to 0.16.8 on
+ both format and lint): **218 files reformatted**, 862 insertions against
+ 1075 deletions, almost all of it collapsing calls that 0.4.10 wrapped and
+ 0.16 fits inside the 140-column limit. It is its own commit with no other
+ change in it.
- Verified live on `postgres:16` with all four chains applied and two tenants
- seeded: bound to one tenant the runtime role sees one row of two in
- `ueba_entity_baselines`, `honeytokens` and `osquery_node`; unbound it sees
- both, which is the fail-open arm the cross-tenant sweeps depend on; a
- cross-tenant insert is refused by the policy; `CREATE TABLE` is refused with
- `permission denied for schema public`; `ALTER TABLE … NO FORCE ROW LEVEL
- SECURITY` with `must be owner of table`; and `SET LOCAL row_security = off`
- raises rather than doing nothing. None of the four chains creates a view, so
- the `security_invoker` hazard does not arise there, and none of the four
- services issues DDL outside its chain.
+ All fourteen declarations now read `>=0.16.8,<0.17` — seven manifests
+ (`services/{api,fusion,osquery-tls}`, `packages/{aisoc-cli,aisoc-detections,
+ plugin-sdk-py,sdk-py}`), three workflows (`ci.yml`, `ai-sdk.yml`,
+ `python-detections.yml`), `.devcontainer/Dockerfile`, and the three
+ `poetry.lock` files that resolve it, now at 0.16.9.
-- **`approval_timers` is in a migration and carries a policy.**
- `services/slack-bot`'s `PostgresTimerStore` created it at startup with
- `CREATE TABLE IF NOT EXISTS`, outside every chain in the repository — so
- `060_rls_coverage.sql` never saw it, it had no `tenant_id` for a policy to
- filter on, and it decides whether a pending containment auto-rejects. Under
- the DML-only runtime role the DDL itself fails, because Postgres checks the
- schema ACL before the existence test, and `main.py` caught that into a
- warning and fell back to the non-durable store: durable approval timers
- would have quietly stopped being durable. `062_approval_timers.sql` adds the
- table, a `tenant_id`, and the canonical policy; the store probes with
- `to_regclass` and refuses with the migration's name rather than creating
- anything, carries the tenant in every statement, and binds
- `app.current_tenant_id` on each pooled connection so the policy engages too.
- A deployment already carrying the runtime-created table converges on the
- same shape, with its existing rows defaulting to an empty `tenant_id` that
- the migration says how to backfill.
+ The three lint findings 0.16 reports on `services/` are fixed rather than
+ suppressed: two `C420` dict comprehensions become `dict.fromkeys` (both
+ values are string literals, so the shared-value hazard the rule exists
+ around does not apply), and one `UP031` percent-format becomes an f-string
+ in a chunker test that counts characters — all three rewrites asserted
+ equal to the originals before the change was kept.
-- **The services no longer connect to Postgres as a superuser, so the 92
- row-level-security policies added below now actually filter.** The previous
- entry closed the coverage gap and measured, in the same breath, that none of
- it did anything: `docker-compose.yml`, the CI service containers, the Helm
- chart and the Terraform environment all ran every service as
- `POSTGRES_USER=aisoc`, which the postgres image creates as a **superuser**,
- and a superuser ignores policies *even under* `FORCE ROW LEVEL SECURITY` —
- FORCE binds the table owner, not a superuser. Sixty-one new policies bought
- nothing operationally.
+ **Scope.** The gate read `ruff check services/`, which left `scripts/` —
+ every CI gate in this repository, the code the rest of its claims are
+ verified by — linted by nothing at all. That is the same gap
+ `check_mypy_baseline.py` closed with its unmanaged scope, and it is closed
+ the same way here: lint now covers `services/ scripts/ tests/ tools/`, the
+ trees the repo-root `ruff.toml` governs that nothing else lints, and
+ `ruff format --check .` covers the whole repository because formatting has
+ no semantic content and `.` needs no edit when a tree is added. That
+ surfaced 17 real findings outside `services/`, each fixed at the source: two
+ `B007` loops switched to `.values()`, a `B904` re-raise in
+ `security_audit.py` now chains `from exc` so the original traceback
+ survives, two `E402` imports that genuinely follow a `sys.path` mutation
+ carry a reason next to the marker, and two over-length lines were wrapped.
- `061_runtime_app_role.sql` splits the credential in two. `aisoc` owns the
- schema and applies migrations; `aisoc_app` is what every service connects
- as, holding `USAGE` on `public`, `SELECT / INSERT / UPDATE / DELETE` on
- tables and views, `USAGE, SELECT` on sequences and `EXECUTE` on functions —
- no `CREATE`, no `TRUNCATE` (`002_rls.sql` had granted `ALL`, which let one
- statement delete every tenant's rows without a policy seeing a `WHERE`
- clause), and ownership of nothing. Measured on `postgres:16` with two alerts
- seeded one per tenant and the session bound to tenant A, reading `alerts`
- with **no tenant predicate at all**: the old role saw 2, the new role sees
- 1, and an insert for tenant B is refused by the policy.
+ Two exclusions, both named rather than implied. `plans/` is excluded in
+ `ruff.toml`: it is the archived prototype subtree `codeql.yml` and
+ `check_mypy_baseline.py` already exclude and the project rules forbid
+ editing, and it holds 844 lint findings and ~280 unformatted files — they
+ are excluded because they cannot be fixed, which is not the same as being
+ clean. `E501` is ignored for `scripts/detection_specs.py` and
+ `scripts/detection_specs_part2.py`: both open their rule table with
+ `# fmt: off`, so the formatter is already opted out of that layout by a
+ deliberate decision, and all 199 long lines there are single dict or string
+ literals — 72 positive fixtures, 66 negative fixtures, 50 `match_when`
+ clauses, 11 descriptions. Two named data modules, not `scripts/`; exactly
+ one long line elsewhere under it needed wrapping.
- Things this turned up along the way, each of which would have survived the
- role switch and quietly undone it:
+ `packages/` and `plugins/` stay outside the lint gate on purpose and the
+ gap is a number rather than a shrug: 91 findings and 2 respectively. Two of
+ the five packages are already linted by their own workflows under their own
+ configs, and bringing the three published SDKs under this gate means
+ behaviour-touching fixes — `subprocess.run` check semantics, blind
+ `except Exception` handling, `zip(strict=)` — that belong in a change
+ reviewed as such rather than in a formatter migration.
- - **Two views read around every policy underneath them.** A view executes as
- its *owner* unless declared `security_invoker`, and both views here are
- owned by the role that ran the chain. Bound to tenant A,
- `mssp_tenant_latest_metrics` returned 2 rows before and 1 after.
- - **`SET LOCAL row_security = off` stops being a no-op and becomes a
- crash.** For a role the policies apply to, Postgres refuses the query
- rather than ignoring the policy. The retention purge, the hunt scheduler's
- sweep and tenant deletion all used it; all three now rely on the
- `OR current_tenant_id() IS NULL` arm and call `assert_cross_tenant_session()`
- first, which raises if a tenant *is* bound — a sweep that sees one tenant
- and reports success is worse than one that fails.
- - **`CREATE TABLE IF NOT EXISTS` still needs `CREATE` on the schema when the
- table already exists**, because Postgres checks the ACL before the
- existence test. Two stores in `services/agents` opened their pool that way
- and swallowed the failure into `logger.debug`, so the agents service would
- have silently recorded no cost telemetry and no institutional memory at
- all. Both tables are already in the migration chain; the bootstrap now
- probes first and reports at `error` if it genuinely has to create one.
+- **`mypy` moved to 2.3.1 across all ten declarations, and the baseline it
+ ratchets became reproducible outside CI.** The bump alone makes
+ `scripts/check_mypy_baseline.py` exit 1 on 35 mismatched `(tree, file, code)`
+ entries, so the re-record is the work — and a re-record is the moment a
+ ratchet can quietly grow, so every entry was triaged before being banked.
- Deployment surfaces updated: `docker-compose.yml`, the demo compose stack,
- `integration.yml`, the Helm chart's `values.yaml`, the Terraform environment
- (which now generates the runtime role's password rather than reusing the RDS
- master user) and `.env.example`. `infra/postgres/initdb/zz_runtime_role_password.sh`
- sets the credential on a fresh volume before the container reports healthy;
- `app.scripts.run_migrations` applies it on every run, which covers an
- upgrade in place. **Operators on a managed Postgres must act**: apply the
- chain as the owner with `AISOC_APP_DB_PASSWORD` set, then point
- `DATABASE_URL` at `aisoc_app` and `DATABASE_MIGRATION_URL` at the owner.
- `002_rls.sql` created `aisoc_app` with the literal password `changeme`; 061
- does not clear it (that would break an operator who had already set a real
- one), so rotate it if none of the automatic paths applies to you.
+ **Zero new findings.** All 35 are the same code in the same direction:
+ `import-untyped` went 38 → 3 and nothing appeared, nothing grew. The cause
+ is a behaviour change rather than a code improvement, and is recorded as
+ such: mypy 2 extends `ignore_missing_imports = true` to silence
+ `import-untyped`, which 1.x did not. Isolated on a two-line file — under
+ 1.20.2 the finding is reported with the option either way; under 2.3.1 it is
+ reported only with the option off. So the ten trees that set it lose the
+ "this third-party import carries no type information" signal, every one of
+ the 35 was `yaml`, and `packages/aisoc-cli` — strict, with no such option —
+ still reports its three. `enable_error_code = import-untyped` does not
+ bring it back; `ignore_missing_imports` wins. The alternative, dropping the
+ option, would surface `import-not-found` for every uninstalled dependency in
+ a deliberately dependency-free run, which is a statement about the runner
+ rather than the tree. Banked at 955, with the reason on the record.
- Two new gates, both with a self-test that runs before the scan and both
- failing over an empty or absent tree. `scripts/check_runtime_db_role.py`
- fails if a runtime DSN connects as the role that surface provisions as the
- database superuser — structurally, by reading `POSTGRES_USER` /
- `db_username` out of the tree rather than matching a name — and in
- `--dsn` mode reads `pg_roles` and `pg_class` directly, catching
- `rolsuper`, `rolbypassrls`, ownership, a view without `security_invoker`,
- and a role still accepting `changeme`. `scripts/check_rls_policy_shape.py`
- fails if a policy is written without the fail-open arm, replaying the chain
- in order so a definition a later migration repaired is not reported. Both
- run in `isolation.yml`; the live half runs in `integration.yml`.
+ **The baseline was only reproducible on a CI runner, which is not
+ reproducible.** Measured rather than assumed: the interpreter is not the
+ variable people assume — Python 3.13 against 3.11 moves exactly 2 findings
+ in 1 file (PEP 701 changed f-string sub-expression line attribution in 3.12,
+ splitting findings 3.11 reports once) — while installing eight project
+ dependencies moves 59. mypy reads type information out of installed
+ packages, so a contributor with a service virtualenv active got mismatches
+ in both directions and nothing saying the cause was their environment. A
+ second cause surfaced while measuring the first: a `.mypy_cache` written
+ under different conditions is reused by the next run, and the same tree
+ answered 990 cold against 988 warm.
- `tests/isolation/test_postgres_rls.py` now reads as `DATABASE_URL` — the
- role the deployment ships — instead of a `NOSUPERUSER NOBYPASSRLS` role it
- constructed for itself, so its 80-table two-tenant replay covers what ships.
- `test_superuser_bypasses_rls_which_is_why_the_probe_role_exists` is replaced
- by its inverse.
+ Both are now closed at the source. The gate passes `--no-site-packages`
+ (measured on `services/connectors`: 73 findings bare, 140 with eight
+ dependencies installed, 73 either way with the flag — and a no-op in CI's
+ own dependency-free environment, verified by re-recording under 1.20.2 and
+ getting the committed file back byte for byte) and `--no-incremental`. It
+ records the mypy version in an `(environment)` block and refuses to compare
+ across majors, because a major *moves* findings and every difference would
+ otherwise print as a file-level diff that describes the version. The
+ interpreter difference is reported as a note rather than a refusal, so a
+ contributor on 3.12 can still run the gate. Verified: the same 955 from a
+ bare interpreter and from one with the project's dependencies installed.
-- **Row-level security covered 31 of 95 tenant-scoped tables; it now covers 92,
- and the seven policies that already existed but could never work are
- repaired.** On the other 64 tables the query predicate was the only thing
- between two customers, so one missing `WHERE tenant_id` was a leak rather
- than something a second layer caught — which is not the design the
- repository documents. `060_rls_coverage.sql` adds a policy to every
- remaining table in the API chain, and the four services that manage their
- own schema (honeytokens, osquery-tls, purple-team, ueba) each carry a
- matching alembic revision. The three that remain are named rather than
- rounded away: `users` is excluded so authentication can resolve a principal
- before a tenant exists, and `case_tasks` / `case_timeline` are ORM models
- that no migration creates.
+ `mypy` is now in `check_dependency_pins.py`'s `CRITICAL` set, so the ten
+ declarations have to move together the way `ruff`'s fourteen do. Nothing
+ enforced that before, which is how a single-path bump could be proposed at
+ all. One real bug was caught by the re-recorded baseline while this was
+ being written — a `used-before-def` introduced into the gate's own
+ self-test — and fixed rather than banked.
- Four things found along the way, each invisible for the same reason:
+- **`sqlglot` moved to the 30 line across all eight declarations, and the
+ forward-compatibility matrix leg was rewritten so it can still reach past
+ the pin.** The bump itself was never the work: a single-path proposal moved
+ 1 of 8 declarations and left six workflows at `>=23,<27`, which
+ `scripts/check_sqlglot_pin.py` fails closed on — correctly, since that gate
+ exists because sqlglot 27 renamed the SELECT's FROM argument key and
+ silently stripped the tenant predicate, the table allowlist and the `url()`
+ exfiltration ban from every single-table query while reporting success. All
+ eight now read `>=30,<31` (`services/api/pyproject.toml` plus `ci.yml`,
+ `integration.yml` ×2, `isolation-live.yml`, `cross-tenant-rbac.yml`,
+ `check-openapi.yml`, `reproducible-builds.yml`), and
+ `services/api/poetry.lock` resolves 30.19.0 inside it.
- - **The application bypasses RLS entirely.** `docker-compose.yml` and the CI
- service containers run every service as `POSTGRES_USER=aisoc`, which the
- postgres image creates as a superuser, and a superuser ignores policies
- even under `FORCE ROW LEVEL SECURITY`. Measured, not inferred: with two
- alerts seeded one per tenant and the session bound to tenant A, that role
- sees both and a `NOSUPERUSER NOBYPASSRLS` role sees one. The security doc
- claimed the opposite — "there is no superuser escape hatch via the
- application's DB role" — and now carries the grant that makes it true.
- - **Seven policies were keyed on a session variable nothing sets.**
- `alert_sla_events` and `tenant_sla_config` read `app.tenant_id`;
- `custom_parsers` and `retention_policies` read `app.current_tenant`;
- `compliance_evidence` had no unset-context arm. All five returned zero
- rows once RLS engaged. `external_assets` and `external_asset_drift` called
- `current_setting` without `missing_ok`, so an unbound session raised
- `unrecognized configuration parameter` instead. All seven are normalised
- to the canonical predicate.
- - **The agents service had the mirror-image bug.** Its four
- `_set_rls_context` helpers wrote `app.tenant_id` while every policy reads
- `app.current_tenant_id`, so the scoping they exist to provide had never
- been applied — the policies fell through their fail-open arm every time.
- - **Five tables had RLS enabled without `FORCE`**, so the table owner —
- which is the application — walked past the policy regardless.
-
- `tests/isolation/test_postgres_rls.py` is the evidence, wired into
- `integration.yml`'s migrations job where a real database with the full chain
- already exists. It seeds an A row and a B row into every RLS-covered tenant
- table (80 of 80 in the API chain), asserts both are visible unscoped before
- asserting either absence, then binds the session to A and asserts B's row is
- unreachable. It also asserts the shipped role's bypass, so a green run here
- can never be read as "tenant isolation is on in production".
-
-- **The attack-path relational fallback is verified against a real Postgres.**
- `GET /graph/attack-path/{case_id}` reads `aisoc_cases` by id whenever the
- Neo4j traversal fails — which for a deployment shipping no graph database is
- always. It gained a tenant predicate in the previous release, but that fix
- was covered by a gate and by review only: the statement uses
- `CAST(:cid AS UUID)`, which SQLite mangles, so no offline suite could execute
- it. Tenant B naming tenant A's case UUID is now demonstrated to be refused
- against the database the query is written for, with tenant A's row asserted
- present first.
-
-- **Fifty-eight routes across four services carried no authentication at all,
- and the previous gate could not see them.** `check_route_tenant_scope.py`
- asks a conditional question — *if* a route takes a tenant, where did the
- tenant come from — so a route that takes no tenant was never in its reach.
- 37 `services/agents` routes took none. Reproduced against the real routers
- with credential material configured, so the result is not "the service was
- unconfigured": an anonymous caller with no `Authorization` header created a
- playbook (201), listed all 64 (200), **executed one** (202) and deleted it
- (204), then read copilot conversations and ran a threat hunt. All seven
- refuse with 401 now, while a valid console session still gets a non-empty
- response.
-
- `services/agents` keeps **dual-mode** auth rather than a bearer-only
- scheme, because the console reaches it directly through a Next rewrite on a
- session cookie: the guard is #813's `require_console_or_service_auth`,
- extended rather than replaced. Its WebSocket could not use the same
- dependency — a browser cannot set an `Authorization` header on a handshake,
- and an `HTTPException` has no defined rendering on a WebSocket scope — so
- `_ws_principal` reads the credential from the header or `?token=`, verifies
- it with the *same* vendored logic, and closes with 1008 before `accept()`.
- That route previously took its tenant from a query parameter defaulting to
- the literal `"default"`, which names no tenant anywhere in this schema, and
- would start a fresh investigation on the connection.
-
- Also closed: `connectors` (9 data routes, including the one that decrypts a
- saved instance's credentials to test them and the two that push into a
- customer's ITSM), `fusion` (5), `osquery-tls` (7), `threatintel` (1),
- `actions` (1) and 20 in `services/api` — among them `PUT /deployment/config`,
- `POST /deployment/airgap/bundle`, `POST /compliance/evidence/collect`, both
- LLM-backed `/translate` routes and the seven STIX/TAXII routes.
-
- Two counts in the original report were **wrong, in the safe direction**, and
- the reason matters: `slack-bot` (5) and `teams-bot` (4) were never open. An
- AST pass sees no `Depends` and calls a route unauthenticated, but Slack Bolt
- verifies a request signature, `/approval-card` compares a shared internal
- token in constant time, and the Teams webhook verifies an HMAC-signed card
- payload with a replay window. `scripts/check_route_auth.py` models these as
- **conditional** exemptions that name the verifier and lapse the moment the
- handler stops calling it.
-
-- **`scripts/check_route_auth.py` — default-deny over all 601 routes.** Every
- route must carry an auth dependency or appear in one of three tables, each
- recording why it is public. It reuses the tenant-scope gate's AST collector
- rather than adding a second parser: building it surfaced that
- `playbooks.py`'s `ExecuteUser = Annotated[AuthUser, Depends(require_permission(...))]`
- was reported as an unauthenticated playbook-run, because the vocabulary
- listed `ReadUser` and `WriteUser` and nobody thought of the third. Auth
- aliases are now resolved by **what they wrap**, which removes the naming
- dependency in both directions — the reflex fix of appending the new name to
- the list is how a list stops describing anything. State after: 517
- authenticated, 73 public with a recorded reason, 11 verified in-band, 0
- unexplained.
-
-- **`scripts/check_tenant_query_predicates.py` — the predicate question.**
- Closing the parameter shape surfaced eight routes matching on an id with no
- tenant predicate; the sweep that found them was opportunistic. This gate
- asks the question structurally over the whole tree, and the model and table
- inventories are **derived from the tree** (a model's `tenant_id` column, the
- migrations' DDL) rather than listed, so a new migration cannot slip past.
- It matters most where nothing stands behind it: of 95 tenant-scoped tables
- only 31 carry an RLS policy, and RLS engages only on a session that ran
- `SET LOCAL app.current_tenant_id`.
-
- Real leaks it found beyond the known eight:
-
- - `GET /graph/attack-path/{case_id}`'s **relational fallback** read
- `aisoc_cases` by id with no tenant predicate. The primary Neo4j path is
- scoped, but the fallback runs precisely when that path failed — and its
- own docstring says it exists so deployments without a graph database keep
- working, i.e. permanently for many of them. An authenticated caller naming
- another tenant's case UUID got its title, severity, MITRE techniques and
- alert ids.
- - `GET /osquery/distributed/{query_id}` was **both** unauthenticated and
- unscoped. `osquery_distributed_query` has no `tenant_id` of its own — it
- is reached through its node — so the lookup now joins onto
- `osquery_node.tenant_id`. Demonstrated with two seeded tenants: before,
- tenant B naming tenant A's `query_id` got A's host telemetry back; after,
- nothing.
- - `alert_explain._resolve_rule_lineage` selected a detection rule by an id
- lifted out of the alert's `raw_event` — vendor-supplied, so a crafted
- event could name another tenant's rule and have its definition explained
- back.
- - The ITSM webhook inserted its system comment into `aisoc_case_comments`
- **without `tenant_id` at all**, leaving rows belonging to no tenant since
- migration 044 added the column.
- - Thirteen by-id writes (`alerts` escalate/snooze/update, four `connectors`,
- three detection-rule routes, `claim_alert`, `run_saved_hunt`) were scoped
- only by a preceding read. Not exploitable as written, and one reorder from
- being scoped by nothing.
-
- What cannot be decided statically sits on a **shrink-only ratchet** — 34
- entries, each with a reason, `MAX_RATCHET` asserted against the table's
- length, and an entry whose statement is now scoped failing as *stale*. That
- last property caught two of this change's own edits.
-
-- **The zero-CodeQL-alert invariant was documented for four months with
- nothing enforcing it.** `apps/docs/docs/operations/security.md` has said
- since 2026-05-15 that "the Python alert count on `main` is zero, and we
- treat that as a CI gate — a new alert breaks the security workflow". No such
- gate existed anywhere in the repository. `codeql.yml` uploads SARIF and
- `github/codeql-action/analyze` does not fail a build on findings; `main` has
- no branch protection, so "Code scanning results" was not a required check
- either; `security.yml`'s only hard job is the claim-to-gate matrix; and
- nothing in the tree queried the code-scanning API. The scan itself was
- healthy — `main` was analysed continuously, most recently minutes before
- this was written — so the usual stale-green and mixed-`codeql-action`-pin
- traps were both ruled out. The enforcement was simply imaginary, which is
- why two `note`-severity alerts could sit open on `main` under a documented
- count of zero.
-
- `scripts/check_codeql_alerts.py` is that gate, wired into the new
- `.github/workflows/codeql-alert-gate.yml` on push to `main`, on pull
- requests, on a `workflow_run` after CodeQL finishes, and daily. It fails on
- any open CodeQL alert at **any** severity — `note` included, since both
- motivating alerts were `note` and carried no `security_severity_level`, so a
- threshold anywhere would have reproduced the original silence exactly. It
- also refuses the vacuous pass: an unanalysed ref, a declared language with
- no analysis, an analysis older than ten days, or one belonging to a
- different commit than the one that triggered the run are all failures rather
- than a clean bill of health, and an input it cannot read exits 2 rather than
- 0. Dismissed alerts stay excluded — that is GitHub's audited escape hatch
- and the repository uses it for ~20 accepted-risk
- `py/request-without-cert-validation` findings — but the count is printed on
- every run so a silent mass-dismissal is visible. `--self-test` injects an
- alert at each severity plus every shape of vacuous pass and requires the
- gate to catch all ten; CI runs it immediately before the gate itself.
-
-- **`scripts/validate_playbooks.py` printed to stderr and called `sys.exit(2)`
- while being imported** (CodeQL `py/print-during-import`, alert #896). Beyond
- the note, this was a live defect:
- `scripts/check_playbook_schema_parity.py` imports the module to read
- `SUPPORTED_TRIGGERS` and wraps the import in `except Exception`, which
- cannot catch `SystemExit` — a broken environment would have killed the
- parity gate's interpreter instead of producing its diagnostic. The import
- now raises `ImportError`, which that handler catches.
-
-- **Thirty routes across six services let the caller name the tenant they were
- reading.** `/fusion/entity-risk/*` was the reported instance and the worst
- one: three routes on the API gateway and three on the fusion service took
- `tenant_id` as a query parameter with no auth dependency at all. The console
- reaches fusion *directly* through a Next rewrite when `FUSION_URL` is set, so
- those routes were reachable from any browser on the internet, and an
- anonymous request naming another tenant's UUID returned that tenant's
- entity-risk queue, stats and per-entity detail. The engine's own docstring
- asserted the missing control — "the tenant_id is part of the key prefix and
- the API service re-checks tenant on read" — and the second half was not true
- on either side. Key prefixing isolates whichever tenant it is handed;
- validating the parameter's *value* fixes nothing, because a UUID that parses
- is still a UUID the caller chose.
-
- An AST pass over all 600 routes in `services/` found the same shape in five
- more places, in two flavours. Taking a tenant with no authentication:
- `services/agents` `/triage/{run_id}`, `/cases/{id}/triage`,
- `/cases/{id}/investigate`, `/investigations` and `/explain`; and six
- `services/osquery-tls` routes. Taking one *with* authentication but never
- intersecting it with the caller's scope: `honeytokens`, `purple-team` and
- `ueba`, where a router-level service token proved the caller was a trusted
- service but said nothing about which tenant it was acting for.
+ One major wide rather than four. The previous range permitted 23–26 and
+ exactly one of those was ever exercised, so three majors were installable
+ and untested; 27, 28 and 29 are likewise unexercised, and a range states
+ what may be installed rather than what upstream has released.
- The tenant now comes from the credential. A new
- `app/security/tenant_scope.py`, vendored into the six services that need it,
- resolves either a console session (the first-party HS256 access token, whose
- verified `tenant_id` claim is authoritative) or a trusted service declaring
- the tenant it acts for on `X-AiSOC-Tenant-ID`. A `tenant_id` on the request
- survives only as a *filter*, intersected with that scope, so an MSSP
- operator can still narrow to one managed customer while naming an outside
- tenant narrows to nothing and returns 403. A service token that declares no
- tenant resolves to an empty scope and is refused: absent is never all, which
- is the shape every cross-tenant leak in this codebase has had. The HS256
- verification is stdlib-only and mirrors `services/realtime/src/auth.ts`,
- rejecting `alg: none`, a refresh token presented for access, and an expired
- or wrongly-signed token.
+ Measured before moving, on 64 adversarial cases written against the
+ documented policy rather than reusing the module's fixtures — DDL, DML,
+ admin verbs, multi-statement input, seven ClickHouse table functions
+ including ones hidden in subqueries and CTE bodies, non-allowlisted tables
+ reached through JOIN, subquery, UNION arm and CTE, and five forged
+ `tenant_id` predicates. 26.33.0 and 30.19.0 return the same disposition on
+ all 64 (23 rewritten, 33 `LakeSqlForbiddenError`, 8 `LakeSqlSyntaxError`)
+ and the rendered SQL of all 23 rewritten cases is byte-identical. Two cases
+ differ in wording only and both remain refusals: `DETACH TABLE` now reaches
+ the statement-kind check instead of failing to parse, and `KILL`'s class
+ moved to `sqlglot.expressions.ddl`, which the parse error quotes. The
+ in-tree suites pass 58/58 on each major.
- Closing these surfaced a worse variant the parameter audit could not see:
- eight routes matched on an id with **no tenant predicate at all**.
- `honeytokens` `GET/PATCH/DELETE /{token_id}` and `/{token_id}/triggers`,
- `purple-team` `PATCH /executions/{id}/detection` and the three
- `/tabletop/{session_id}` routes, and `ueba`
- `PATCH /anomalies/{id}/acknowledge`. Any caller could read, revoke or delete
- another tenant's honeytoken, overwrite another tenant's detection outcome,
- or acknowledge away another tenant's anomaly by naming its UUID. All eight
- now filter on the caller's tenant as well as the id, so a foreign row is a
- 404 — which is what it is, from that caller's point of view.
+ The forward leg is the part that needed a decision. It read `>=27,<31`,
+ which only reached past the pin while the pin sat below 27; there is no
+ sqlglot 31, so with the pin on the 30 line every way of naming a ceiling is
+ worse than leaving it off — `>=31` cannot be installed and reds the job
+ forever, `>=30,<31` duplicates the shipped leg and reports two greens for
+ one piece of evidence, and retargeting to `>=27,<30` tests majors below the
+ pin, which is regression coverage for versions nothing installs and leaves
+ nothing between the next major and production. The leg is now unbounded
+ above, so it resolves whatever is newest and arms itself the day a higher
+ major is published with no edit. Because that means it re-runs the shipped
+ version today, a `Forward coverage` step emits a warning annotation saying
+ so rather than letting a duplicate green pass for evidence, and
+ `check_sqlglot_pin.py` gained the structural half in both directions: the
+ forward leg must carry no upper bound and must not float below the shipped
+ floor, and the step's `SHIPPED_CEILING` must equal the shipped range's `<N`
+ bound so a stale copy cannot report the wrong verdict. Seven injected
+ violations in `--self-test`, which runs before the gate. What is
+ deliberately not asserted is that a higher major exists: upstream's release
+ schedule is not this repository's to require, and a gate red until sqlglot
+ ships 31 would be red for a reason no change here could fix.
-- **`services/osquery-tls` enrolled every node under the literal `"default"`,
- which resolves to no tenant on any seeded deployment.** The service keys
- `tenant_id` as a `String(64)` while the platform keys UUIDs, and nothing
- translated. Migration `001` seeds the canonical tenant with slug `default`,
- but the demo seed renames that slug to `demo`, so the literal matched
- neither the UUID nor the slug and silently matched nothing: FIM events were
- written under a string the console could never ask for, and the FIM surface
- returned an empty table that looked like "no file changes" rather than "the
- read and the write disagree". `app/services/tenant_resolver.py` now resolves
- the placeholder to the canonical seed tenant by its **stable UUID** —
- ignoring whatever the slug has been renamed to — falling back to the sole
- tenant of a single-tenant install, and refusing enrolment for a genuinely
- unknown ref rather than filing the node under an unreadable tenancy. Skips
- log at `warning` with the ref; a silent `debug` skip is how the original bug
- survived.
+- **Four root-workspace dependency bumps landed as one lockfile change.**
+ `cytoscape` 3.34.0 → 3.34.3, `lucide-react` 1.28.0 → 1.48.0,
+ `@modelcontextprotocol/sdk` 1.29.0 → 1.30.1 and `tsx` 4.23.1 → 4.23.15 all
+ resolve through the single root `pnpm-lock.yaml`, so merging them
+ separately makes each one rebase the next until they all report
+ `CONFLICTING`. Regenerated once from the four manifests together: 47
+ insertions against 56 deletions, versus 1,130 insertions across the four
+ individual proposals, because each of those re-churned the same peer keys.
+ The `esbuild` resolutions are byte-identical before and after — `0.25.12`
+ and the `vite>esbuild` / `tsup>esbuild` / `bundle-require>esbuild`
+ override's `0.28.1` — so the `tsx` bump does not reach Turbopack's bundler,
+ which is the reason those overrides are scoped to named parents rather than
+ applied workspace-wide.
-- **The maintainers' hosted origin shipped in the default CORS allow-list of
- nine services.** `services/{api,agents,connectors,honeytokens,purple-team,ueba}`
- (six byte-identical copies of the shared `cors.py`), `services/realtime`, and
- the Go `ingest` and `enrichment` servers all listed `https://tryaisoc.com` and
- `https://www.tryaisoc.com` among the origins they trust when
- `AISOC_CORS_ORIGINS` is unset. That is one deployment's public origin baked
- into every self-hosted install, trusted for credentialed cross-origin
- requests its operator never opted into — and if that domain ever changed
- hands, the grant travels with it. The default is now local development only;
- the hosted deployment already sets `CORS_ORIGINS` explicitly
- (`infra/fly/api/fly.toml`), so nothing legitimate depended on the default.
- `apps/docs/docs/deployment/env-vars.md` documented the old list and was
- corrected with it, and a new gate pins the six vendored `cors.py` copies
- byte-identical — nothing enforced that before, and a single drifted copy is
- exactly how one service would quietly keep the origin.
+- **Three more root-workspace bumps landed as one lockfile change.**
+ `@testing-library/user-event` 14.6.4 → 14.6.7, `autoprefixer` 10.5.2 →
+ 10.6.1 and `zustand` 5.0.14 → 5.0.15, regenerated once from the three
+ manifests together: 19 insertions against 22 deletions, versus 636
+ insertions across the three individual proposals. The only transitive
+ movement is `caniuse-lite` 1.0.30001803 → 1.0.30001810, which
+ `autoprefixer` carries as a data table. Verified on the merged result
+ rather than on the three green ticks, because none of these three has a CI
+ job that exercises what it changes beyond the shared web gates:
+ `pnpm install --frozen-lockfile`, `eslint .` (0 errors), `tsc --noEmit`,
+ `vitest run --coverage` (62.17% statements, gate green), `next build` and
+ `storybook build` all pass.
-- **The seeded demo identity was a live, operator-owned domain.**
- `[email protected]`, paired with a published password, appeared across
- compose, fly, render, coolify, railway, two workflows, the web Dockerfile and
- the docs. It told a self-hoster to type somebody else's hostname to sign in
- to their own install, and published a well-known credential pair against a
- real domain. Now `[email protected]` — RFC 2606 reserved, so it can never be
- registered and can never receive mail, and it still satisfies the
- `pydantic.EmailStr` check that rejected the earlier `[email protected]`. Every
- config and doc moved with it. Safe to re-run: `seed_demo._ensure_user`
- reconciles on `DEMO_USER_ID`, not on the address, and rewrites a stale email
- in place.
+- **Five more root-workspace bumps landed as one lockfile change.**
+ `@vitest/coverage-v8` 4.1.10 → 4.1.11, `axe-core` 4.11.4 → 4.13.0,
+ `react-hot-toast` 2.6.0 → 2.6.1, `framer-motion` 12.43.0 → 13.4.0 and
+ `zod` 4.4.3 → 4.6.5 (the last in `services/mcp`), regenerated once: 61
+ insertions against 84 deletions, versus 1,040 insertions across the five
+ individual proposals.
+ The `@vitest/coverage-v8` bump is the first pull request the new `vitest`
+ group produced, and it closes the exact-version peer drift that group was
+ added for — `pnpm install` stops reporting
+ `unmet peer @vitest/[email protected]: found 4.1.10`.
+ `framer-motion` is a major and what installs is **13.4.3**, not the 13.4.0
+ in the proposal's title, because `^13.4.0` resolves above it. Verified on
+ 13.4.3 rather than on the proposed number: `tsc --noEmit` passes, which is
+ the gate that would see a removed API, and this app's entire framer-motion
+ surface is `motion`, `AnimatePresence` and `useReducedMotion` across 37
+ import sites. v13 drops the optional `@emotion/is-prop-valid` peer that
+ v12 declared, and this workspace does not use emotion. 615 tests pass with
+ zero errors and coverage unchanged to the digit (62.17% / 57.71% / 54.77% /
+ 64%), `next build` generates all 104 static pages, and `storybook build`
+ succeeds.
-- **`/api/v1/identity-timeline` read every tenant's alerts.** Both routes bound
- an authenticated user and never used it: the SQL against `aisoc_alerts`
- carried no `tenant_id` predicate, so any authenticated caller could pull any
- tenant's alerts whose title or evidence matched a substring — and the
- substring is the search term, so the match is caller-controlled. Both routes
- are now scoped to the caller's tenant.
+### Fixed
-- **`/api/v1/playbooks` had no authentication at all.** The module declared no
- `Depends` of any kind across eight routes and there is no global auth
- middleware, so every route was reachable unauthenticated — including
- `POST /playbooks/{id}/run`, which executes a playbook against the estate.
- Each route now demands `playbooks:read`, `playbooks:write` or
- `playbooks:execute`; all three permissions already existed in
- `ROLE_PERMISSIONS` and had no reader. `:execute` stays distinct from
- `:write` so an analyst can run a governed playbook without editing one.
+- **A new user who followed only the README could not log in.** `make up`
+ worked, `make smoke` passed 8/8, and then authentication was impossible for
+ three independent reasons at once. The only account was `[email protected]`,
+ and `LoginRequest.email` is a pydantic `EmailStr`, which rejects RFC 6761
+ special-use domains — so the address returned `HTTP 422 "value is not a
+ valid email address: The part after the @-sign is a special-use or reserved
+ name"` *before the password was ever compared*. The bcrypt hash migration
+ 001 seeded matched neither the `changeme` that four documentation pages
+ published nor the `admin` its own inline comment claimed; checked with this
+ service's `verify_password`, every candidate returned `False`, so it
+ corresponded to no known secret. And nothing existed to create a user with.
+ `make demo` repaired it incidentally by writing a valid address, which left
+ the demo path working and the path for running AiSOC on your own data
+ broken.
-- The case-timeline linked-alert hydration in `cases.py` now binds a tenant as
- defence in depth. Reaching it already required a tenant-scoped case, so this
- was not a live read, but a poisoned `alert_ids` array would otherwise have
- surfaced another tenant's alert title.
+ The first administrator is now created by the deployment rather than
+ committed to the repository. `services/api/app/scripts/bootstrap_admin.py`
+ creates `[email protected]`, generates a password, and prints it once —
+ it is stored nowhere. `make up` calls it, so the documented quick start ends
+ with a credential on screen; `make bootstrap` runs it on its own and
+ `make bootstrap ARGS=--reset-password` mints a new one. It is idempotent: a
+ second run reports the existing account and changes nothing, which is what
+ makes it safe for `make up` to call unconditionally. The address is
+ validated with the same library the login route uses, so an address that
+ cannot sign in is refused here with an explanation instead of becoming an
+ account that fails at a login form with a schema error.
-- **Any authenticated user could disable detection rules inside any other
- tenant.** `_ensure_mssp_parent`, the guard on the MSSP write surface, had
- `pass` for a body. Four routes took a caller-supplied child tenant id and
- wrote it onto a row without checking whose child it was.
+ Migration 001 no longer seeds a user at all, and `059` deactivates the
+ orphaned row on databases that already ran it. A gate
+ (`tests/test_first_run_gate.py`) now asserts that no migration ships a
+ password hash and that every login example in the documentation uses an
+ address the API accepts — the four pages were each self-consistent with the
+ broken seed, which is why reading them found nothing.
+ `golden-pipeline.yml` runs `make bootstrap` against the live stack and
+ authenticates with the credential it printed, so the quick start's last step
+ is covered by the same job that covers its first.
- The consequential one was `POST /api/v1/mssp/overrides`. An override with
- `action: "exclude"` is read back by `resolve_effective_rules`, filtered on
- `child_tenant_id == <the reader's tenant>`, and the rule is popped out of the
- set `POST /api/v1/rules/hunt` runs. So naming another tenant's id silently
- deleted a named detection from their hunts, and the victim's only symptom was
- a hunt that stopped matching. Reproduced against the previous commit: a
- tenant's effective ruleset went from one critical cloud rule to zero on an
- override written by an unrelated tenant.
+- **The README never told anyone to create `.env`.** `make doctor` correctly
+ reported it missing and printed the fix; the quick start it was meant to
+ support skipped the step. It is now the second line of the quick start.
- Closing those four routes alone would not have been enough, because
- `POST /api/v1/mssp/children/{id}/onboard` let anyone *become* the parent
- first — its only check was a `409` when the target already had a parent, so
- every standalone tenant on a deployment was adoptable by any authenticated
- user. Adoption now requires the child to have invited that specific parent by
- setting `settings.mssp_parent_invite` through `PATCH /api/v1/tenants/me/settings`,
- which only ever writes the caller's own row and is gated on `settings:write`.
- The invite is single-use. The child-scoped routes answer `404` rather than
- `403` for a tenant that is not yours, so they cannot enumerate tenant UUIDs.
+- **`make up` printed an API docs URL that 404s.** The README was corrected on
+ its own; the tooling was not, so `make up`, `install.sh`, `install.ps1`,
+ `scripts/lab.sh` and three documentation pages went on telling every new
+ user to open `http://localhost:8000/docs` while the app mounts `/api/docs`.
+ `test_readme_api_docs_url.py` now walks the tool output as well as the
+ README.
-- **`python-jose` is gone, and `ecdsa` with it.** `ecdsa` carried
- CVE-2024-23342 (Minerva timing attack on P-256) with no patched release —
- OSV records the affected range as introduced at 0 with no fixed event,
- because upstream states python-ecdsa offers no side-channel resistance and
- will not fix it. It was an unconditional requirement of `python-jose`, so
- the suppression was renewed rather than resolved. `services/api` now signs
- and verifies with `PyJWT`, which it already depended on for the OIDC and
- SAML paths; `python-jose`, `ecdsa` and `rsa` all leave the dependency tree.
- Six CI workflows installed `python-jose[cryptography]` and never installed
- `PyJWT`, and reached `cryptography` — a declared direct dependency of
- `services/api` — only through that extra; they now install both by name.
+- **`make doctor` reported a port held by a foreign process as held by us.**
+ The check was `docker compose ps -q <service>`, which lists containers in
+ any state — so a Postgres that exited *because* the port was taken still
+ counted, and the doctor printed "port 5432 in use by aisoc postgres" while
+ an unrelated process had it. The two remedies are opposites. It now asks
+ which host port our own container actually publishes, names the container or
+ process that holds the port otherwise, and reports a service already
+ remapped to a different port as fine rather than as a conflict.
-- **`image-size` moved to a patched release instead of staying suppressed.**
- Both advisories were held open on the reading that no fix existed. They
- record a vulnerable range of `<= 2.0.2`, and npm has published 2.0.3 and
- 2.0.4; a null `first_patched_version` is not the same claim as no fix
- existing. A pnpm override pins `>=2.0.4 <3`, which
- `@docusaurus/mdx-loader`'s `^2.0.2` range accepts. This clears the only two
- high-severity advisories in the pnpm workspace.
+- **`make up` hit port conflicts with no explanation.** `docker compose up`
+ reports `port is already allocated` against whichever container lost the
+ race, after a minute of unrelated output and without naming what holds it.
+ `make up` now runs the port check first (`doctor.sh --ports-only`) and stops
+ before starting anything.
-- **The dependency suppression list is empty.** All 42 entries in
- `scripts/security_audit_ignores.txt` were re-verified against OSV and
- against the versions the lockfiles actually resolve. Every one was
- resolvable, and most of the justifications had stopped being true: nine
- starlette entries blamed a `fastapi<0.137` cap that exists in neither bot
- service, three cryptography entries blamed `<50`/`<49` caps that exist
- nowhere in the repository, and the langchain, weasyprint, anyio, aiohttp,
- h2, idna and pydantic-settings entries each named a version older than the
- one their lock resolves. The file now records what was measured, so the next
- review starts from evidence rather than from the previous reason string.
+- **The compose security note recommended a remedy that does nothing.** Both
+ compose files told operators to change a host binding with a
+ `docker-compose.override.yml`. Compose *appends* sequences when it merges,
+ so an override without `!override` publishes the new binding alongside the
+ old one and leaves the conflict in place. Both notes now show the tag, and
+ `ports: !reset []` for removing a publishing entirely.
-- **The pnpm and Go arms of the audit could report success without scanning.**
- A failed `govulncheck`, an unparseable `pnpm audit` response and a registry
- that never answered were all recorded as warnings, which exit 0 — so the job
- printed "0 findings" for ecosystems it had not read. All three now record a
- coverage gap, which `exit_code_for` already failed on for the Python arm.
- That property — an unscanned target fails the build — had no test; it does
- now.
+- **AI triage could not reach a model in the default deployment, and it was
+ not a missing key.** `docker-compose.yml` set `LLM_GATEWAY_URL` on the `api`
+ and `agents` services. Nothing read it. Both resolvers honoured only
+ `OPENAI_BASE_URL`, which compose never set, and each carried a comment
+ explaining that the refusal was deliberate — routing through the gateway
+ should be an explicit choice so it is never ambiguous whether the bearer is
+ the gateway master key or a provider key. The reasoning was sound and the
+ result was that every `aisoc-<role>` alias went to `api.openai.com`, which
+ cannot resolve one, and the caller's `except` rendered that as "no LLM
+ available". `preflight_llm()` logged exactly this condition at boot with the
+ remedy, so the diagnosis existed and the wiring did not.
-### Fixed
+ `LLM_GATEWAY_URL` is now the lowest-precedence base URL, and the ambiguity is
+ resolved rather than avoided. **The model decides the route:** the gateway is
+ adopted only for a gateway alias, which resolves nowhere else, while a
+ concrete `AISOC_MODEL_PIN_<ROLE>` — the documented direct-to-provider escape
+ hatch — is left pointing at its provider. **The route decides the bearer:**
+ when AiSOC picks the gateway itself it sends `LITELLM_MASTER_KEY`, which
+ compose now supplies to both services, and never a provider key. An explicit
+ `OPENAI_BASE_URL` still outranks both. The rule lives once, in
+ `services/agents/app/llm/routing.py`, shared with the BYOK resolver and
+ mirrored in `services/api/app/services/model_aliases.py` for the service that
+ cannot import it.
-- **The connector catalog proxy never authenticated, and served a stale list
- on every request because of it.** `_fetch_catalog()` issued a bare
- `client.get(url)` with no `Authorization` header at a connectors service
- that is default-deny on every route, so it was answered `401` on every
- single call and fell through to the catalog bundled in the API image *every
- time*. The bundle had **26 entries against a live registry of 84**. Nothing
- failed: the wizard rendered a confidently wrong list, and because
- `connector_type` is validated against that same list, the 58 connectors it
- had never heard of were rejected as "unknown connector_type".
+ Consumers audited and fixed alongside it: the API copilot and the NL-query
+ translator (both already went through the shared helpers); the agents
+ **contextual copilot**, which built `ChatOpenAI(model=…)` with no base URL at
+ all and reported "LLM not configured" from a key check that could not see the
+ gateway's; the **detection builder** (`/nl-detection`), the only consumer that
+ could not be pointed anywhere — it POSTed to a hardcoded
+ `https://api.openai.com/v1/chat/completions` with `OPENAI_MODEL`; the four API
+ endpoints (translation, hunts, knowledge base, phishing) that read
+ `LLM_BASE_URL` and defaulted to OpenAI without consulting `OPENAI_BASE_URL`;
+ the BYOK / "explain this alert" resolvers in both services, which defaulted
+ straight to `https://api.openai.com` and took an alias there; and
+ `GET /llm/status`, so the indicator describes the route the explain path
+ actually takes. The **MITRE RAG embedding** path is a deliberate exclusion —
+ `infra/litellm/config.yaml` declares chat aliases only, so an embedding call
+ routed there would 400 on every batch. It has its own
+ `AISOC_EMBEDDING_BASE_URL` / `AISOC_EMBEDDING_MODEL` pair and the gate checks
+ the exclusion in both directions.
- The proxy now presents the service credential and the tenant it is acting
- for, mirroring the fusion gateway. Every call site passes the tenant
- explicitly — it is a required parameter, not a defaulted one, because a
- defaulted parameter is one call sites forget.
+- **The LLM gateway moves from the `full` profile into CORE, so the documented
+ default install can actually do AI triage.** It was `full`-profile on the
+ reasoning that the gateway is "only needed when a provider key is
+ configured" — which skipped a step: every task role resolves to an
+ `aisoc-<role>` alias, and an alias resolves at the gateway and nowhere else,
+ so a CORE deployment could not use a key either. `make up`, the quickstart's
+ Path B, and a plain `docker compose up -d` all now start it.
- The fallback is kept, and given a stated job: **keep the console usable when
- the connectors service is not deployed or not answering.** For that to be
- defensible it has to be distinguishable from the real thing, so
- `/connectors/catalog` now returns `source` (`live` / `bundled`), `degraded`
- and `reason`. A deployment with no connectors service is `bundled` but not
- degraded — there the bundle *is* the source of truth, and flagging it would
- train operators to ignore the flag.
+ With no provider key it boots, serves its seven aliases and answers
+ `/health/liveliness` (verified against the bundled config with both provider
+ keys empty); AiSOC makes no LLM call and the deterministic path is
+ unchanged. CORE goes from 10 to 11 services and ~6 GB to ~6.5 GB — the image
+ is 1.67 GB and idles at 451 MiB, measured, so "one lightweight container"
+ was not true and the README's numbers moved rather than staying put. The
+ gateway is also the only party that can report what a call cost, so CORE's
+ cost figures were previously unmeasurable by construction. Reasoning and the
+ two rejected alternatives: `docs/decisions/0006-llm-gateway-in-core.md`.
+ While correcting that table, the `full` row's service count was checked and
+ was wrong: `make up-full` starts **21** services, not 30 (30 is `full` plus
+ the `monitoring`, `chatops`, `extras` and `osquery` profiles).
- While degraded, an unrecognised `connector_type` returns **503 rather than
- 422**: a connectors service rolled ahead of the API image legitimately knows
- types that image does not, and "unknown connector_type" sends an operator to
- debug a connector that is fine.
+- **`OPENAI_MODEL` replaced the triage role's model on the highest-volume path
+ in the product.** `.env.example` shipped `OPENAI_MODEL=gpt-4-turbo-preview`,
+ a model no gateway config in this tree has ever defined. The auto-triage
+ worker layers a tenant's BYOK configuration over the role pin — but the
+ resolver's `LlmConfig` collapsed provenance into a single `source` word, so
+ the worker could not tell a per-tenant override from the process-wide env
+ baseline and bound all three fields unconditionally. Once traffic reached the
+ gateway that would have produced `Invalid model name` on every triage call:
+ the failure the commercial deployment already hit, where the copilot and every
+ triage agent degraded to empty output.
+
+ `LlmConfig` now carries `model_from_tenant` / `base_url_from_tenant` /
+ `api_key_from_tenant`, computed from state the resolvers already tracked, and
+ only a field the tenant actually set is treated as an override. A real BYOK
+ model still wins. `.env.example` ships `OPENAI_MODEL=aisoc-summary` — an alias
+ the bundled gateway defines — and the variable is documented as the BYOK /
+ explain path only, never a task role.
+
+- **A model with nowhere to go is now an error instead of a fallback.**
+ `make_chat_model` and `chat_completions_url` raise `UnroutableModelError`,
+ naming the alias and the remedy, when a gateway alias is requested with no
+ gateway configured — rather than building a client destined to 404 and
+ letting a caller read that as "the LLM is unavailable". `run_auto_triage`
+ constructs its model inside its own `try` so the failure arrives as
+ `AutoTriageError` like every other LLM failure; `auto_triage_node` catches
+ that specifically, so constructing outside it would have failed a whole graph
+ run over a configuration problem the deterministic path handles. The
+ deterministic floor is unchanged and still reports `call_count=0 models=[]
+ total_tokens=0` with no `recommended_actions`. `preflight_llm()` now reports
+ both directions at boot: an alias with no gateway, and a concrete model
+ pointed at the bundled gateway.
-- **The bundled catalog is generated, not refreshed by hand.**
- `scripts/generate_connector_catalog_fallback.py` builds it from the
- connector registry and `--check` fails the build on drift, so the artefact
- cannot fall behind the registry it copies. The gate compares **three**
- independent readings of connector identity, in both directions: what
- `_CONNECTOR_CLASSES` declares (read through the AST by
- `generate_connector_types.parse_registry`, so there is one definition of
- "the registry" rather than two), what reaches `CONNECTOR_REGISTRY` at
- runtime, and what each `cls.schema()` **advertises**.
+- **The natural-language playbook drafter could never reach an LLM.**
+ `nl_drafter._llm_factory` called `make_chat_model()`, and `role` is a required
+ positional parameter; every other caller in the tree passes one. The
+ `TypeError` was caught by the `except Exception` two lines down, which logged
+ "no chat model available" and returned `None`, so the drafter fell back to its
+ deterministic substrate path on every call, permanently, while reporting the
+ condition as a missing provider. Every test monkeypatches `_llm_factory`, so
+ the one line that mattered was the one line nothing exercised. Now passes
+ `"nl"`, the declared role in `app.llm.model_pins`.
- The third is the reading nothing looked at. `list_connector_schemas()`
- iterates the registry but takes each entry's `connector_id` from the schema,
- so a class registered as `tenable_io` whose `schema()` said `tenable` would
- be offered in the wizard under a name `get_connector_class()` cannot
- resolve — 84 independent opportunities for two declarations to disagree.
- Identity is read off the class, never the filename: `jira_connector.py`
- declares `jira` and `tenable.py` declares `tenable_io`, and `--self-test`
- asserts that property rather than trusting a comment. The self-test runs
- before the gate in CI and injects nine defects — a dropped connector, a
- renamed one, a class that never registers, a deleted artefact, an unhooked
- consumer, an empty corpus — requiring each to be caught.
- **UEBA could never write a baseline or an anomaly, and said it was
healthy the whole time.** Three separate defects on one code path, each
reached by the first scoreable event rather than by some rare branch:
@@ -2520,463 +2020,627 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
and `/health` answers 503 listing any detached topic instead of a
hardcoded `status: 'healthy'`.
-- **Nothing invoked the four alembic chains.** `honeytokens`,
- `osquery-tls`, `purple-team` and `ueba` own their schemas through
- alembic; every container command was a plain `uvicorn`, so the documented
- quickstart booted them with empty schemas and none of their
- row-level-security policies. Each image now runs `python -m app._migrate`
- before its server: it applies the chain as the owner credential, takes a
- shared transaction-scoped advisory lock so four chains starting at once
- queue instead of deadlocking, reads the applied revision back, and
- refuses to start the server unless it matches head. An exit code is not
- evidence — `alembic upgrade head` exits 0 when it applies nothing, which
- is exactly what happened when the chains shared one version table. All
- four also gained a compose healthcheck; they had none, which is why a
- container that had done nothing still read as `running`.
+- **A consumer that retried a permanent failure in silence, forever.** The
+ `graph_ws` broadcaster in `services/ingest` could not die the way the UEBA
+ consumer did — its loop `continue`s past an error — but it answered every
+ error the same way: a flat 50ms sleep with the error discarded. Against a
+ broker that was never coming back that is twenty reconnect attempts a
+ second behind a container reporting `running`, restarts 0 and `/health`
+ 200, with no line in any log and no counter anywhere. Not a silent death;
+ a permanent failure wearing the costume of a transient one.
+ Most of those errors never reached the loop at all. `kafka.NewReader`
+ falls back to a **silent logger** when `ErrorLogger` is nil, and with a
+ `GroupID` set the consumer group's dial, join and rebalance failures are
+ reported *only* through it — `ReadMessage` stays blocked — so the single
+ most likely permanent fault, an unreachable or misnamed broker, was
+ discarded inside the library. `ErrorLogger` is now wired.
+ Errors are classified into what the loop can do about them: **transient**
+ (retry, with exponential backoff capped at 30s instead of a flat 50ms),
+ **permanent** (the loop stops, because a loop over a fault no retry can
+ clear turns a misconfiguration into indefinite churn — the same reasoning
+ the UEBA consumer records), and **poison** (one undecodable envelope,
+ counted and skipped without backing off a healthy subscription). A fourth
+ state is a duration rather than a class: a `no such host` can be a startup
+ race for a few seconds and a variable nobody set for ever, so it is
+ reported as transient and then, after two minutes, as *not resolving* —
+ the point at which the operator's next move stops being "wait".
+ Permanence is decided by an enumerated set of protocol codes, not by
+ negating `kafka.Error.Temporary()`: `REBALANCE_IN_PROGRESS` is
+ non-retriable in the protocol's sense and happens on every deploy, so
+ deriving the set that way would have stopped the consumer on a routine
+ rebalance. A test walks the entire error table in both directions.
+
+- **Readiness that reported on the process and not on the subscription.**
+ Ingest's `/readyz` now names every registered background subscription and
+ its state, in the healthy case too, so a 200 says what was checked rather
+ than only that nothing was wrong. The verdict stays keyed on the publish
+ path deliberately: unlike the Python services, consuming is not ingest's
+ job, and failing readiness for an opt-in WebSocket fan-out would pull the
+ pipeline's front door out of the load balancer to fix a broadcast. The
+ surface that depends on the subscription reports on it directly —
+ `/v1/graph_ws/stream` answers 503 with the reason instead of completing a
+ handshake onto a socket that will stay silent — and
+ `aisoc_graph_ws_source_attached` / `_errors_total` / `_not_resolving` are
+ the alertable signals.
+
+- **Two scheduler loops that logged that a tick failed and never what.**
+ `retention_purge` and `hunt_scheduler` logged `err=%s` with
+ `type(exc).__name__` and nothing else: `err=ProgrammingError` every thirty
+ seconds says a tick failed, never why, and never whether waiting is the
+ right response. Both now report the sanitised message and escalate from
+ `warning` to `error` once the run of failures has outlived a transient
+ explanation.
+
+- **Nothing invoked the four alembic chains.** `honeytokens`,
+ `osquery-tls`, `purple-team` and `ueba` own their schemas through
+ alembic; every container command was a plain `uvicorn`, so the documented
+ quickstart booted them with empty schemas and none of their
+ row-level-security policies. Each image now runs `python -m app._migrate`
+ before its server: it applies the chain as the owner credential, takes a
+ shared transaction-scoped advisory lock so four chains starting at once
+ queue instead of deadlocking, reads the applied revision back, and
+ refuses to start the server unless it matches head. An exit code is not
+ evidence — `alembic upgrade head` exits 0 when it applies nothing, which
+ is exactly what happened when the chains shared one version table. All
+ four also gained a compose healthcheck; they had none, which is why a
+ container that had done nothing still read as `running`.
+
+- **Two alembic chains could never run.** `services/honeytokens/alembic/` and
+ `services/purple-team/alembic/` each contained an empty `__init__.py`, which
+ shadows the installed `alembic` package whenever the working directory is on
+ `sys.path` — so `alembic upgrade head` failed with
+ `No module named 'alembic.config'` from inside the service directory, and
+ `env.py`'s own `from alembic import context` would have failed the same way.
+ Removed, matching `services/ueba/alembic/`, which never had one. All four
+ service chains are now verified upgrade → downgrade → upgrade against a live
+ Postgres.
+
+- **`docker compose down && docker compose up` could not bring Kafka back.**
+ `kafka_data` is a named volume and ZooKeeper had none, so the second `up`
+ on kept data generated a fresh cluster id and the broker refused it with
+ `InconsistentClusterIdException`. `make down && make up` — documented as
+ "stop the stack, keep the data" — therefore left every Kafka-dependent
+ service in `created`, and only `make clean`, which deletes the data,
+ recovered. ZooKeeper now persists `/var/lib/zookeeper/data` and `/log`.
+
+- **`osquery-tls` published a container port nothing listened on.** Compose
+ mapped `127.0.0.1:8091:8007` while the service runs uvicorn on 9001. With
+ no healthcheck either, the container read as `running` against a closed
+ socket. Found by adding the healthcheck and watching it never leave
+ `starting`.
+
+- **`docker-compose.override.yml` was committed.** Its own header reads
+ "untracked, never committed": it is a QA port remap that moved PostgreSQL
+ to 55432 and ClickHouse to 58123/59000, contradicting every port the
+ documentation quotes. Removed.
+
+- **The connector catalog proxy never authenticated, and served a stale list
+ on every request because of it.** `_fetch_catalog()` issued a bare
+ `client.get(url)` with no `Authorization` header at a connectors service
+ that is default-deny on every route, so it was answered `401` on every
+ single call and fell through to the catalog bundled in the API image *every
+ time*. The bundle had **26 entries against a live registry of 84**. Nothing
+ failed: the wizard rendered a confidently wrong list, and because
+ `connector_type` is validated against that same list, the 58 connectors it
+ had never heard of were rejected as "unknown connector_type".
+
+ The proxy now presents the service credential and the tenant it is acting
+ for, mirroring the fusion gateway. Every call site passes the tenant
+ explicitly — it is a required parameter, not a defaulted one, because a
+ defaulted parameter is one call sites forget.
+
+ The fallback is kept, and given a stated job: **keep the console usable when
+ the connectors service is not deployed or not answering.** For that to be
+ defensible it has to be distinguishable from the real thing, so
+ `/connectors/catalog` now returns `source` (`live` / `bundled`), `degraded`
+ and `reason`. A deployment with no connectors service is `bundled` but not
+ degraded — there the bundle *is* the source of truth, and flagging it would
+ train operators to ignore the flag.
+
+ While degraded, an unrecognised `connector_type` returns **503 rather than
+ 422**: a connectors service rolled ahead of the API image legitimately knows
+ types that image does not, and "unknown connector_type" sends an operator to
+ debug a connector that is fine.
+
+- **The bundled catalog is generated, not refreshed by hand.**
+ `scripts/generate_connector_catalog_fallback.py` builds it from the
+ connector registry and `--check` fails the build on drift, so the artefact
+ cannot fall behind the registry it copies. The gate compares **three**
+ independent readings of connector identity, in both directions: what
+ `_CONNECTOR_CLASSES` declares (read through the AST by
+ `generate_connector_types.parse_registry`, so there is one definition of
+ "the registry" rather than two), what reaches `CONNECTOR_REGISTRY` at
+ runtime, and what each `cls.schema()` **advertises**.
+
+ The third is the reading nothing looked at. `list_connector_schemas()`
+ iterates the registry but takes each entry's `connector_id` from the schema,
+ so a class registered as `tenable_io` whose `schema()` said `tenable` would
+ be offered in the wizard under a name `get_connector_class()` cannot
+ resolve — 84 independent opportunities for two declarations to disagree.
+ Identity is read off the class, never the filename: `jira_connector.py`
+ declares `jira` and `tenable.py` declares `tenable_io`, and `--self-test`
+ asserts that property rather than trusting a comment. The self-test runs
+ before the gate in CI and injects nine defects — a dropped connector, a
+ renamed one, a class that never registers, a deleted artefact, an unhooked
+ consumer, an empty corpus — requiring each to be caught.
+
+- **An event ingested under a connector type with no profile became an alert
+ with no host, no user and no source IP.** `_canonicalAliases` in
+ `services/ingest/internal/normalizer/normalizer.go` already resolved
+ `actor.user.name`, `device.name` and `src_endpoint.ip` from the spellings
+ connectors actually use, but the pass that applied it was gated behind the
+ canonical-envelope branch. Anything reaching the generic fallback — every
+ push through `/v1/ingest/batch`, which sends a flat payload and so never
+ matches that branch — resolved `title` and `external_id` and nothing else.
+
+ Identity is what the rest of the platform is built on: entity extraction,
+ the Investigation Rail's pivots, the `{tenant}:{entity}:{tactic}`
+ correlation key, the entity graph and UEBA all key off those three fields.
+ An alert still appeared, which is what made it look like it had worked, but
+ it carried no entity chips and correlated into the `unknown` bucket. The
+ alias pass now runs for every profile and fills only destinations the field
+ map left empty, so a vendor profile's own mapping still wins.
+
+ Three defects in the same family, found while fixing it:
+
+ - **A nested vendor object was written whole into a scalar identity slot.**
+ Fourteen of the 84 registered connectors emit `actor`, and four emit it as
+ the vendor's object rather than a name, so `actor.user.name` became a map
+ — an entity that renders as a map and correlates as garbage. Identity
+ resolution now takes strings only and digs one level (`actor.name`,
+ `actor.displayName`, `user.name`) for the scalar underneath.
+ - **A vendor profile's severity ladder is spelled the way its vendor spells
+ it.** `crowdstrike_falcon`'s is capitalised, and a pushed payload is
+ whatever the caller wrote, so a lowercase `high` scored 0 and rendered as
+ Unknown. The shared five-tier ladder is consulted when the profile's own
+ has no entry for the value; `critical` stays its own tier.
+ - **A vendor profile left `message` empty for a pushed payload**, because it
+ maps its vendor's field name, so the promoter generated
+ "Security Finding from <product>" over the title the caller actually sent.
+
+- **`connector_type` values the product advertises did not reach their
+ profile.** The connectors service declares `crowdstrike` and `okta`; the
+ ingest profiles were keyed `crowdstrike_falcon` and `okta_system_log`. The
+ README's own push example used `crowdstrike`, so the example a new user
+ copies missed the lookup, fell to the generic fallback and lost its vendor
+ attribution — 80 of the 84 declared connectors had no profile entry under
+ the name they are registered with.
+
+ A `connectorTypeAliases` map resolves both spellings rather than renaming
+ either. The longer names are load-bearing elsewhere — `packages/types`'
+ `ConnectorType` union, the CLI's default, the graph extractor, the actions
+ credential resolver — so renaming one side would have broken the other.
+
+- **Two registered connectors reached no normalization path at all.**
+ `auditd` emitted `source` without `raw_event`, so it failed the canonical
+ envelope check and lost the host it carries; it now emits both.
+ `email_inbox` returns the message envelope shaped for
+ `email-forwarded.yaml`, so it matched neither branch and strict mode
+ rejected it outright; it now has a profile mirroring that template, the way
+ `ai_runtime` mirrors `ai-runtime.yaml`.
+
+- **The connector-type ratchet is at zero: ten `ConnectorType` union members
+ named nothing the normalizer could resolve.** They were the console's older
+ vocabulary, a third name space beside the ids `services/connectors` declares
+ and the keys `connectorProfiles` uses, and they were recorded rather than
+ fixed because the file was shared with console work in flight.
+
+ Six denote a source the platform really does ingest, under a longer name,
+ and now fold onto the declared id through a new `connectorTypeCanonical` map
+ applied once at the top of `Normalize` — so the profile lookup, the alias
+ map, `canonicalClassByConnector` and the product identity on the canonical
+ path all agree on one name. The connectors' own `connector_name` values are
+ what make the fold safe rather than a guess: `qradar` *is* "IBM QRadar",
+ `chronicle` *is* "Google Chronicle", `syslog_cef` *is* "Syslog / CEF".
+
+ - `ibm_qradar` → `qradar`. `services/fusion/alert_sink.py` and
+ `services/actions/executors/siem.py` already aliased exactly this pair.
+ - `google_chronicle` → `chronicle`
+ - `palo_alto_cortex` → `cortex_xdr` (whose description reads "Palo Alto
+ Cortex XDR incidents via the public REST API"; XSIAM is a distinct later
+ product and stays reachable under `cortex_xsiam`)
+ - `slack` → `slack_audit`, the only Slack data the platform ingests
+ - `syslog` → `syslog_cef`
+
+ Four denote nothing the platform ingests and are removed from the union:
+ `vectra_ai` (no Vectra connector exists), `teams` (a ChatOps destination,
+ not a source) and `custom_webhook` / `http_pull` / `kafka` (transports — the
+ webhook path is the tenant inbox, which keys off a template id and never
+ sets `connector_type`). Nothing in the tree referenced any of them;
+ `ConnectorType` has no consumer outside its own file.
+
+ What this cost while it stood: an event tagged `ibm_qradar` reached no
+ profile and no connector, so strict mode rejected it outright and lenient
+ mode minted `OcsfProduct{Name: "ibm_qradar"}` — a second, parallel alert
+ source for the same QRadar deployment `qradar` already fed. The new Go tests
+ assert the two spellings produce one event (same product, class, category
+ and severity, with `critical` staying the fifth tier and category 2 so
+ `should_promote()` has no severity floor to clear), and that the fold cannot
+ be used to smuggle an unknown type past strict mode. A third test reads the
+ connector ids out of `services/connectors/app/connectors/` rather than
+ hardcoding them, and fails on an empty read instead of passing.
-- **`docker compose down && docker compose up` could not bring Kafka back.**
- `kafka_data` is a named volume and ZooKeeper had none, so the second `up`
- on kept data generated a fresh cluster id and the broker refused it with
- `InconsistentClusterIdException`. `make down && make up` — documented as
- "stop the stack, keep the data" — therefore left every Kafka-dependent
- service in `created`, and only `make clean`, which deletes the data,
- recovered. ZooKeeper now persists `/var/lib/zookeeper/data` and `/log`.
+- **The dashboard published two numbers that contradicted the database, and a
+ third surface that contradicted both.** A live acceptance pass measured a
+ tenant with two cases closed in the last seven days at a 90-minute mean.
+ `/mssp/portfolio` reported that correctly as 1.5h. The dashboard reported
+ **CASES CLOSED (7D) 0** and **MTTR 0.0 hrs**, directly above its own
+ "CASES OPENED (7D) 2".
-- **`osquery-tls` published a container port nothing listened on.** Compose
- mapped `127.0.0.1:8091:8007` while the service runs uvicorn on 9001. With
- no healthcheck either, the container read as `running` against a closed
- socket. Found by adding the healthcheck and watching it never leave
- `starting`.
+ The portfolio was right and the dashboard was wrong, for two separate
+ reasons that each read plausibly in isolation:
-- **`docker-compose.override.yml` was committed.** Its own header reads
- "untracked, never committed": it is a QA port remap that moved PostgreSQL
- to 55432 and ClickHouse to 58123/59000, contradicting every port the
- documentation quotes. Removed.
+ - `cases_closed_7d` filtered `status = 'resolved' AND updated_at >= …`.
+ `resolved` is an *intermediate* state — the lifecycle's terminal one is
+ `closed`, and it is that transition which writes `closed_at` — so a case
+ that completed its lifecycle was invisible to the count. `updated_at` was
+ also the wrong clock: it moves whenever anyone edits the case, so an old
+ case gets a comment and re-enters the window while a closed one
+ eventually leaves it.
+ - `mttr_hours` averaged `alerts.resolved_at - alerts.created_at`, a
+ different lifecycle on a different table from the one the portfolio
+ measures. Nothing in ordinary case work writes that column, so the
+ average was over zero rows and `float(None or 0.0)` published the empty
+ result as a confident `0.0`.
-- **Two more gates were exempt from the empty-corpus rule only by accident.**
- `check_route_auth.py` and `check_tenant_query_predicates.py` did not exit 0
- over an empty tree, but neither had a corpus floor: each happened to fail
- first on its own stale-exemption ratchet, because every entry stopped
- matching at once. That is a true statement about the wrong thing, and it
- disappears the moment somebody empties the table. Both now refuse the
- corpus itself, ahead of every output mode including `--inventory`, which CI
- runs as its own step — a green step printing `TOTAL 0` is the same defect
- one level out. The route gates share one floor, in
- `check_route_tenant_scope.py`, because they share one collector; the
- predicate gate refuses each of the three ways its scan can empty
- separately, since files, tenant-scoped tables and statements reaching zero
- are three different losses and the middle one is the quiet one. Proven both
- ways: with every exemption table emptied the floor is still what refuses,
- and the real repository is unaffected.
+ Both now come from `app/services/resolution_time.py`, which owns the window
+ and the SQL expression that *both* surfaces use, so the two cannot quote
+ different MTTRs for one tenant again. The portfolio keeps computing its
+ figure inside one bound cross-tenant statement — one round trip for the
+ whole portfolio — and interpolates the shared fragments rather than calling
+ the shared function; a test asserts it still does, and the load-bearing
+ assertion is that the two surfaces produce the *same* number from the same
+ rows rather than that each matches a hardcoded 90.0.
- `check_route_auth.py` now imports its scanner as a module rather than by
- name. It took `REPO_ROOT` and `SERVICES_DIR` by value at import time, so the
- shared self-test rebinding them would have left this gate scanning the real
- checkout while believing it was pointed at an empty one — a self-test that
- proves nothing, which is the shape the whole exercise is about.
+ The third surface was the `MTTR` tile in the Security Operations Center
+ strip, reading `alerts.mttr` off `/metrics/dashboard` — the same dead column,
+ and rendered with an `m` suffix although the field is hours, so a real
+ 1.5-hour MTTR would have displayed as "1.5m" had it ever been non-zero.
-- **`check_gate_contract.py` can now ask "the directory exists and is empty",
- not only "the repository is not there".** Its scratch tree omitted
- `services/` and `detections/` entirely, so a gate opening with
- `if not X.is_dir(): return 2` refused it for a reason that says nothing
- about its corpus — and the way a corpus is actually lost is a renamed
- package or a glob that stopped matching, neither of which removes the
- directory. A second **skeleton** shape creates them empty and every check is
- probed against both. The cost was measured rather than assumed before
- committing to it: 64 of the 71 checks already refused the skeleton, and of
- the five that did not, three were defects now fixed and two were already
- recorded exceptions. The exception table is keyed by shape and has five
- entries. Probe runtime went from ~8 s to ~16 s.
+ Rather than make the three means nullable — a breaking response change for
+ every existing client — each now travels with the number of rows it was
+ averaged over (`mttd_sample_count`, `mttr_sample_count`,
+ `mttc_sample_count`, and `alerts.mttr_sample_count`). A mean over zero rows
+ is unmeasured, not zero, and the tiles say "not measured" instead of
+ claiming an unbeatable response time for a tenant that has resolved nothing.
+ The fields are additive: a client that ignores them sees what it saw before.
-- **`check_route_tenant_scope.py` reported OK having scanned zero routes.** It
- refused a tree with no `services/` directory, which is not how a scan loses
- its corpus: a renamed package, a changed decorator spelling or a walk that
- stops descending all leave the directory in place. Against a `services/`
- tree with no routes in it the gate printed `scanned 0 routes across 0 files`
- and then `OK: every route taking a tenant identifier authenticates` — the
- same sentence CI shows on a real pass. It now exits 2, and `--self-test`
- carries the case. Naming the count was only half the fix: the number was
- already printed, directly above the clean verdict.
+- **The alerts list was empty on every deployment, under a row count that was
+ real.** `AlertListResponse` returns the rows under `items`; the web client
+ read `raw.alerts`, which is never present, so `Array.isArray(undefined)` was
+ false and each page resolved to `[]` while `total` carried the true figure.
+ The queue therefore rendered "1,247 alerts" above an empty table with no
+ error to explain it, and an operator's most reasonable reading of that screen
+ was that their estate was quiet. The client now reads `items`, still accepts
+ the legacy `alerts` key the responder routes emit, and a test asserts the two
+ halves of the contract against each other so they cannot drift apart again.
-- **The dependency audit reported success over a tree with no manifests.**
- `security_audit.py`'s pnpm, python and go arms each discovered zero targets
- and printed `0 findings`, which is the sentence a clean audit of twenty
- services prints. The file already treated an unscanned service as a failure
- rather than a warning — the gap was that a corpus of zero was never
- *unscanned*, just empty. All three now record a coverage gap when discovery
- finds nothing, and `validate-ignores` refuses a missing policy file instead
- of reporting `Validated 0 ignore entries` and exiting 0. `get_repo_root()`
- also stopped falling back to `Path.cwd()`, which meant a run from anywhere
- else audited whatever manifests happened to be under it.
+- **The entity-risk queue sent a tenant slug where a UUID was required, then
+ blamed a healthy service for the rejection.** `apps/web/next.config.js`
+ inlined `NEXT_PUBLIC_TENANT_ID` with a fallback of the literal string
+ `'default'`. Three tenant-scoped surfaces pass that value as a query
+ parameter to routes typed `tenant_id: UUID` — `/fusion/entity-risk/*`,
+ `/honeytokens/*` and `/business-context/*` — and all three answered 422.
-- **Four more known-red CI entries, and the reason each could sit on `main`.**
- None was anyone's recent breakage; all four reproduced on a pristine
- checkout. What they had in common is more useful than what broke: in every
- case the thing that should have objected either did not run, was not
- required, or could not fail.
+ Two things kept it hidden. `lib/api.ts` carries the correct canonical UUID
+ as its own fallback, so reading it suggested the console was already doing
+ the right thing; Next's inlining runs first, which made that fallback
+ unreachable code. And the slug is not a dependable handle either: migration
+ 001 seeds tenant `…0001` with slug `default` and the demo seed renames that
+ slug to `demo`, so on a seeded install the literal matched neither the id
+ nor the slug. The entity-risk client also read the build-time constant
+ rather than `getActiveTenantId()`, so the queue ignored both the logged-in
+ user and the tenant switcher; it now resolves the tenant the same way
+ `request()` resolves the `X-Tenant-Id` header.
- - **`Backup → destroy → restore` had lost its object store for the second
- time.** `minio/minio` was withdrawn from Docker Hub (the repository 404s),
- the job was moved to the quay.io mirror pinned by digest, and quay.io now
- answers **401 UNAUTHORIZED for every tag and for that digest** — the whole
- repository, not one tag. A third MinIO coordinate would be the same bet a
- third time, so the fixture is now Versity's Apache-2.0 S3 gateway
- (`ghcr.io/versity/versitygw`, pinned to the v1.8.0 multi-arch index
- digest). It is a real S3 server rather than a mock, so `backup.sh` and
- `restore.sh` still drive `aws s3 cp/ls/rm` and `s3api head-object` for
- real; and it is on GHCR, which this workflow's own images already use and
- which cannot be withdrawn independently of the CI it runs on. The job also
- gained a readiness probe: `docker run -d` succeeding only means the
- container was created, and the previous shape would have spent the bucket
- loop's retries before failing with a connection error that said nothing
- about why.
- - **`docker compose up — full stack` failed building `services/realtime`,
- and not for the reason it looked like.** The lock and the manifest do
- agree (`npm install --package-lock-only` is a no-op against the committed
- lock), the Dockerfile does copy `package-lock.json` into both stages, and
- Node 22 is consistent across the Dockerfile, CI and `@types/node`. The
- failure was `esbuild`: a transitive of `tsx` that the image never runs,
- whose install script hardlinks the platform binary into place and then
- immediately execs it to read its version, with no retry — under BuildKit
- that exec loses a race with the writer still holding the inode and fails
- `ETXTBSY`. The builder stage now installs with `--ignore-scripts`, which
- it can do because the only thing it needs from devDependencies is `tsc`.
- The emitted `dist/` is byte-identical. The runtime stage deliberately
- keeps its install scripts: what it installs is what ships.
- - **`test_business_context_hotpath.py` reached a real database.**
- `FusedAlertTriageWorker.triage()` opens asyncpg connections through
- `business_context._load_tenant_rules` and `ledger.persist_auto_triage`, so
- with an inherited `DATABASE_URL` two tests failed with `InterfaceError:
- cannot perform operation: another operation is in progress` — pytest-asyncio
- gives each test its own event loop, and the module-level pool one test
- opens is unusable by the next. The agents unit suite now declares that it
- runs with no database, in `tests/conftest.py`, and **enforces it**: the
- inherited DSN is removed and a real asyncpg connection raises. The guard
- is a `BaseException` on purpose — every call site it covers is wrapped in
- a fail-soft `except Exception`, which is correct in production and is
- exactly why the bug was invisible, so an `Exception` here would be
- swallowed by the handlers it exists to police. Two tests that drive the
- real client against a DSN they set themselves are marked
- `touches_database`. Replaces two earlier workarounds that did not work:
- `os.environ.setdefault("DATABASE_URL", "")` is a no-op precisely when the
- variable is set.
- - **`test_a_missing_handler_is_not_retried` asserted nothing that could
- fail.** It named `RUN_AV_SCAN` as its missing handler; that stopped being
- true when the response verbs were wired into `_HANDLERS` via
- `RESPONSE_STEP_TYPES`, so the step took the *retry* branch and was
- attempted four times over fourteen seconds — the behaviour the test's own
- docstring forbids — while reporting green. Its one behavioural assertion,
- `_elapsed_ms == 0`, could not notice: `t0` resets at the top of every
- attempt, so the field is the duration of the final attempt alone, and it
- reads 0 both when the engine skips the retry loop (which writes a literal
- `0`) and when a handler fails in under half a millisecond. The test now
- takes its step type from the registry instead of naming one, so it cannot
- silently rot the same way again, and asserts the property directly by
- recording backoff sleeps. The file went from 14.13s to 0.16s. The
- `patch_handlers` fixture now saves originals with `setdefault`, so a
- second install of one key cannot write a test double permanently into the
- process-wide handler registry.
+ `components/fim/FimDashboard.tsx` is deliberately pinned to `'default'`
+ instead: `services/osquery-tls` types its tenant as a plain string and
+ enrols nodes under that literal, so this one surface is keyed on that
+ service's own convention and reading the shared UUID here would match no
+ enrolled node. The two tenancy models still need reconciling in
+ osquery-tls.
- Why `main` carried them: `backup-restore` and `docker compose up — full
- stack` fail hard but are **not** among the nine required checks, and compose
- smoke is pull-by-default — it only builds a service when that service's
- build context changed, so it is usually green without building the thing
- that was broken. `test_business_context_hotpath.py` runs inside `Python —
- Tests`, which **is** required, and passed only because GitHub's runners have
- nothing on 5432 and the agents step sets no `DATABASE_URL` — green by
- property of the runner, not of the code. `test_playbook_engine_correctness.py`
- was named by no workflow at all; it is one of **62 of the 82** test files
- under `services/agents/tests` in that position. The playbook and per-tenant
- business-context files are added to the agents test list (gated agents tests
- 529, up from 303); the remaining gap is real and is not closed here.
+ With the request failing, the queue rendered five invented entities —
+ `[email protected]`, `updates.evil-cdn.xyz`, "last seen 5 months ago" on a
+ stack that had been up for an hour. Above them, and *outside* the amber
+ banner that disclosed them, four cards read "Contributing alerts 26" and
+ "Alert → Incident 13.0:1" off the same sample payload. A disclosure that
+ covers the rows and not the headline numbers is decorative, so the banner
+ moved above the cards and every card now reads "not measured" when its
+ request fails; the derived alert-to-incident ratio is withheld whenever the
+ counts it divides are unknown.
-- **Four tests failed on a clean checkout and belonged to nobody.** A suite
- with known-failing tests teaches everyone to skim past red, so each is now
- either fixed or skipped with a reason that says what to do about it.
+ The banner also named the wrong subsystem. It said "Fusion service
+ unreachable" while fusion was healthy and the actual failure was a 422 —
+ a diagnosis that sends an operator to debug something that is not broken,
+ which is worse than none. It is now derived from the response status: a 422
+ reads as a console bug rather than an outage, a 5xx names fusion because
+ that is when fusion is genuinely at fault, status 0 reads as unreachable,
+ and an unrecognised failure stays vague rather than guessing. Every variant
+ says the queue is *unknown* rather than empty.
- - `test_hunt_scheduler_cron.py::TestNextFireAt::test_basic_hourly` asserted
- croniter's answer (the top of the next hour) from a class that, unlike its
- neighbour, carried no `_CRONITER_AVAILABLE` guard — so without the
- dependency it failed rather than skipped. Not environmental: the fallback
- is an interval stepper with defined behaviour that nothing was checking.
- It now asserts the right answer on each path, so both are tested.
- - `test_router_report.py::test_findings_with_html_tags_are_escaped` required
- the literal `&lt;script&gt;`, which only the `markdown` branch produces —
- the `<pre>` fallback escapes the already-escaped body again and emits
- `&amp;lt;script&amp;gt;`. That is more escaped, not less, so a correct
- branch was reporting a security failure. It now asserts the property that
- matters (nothing reaches the document as an executable tag, and the
- payload is still present rather than dropped) on either branch.
- - `test_router_report.py::test_html_wraps_markdown_with_document_chrome`
- genuinely needs `markdown` — a declared dependency of the service — and
- now skips with that stated, while CI installs it and runs the file, which
- it had never done.
- - `test_graph_freshness.py::test_graph_writer_does_not_block_fusion_on_failure`
- treated "something answered on port 8080" as "the ingest service
- answered". Its only escape hatch was a connection error, so an unrelated
- local service returning 401 became a failure reading "fusion should never
- block on graph". It now checks `GET /health` reports `service: ingest`
- before asserting anything, and the skip names what did answer.
+- **`/alerts/[id]` rendered a confidence of 21 as `2100%`, and negative
+ evidence as `+-0.30`.** Confidence reaches the console on two keys at two
+ scales: the API surfaces `confidence` as an integer 0-100, while fusion's
+ `confidence_score` is the raw [0.0, 1.0] float the band was derived from.
+ `normalizeAlert` accepted whichever key appeared first and passed it
+ through unchanged, so `Alert.confidenceScore` meant one thing or the other
+ depending on the payload — and its consumers guessed differently.
+ `AlertDetailView` multiplied by 100; `AttackStory`, on the same page,
+ divided and rendered "21/100". Each was right for one payload shape, and
+ each had a passing test because its own mock used the scale it assumed.
-- **Two alembic chains could never run.** `services/honeytokens/alembic/` and
- `services/purple-team/alembic/` each contained an empty `__init__.py`, which
- shadows the installed `alembic` package whenever the working directory is on
- `sys.path` — so `alembic upgrade head` failed with
- `No module named 'alembic.config'` from inside the service directory, and
- `env.py`'s own `from alembic import context` would have failed the same way.
- Removed, matching `services/ueba/alembic/`, which never had one. All four
- service chains are now verified upgrade → downgrade → upgrade against a live
- Postgres.
+ Normalised once at the boundary to the canonical 0-100 integer, deciding the
+ scale from the *key* rather than the magnitude: a genuine confidence of 1 is
+ indistinguishable from a raw score of 1.0 by value alone, so the tempting
+ `v <= 1 ? v * 100 : v` would render the least-confident alert in the estate
+ as the most confident. The two mocks that encoded the old scale were
+ corrected with it, since sample data on a different scale than the real
+ payload is what hid the bug.
-- **Every live Okta password reset raised `TypeError` and came back FAILED.**
- `ResetPasswordExecutor` documents `parameters.send_email` as "used only for
- the Okta path", reads it, and passed `send_email=` to
- `OktaClient.reset_password(self, login_or_id)` — which did not accept it. The
- executor's `except Exception` turned the `TypeError` into a FAILED
- `ActionResult`, so the verb did not crash; it simply could never succeed
- against Okta, and the failure read as an Okta problem. Simulation mode never
- constructs a vendor client, which is why no test saw it: the one that covers
- this executor stubs the client as `async def reset_password(self, *_args,
- **_kw)`, and a fake that accepts anything proves the dispatch order and
- nothing about the call. The client now takes `send_email` and maps it to
- Okta's `sendEmail` parameter, and `test_identity_client_signatures.py`
- autospecs all three IdP clients across every executor arm — the companion to
- `test_siem_client_signatures.py`, for the third instance of the same defect.
+ The rationale rows hardcoded a `+` prefix on a signed contribution, so a
+ factor that argued *against* the verdict read `+-0.30`; they now carry one
+ sign, matching the glyphs the narrative builder uses. Those rows also
+ clamped `contribution / weight` into [0, 1], which rendered every negative
+ factor at zero width — an invisible bar beside a nonsense label. Width now
+ follows the magnitude and colour follows the sign.
-- **The natural-language playbook drafter could never reach an LLM.**
- `nl_drafter._llm_factory` called `make_chat_model()`, and `role` is a required
- positional parameter; every other caller in the tree passes one. The
- `TypeError` was caught by the `except Exception` two lines down, which logged
- "no chat model available" and returned `None`, so the drafter fell back to its
- deterministic substrate path on every call, permanently, while reporting the
- condition as a missing provider. Every test monkeypatches `_llm_factory`, so
- the one line that mattered was the one line nothing exercised. Now passes
- `"nl"`, the declared role in `app.llm.model_pins`.
+- **Every entity chip in the Investigation Rail was a 404.**
+ `alert_rail.py` built its pivots as `/attack-graph?entity=…` and there has
+ never been an `attack-graph` route, so host, user, asset, IP and domain chips
+ all failed before the query parameter mattered. The rail's own docstring
+ described the working behaviour ("the same `?entity=` query the
+ AttackGraphView already parses"), and `pivot.ts` states as fact that the rail
+ emits `/graph?entity=…`; the code agreed with neither. Pivots now target
+ `/graph`, which reads the parameter and selects the node.
-- **A cancelled context-graph walk crashed the context bundle.**
- `_fetch_neighborhoods` and `_fetch_ueba_baselines` filter
- `asyncio.gather(..., return_exceptions=True)` results with
- `isinstance(r, Exception)`, then unpack the survivors as a tuple.
- `asyncio.CancelledError` is a `BaseException` and not an `Exception` on 3.8+,
- so a cancelled child task passed the filter and reached the unpack as
- `TypeError: cannot unpack non-sequence CancelledError`. `services/slack-bot`
- carries a comment spelling out this exact trap; these two sites got it wrong.
- Both now filter on `BaseException`.
+ The existing test pinned all six strings under a docstring claiming the pin
+ "prevents an accidental rename". It cannot — it compares the producer against
+ a copy of itself, which is how a route that never existed stayed asserted.
+ `services/api/tests/test_pivot_routes_resolve.py` derives the route table
+ from `apps/web/src/app` and checks it against what `build_related_entities`
+ actually returns, so the comparison now runs in the direction that drifts.
-- **`_SECRET_PATTERNS` was declared with a type its value cannot have, which
- switched off checking of the secret-masking loop.** The annotation in the AI
- SDK's redaction module read
- `tuple[tuple[str, re.Pattern[str]], None | str] | tuple`, whose first member
- is a two-element tuple — structurally impossible for the eight-pair value, so
- it only ever matched through the bare `| tuple`. That erases the element type
- to `Any`, which is why nothing objected to calling `.subn` on something typed
- as possibly a `str`. In a path whose job is to stop secrets leaving the
- process, a checker that has been silently switched off is worse than none.
- Declared `tuple[tuple[str, re.Pattern[str]], ...]`, which is what it is.
+ The same test caught a second defect in the producer: values went into the
+ URL unencoded, so an asset named `Finance & Legal #2` pivoted to
+ `Finance & Legal` and looked like it had worked. Values are now encoded the
+ way `pivot.ts` encodes them.
-- **`diff_fingerprints` was declared to return `dict[str, list[str]]` and never
- has.** It returns `unchanged_count` as an `int`, which the connector
- scheduler stores and a test asserts on. Because the scheduler assigns the
- result straight into a `dict[str, Any]` variable, the wrong declaration
- narrowed that variable and made the three correct lines underneath it look
- like type errors instead — one wrong annotation producing four findings across
- two modules. Declared `dict[str, Any]`, which is what both consumers and the
- JSONB column already expect.
+- **The Investigation Rail's Details tab showed analysts the markup.**
+ `build_narrative` documents its output as markdown-light — `**bold**`,
+ backtick code spans, `- ` bullets, blank-line paragraphs — and the rail put
+ the string in a `whitespace-pre-wrap` paragraph, which preserves the
+ newlines and the asterisks alike. The panel read `**Medium** alert: … on
+ **Finance & Legal #2**`.
-- **A Google Workspace key that was valid JSON but not an object failed inside
- the JWT signing path.** `json.loads` returns a `str`, `list` or `int` for a
- document that is valid JSON and not an object, and the constructor accepted
- it; the failure surfaced later as `TypeError: string indices must be integers`
- at `self._key["client_email"]`, at the moment an operator triggered a live
- action. The configuration is wrong either way — it now says so when the
- credential is saved rather than when it is used.
+ `components/alerts/NarrativeMarkdown.tsx` renders exactly that dialect and
+ nothing more; an unrecognised construct falls through as literal text rather
+ than being dropped. It builds React elements, never markup: the narrative
+ embeds the alert title and entity names, which originate in connector
+ payloads, so an HTML path here would make anyone who can name a host an XSS
+ author. It introduces no heading, leaving the page's heading order intact.
-- **Five LLM input-contract handlers raised `TypeError` instead of degrading.**
- `/translation`, `/phishing`, `/knowledge-base`, `/hunts` and `/detection-loop`
- each catch `LLMContractViolation` — the untrusted-input boundary refusing a
- prompt — and logged it as `logger.warning("<event>", reason=exc.reason)`.
- `logger` is `logging.getLogger`, not structlog, and the stdlib `Logger`
- rejects an unknown keyword with a `TypeError`. An exception raised inside an
- `except` block is not caught by a sibling handler, so the `except Exception`
- sitting directly beneath it never saw it: the one path whose job is to
- degrade gracefully was the only path that raised. `phishing.py` carries a
- comment explaining that the log line was added *because* the handler used to
- swallow everything — the fix for the silent failure was itself throwing. Only
- six of thirteen trees declare `[tool.mypy]`, so an AST scan swept the rest of
- `services/` and `packages/`; those five were the only instances repo-wide.
+- **Six funnel metrics were published as zero whatever the database held.**
+ `_funnel_window` merges `_triage_quality`'s output into the dict it returns,
+ but the `FunnelMetrics(...)` call never named `triaged_alerts`,
+ `abstentions`, `abstention_rate`, `ungrounded_demotions`,
+ `mean_groundedness` or `scored_verdicts`, so all six fell back to their
+ field defaults on every response. A published zero is a stronger claim than
+ silence: it reads as "this tenant never abstained, and nothing was ever
+ demoted for being ungrounded". The existing test asserted the six names were
+ present in `model_fields`, which they were — declaring a field and forwarding
+ it are different things. The replacement drives the endpoint and fails for
+ any field the window computes and the response drops, including ones added
+ later.
-- **`DEFAULT_SLA_TARGETS` was declared twice with disagreeing bodies, and
- neither matched the migration.** The second definition won by being later in
- the file and put `info` at `(480, 1440, 2880)`; the first said
- `(240, 960, 2880)`; migration 040 seeds `(240, 1440, 4320)`. A tenant's
- info-tier deadline therefore depended on whether it had a seeded row
- (240 min) or fell back to Python (480 min) — and `alert_queue` reads exactly
- this row as the `sla_due_at` catch-all for severities outside the four-tier
- ladder. One definition now, matching the migration, which is what is in the
- database.
+- **`false_positive_rate` and `escalation_rate` published `0.0` on an empty
+ denominator.** `x / n if n > 0 else 0.0` renders an undefined ratio as a
+ measured zero, and for these two the zero is flattering: "0% false
+ positives" and "0% escalated" are the two best numbers on the SOC metrics
+ page, and a tenant that had resolved nothing and gated nothing scored both.
+ This is the same defect as the MTTD/MTTR/MTTC means that shipped a NULL
+ average as a confident `0.0`, one step removed, and it takes the same fix:
+ `/metrics/soc` now reports `false_positive_rate_sample_count` and
+ `escalation_rate_sample_count` alongside the rates, and the console renders
+ "not measured · no resolved alerts in 7d" rather than a percentage. The
+ fields are additive and optional, so a console running against an older API
+ keeps its previous behaviour rather than blanking the tiles — a paired test
+ holds that direction too, because "blank everything" would be a worse
+ regression than the bug.
-- **`posture_loader` returned `None` from a function declared to return a
- dict** when a 200 response carried no `config` key, while the non-200 branch
- immediately above already returned `{}` for the same "nothing to load"
- outcome.
+- **The connector fleet badge claimed every source was reporting when none
+ was.** `ConnectorFleetPanel` rendered the badge whenever the endpoint
+ answered, and with zero connectors `failed + degraded` is zero — so a green
+ "All sources reporting" sat directly above the panel's own "No connectors
+ configured". Zero sources reporting is not the same statement as all of
+ them reporting, and the green is the part an operator scans for. There is
+ now no badge until there is a fleet, and when there is one it names the
+ count it is vouching for.
-- **Six guards inspected a different call's result from the one they
- guarded.** `x.get(k) if isinstance(x.get(k), dict) else {}` calls `get`
- twice; it is safe for a plain dict and that is not a property anything
- enforces, which is what the twenty `union-attr` findings underneath it were
- reporting. Fetched once and then tested, in `playbook_step_dispatch`,
- `siem_writeback` and `sla`.
+- **Two dashboards published fabricated security data as tenant state.**
+ `DashboardView` and `SOCMetricsDashboard` both wrapped their SWR
+ `fallbackData` in `demoFallback()`, which is `undefined` outside the hosted
+ demo — and both then defeated that gate a few lines later with an
+ unconditional `const resolved = isValid ? data : MOCK`. The mock was
+ therefore exactly what rendered during first paint and after any API error,
+ which for a self-hoster with an empty or unreachable backend is the whole
+ session.
- Together with a `_compute_durations` return annotation that claimed
- `dict[str, int | None]` while returning a `str` severity, these take the mypy
- ratchet from 690 to 656. `packages/sdk-py` declared `[tool.mypy]` with no
- `python_version`, so it was type-checked against whichever interpreter the
- job ran and its share of the baseline moved with CI rather than with the
- code; it is pinned to 3.11 like the other five, and the toolchain gate now
- fails on a tree that asks to be type-checked without saying against which
- Python. What remains on the ratchet is annotation hygiene and artefacts of
- the deliberate no-dependencies environment the baseline is recorded in — the
- 19 surviving `union-attr` are all `mock.call_args` in tests.
+ What that put on screen as the reader's own numbers: a connector inventory
+ they do not run (`CrowdStrike EDR`, 412 events), a MITRE tactic ranking, a
+ 24-hour alert-volume curve, MTTD 1.4h / MTTR 6.2h, and an LLM spend line of
+ $76.65 naming three models they had never configured. Three "vs yesterday"
+ trend deltas sat beside the real alert counts as literals — `/metrics/dashboard`
+ publishes no period-over-period comparison, so there was nothing to derive
+ them from.
-- **The connector-type ratchet is at zero: ten `ConnectorType` union members
- named nothing the normalizer could resolve.** They were the console's older
- vocabulary, a third name space beside the ids `services/connectors` declares
- and the keys `connectorProfiles` uses, and they were recorded rather than
- fixed because the file was shared with console work in flight.
+ Every panel now renders one of three honest states: real figures, an empty
+ state naming what would populate it, or an error state carrying the failure
+ and a retry that re-issues the request. Sample data still populates the
+ hosted demo, which is the only reason it exists.
- Six denote a source the platform really does ingest, under a longer name,
- and now fold onto the declared id through a new `connectorTypeCanonical` map
- applied once at the top of `Normalize` — so the profile lookup, the alias
- map, `canonicalClassByConnector` and the product identity on the canonical
- path all agree on one name. The connectors' own `connector_name` values are
- what make the fold safe rather than a guess: `qradar` *is* "IBM QRadar",
- `chronicle` *is* "Google Chronicle", `syslog_cef` *is* "Syslog / CEF".
+- **The MSSP console showed six invented tenants and made no API call at all.**
+ `MSSPDashboardView.tsx` declared `const TENANTS = [...]` — "Acme Financial",
+ "GlobalRetail Corp", "MedSecure Health" and three more, with invented alert
+ counts, MTTD/MTTR figures, risk scores, analyst headcounts and ARR — handed
+ it to `useState`, and never fetched anything. Every operator on every
+ deployment saw the same six rows, permanently, with no state in which they
+ would not. The "Export Report" button raised a success toast and did nothing.
- - `ibm_qradar` → `qradar`. `services/fusion/alert_sink.py` and
- `services/actions/executors/siem.py` already aliased exactly this pair.
- - `google_chronicle` → `chronicle`
- - `palo_alto_cortex` → `cortex_xdr` (whose description reads "Palo Alto
- Cortex XDR incidents via the public REST API"; XSIAM is a distinct later
- product and stays reachable under `cortex_xsiam`)
- - `slack` → `slack_audit`, the only Slack data the platform ingests
- - `syslog` → `syslog_cef`
+ It now reads `GET /api/v1/mssp/portfolio` and
+ `GET /api/v1/mssp/portfolio/alerts` with no sample-data fallback and no SWR
+ `fallbackData` (supplying it disables revalidation, so a placeholder becomes
+ what the view permanently shows). Four states, each saying which it is:
+ loading; a `403` explained as "you do not manage any tenants" rather than an
+ outage; a portfolio failure surfaced with its message and a retry; and an
+ empty portfolio that distinguishes "this organisation manages no tenants"
+ from "you were granted none" using `portfolio_wide` and `scoped_tenants`,
+ because those have different fixes. The alert feed has its own states so a
+ failure there does not claim the portfolio is down. Export now writes a CSV
+ of the rows on screen — a real action rather than a toast.
- Four denote nothing the platform ingests and are removed from the union:
- `vectra_ai` (no Vectra connector exists), `teams` (a ChatOps destination,
- not a source) and `custom_webhook` / `http_pull` / `kafka` (transports — the
- webhook path is the tenant inbox, which keys off a template id and never
- sets `connector_type`). Nothing in the tree referenced any of them;
- `ConnectorType` has no consumer outside its own file.
+ **ARR, risk score and analyst allocation are gone, not sourced.** There is no
+ revenue, composite-risk or analyst-allocation data anywhere in this product.
+ A column of nulls would still imply the measurement exists.
- What this cost while it stood: an event tagged `ibm_qradar` reached no
- profile and no connector, so strict mode rejected it outright and lenient
- mode minted `OcsfProduct{Name: "ibm_qradar"}` — a second, parallel alert
- source for the same QRadar deployment `qradar` already fed. The new Go tests
- assert the two spellings produce one event (same product, class, category
- and severity, with `critical` staying the fifth tier and category 2 so
- `should_promote()` has no severity floor to clear), and that the fold cannot
- be used to smuggle an unknown type past strict mode. A third test reads the
- connector ids out of `services/connectors/app/connectors/` rather than
- hardcoding them, and fails on an empty read instead of passing.
+- **`/analytics/team` ranked six invented analysts by invented accuracy.**
+ "Sarah Chen, 47 cases closed, 96.2% accuracy, score 945" and five more, plus
+ a highlights feed of things that never happened. Nothing in the platform
+ measures per-analyst performance — no route, no table, no column — so the
+ sample is confined to the hosted demo via `canUseDemoData()` and everyone
+ else is told plainly that the measurement does not exist yet. The aggregate
+ tiles read `—` rather than dividing by zero.
-- **`wet-eval.yml` reported success on eight consecutive weekly runs while
- evaluating nothing.** The preflight step exited 0, every subsequent step was
- `if: should_run == 'True'` and skipped, the notice step succeeded, and the
- job went green. A green check that means "I did not run" is worse than a red
- one: it is the only signal a reader gets and it says the opposite of the
- truth. Preflight is now its own job and the run is gated on
- `needs.preflight.outputs.should_run`, so with no funded key the benchmark
- job reports as **skipped** — visibly distinct from success in the checks
- list — and the preflight writes to the run summary that the published
- numbers were not refreshed. Failing outright was the alternative and is
- wrong here: a fork cannot configure the secret, and a weekly red cross
- nobody can clear trains people to ignore the page. Staleness of the
- published numbers is separately gated and does fail closed
- (`scripts/check_scoreboard.py`, 45 days).
+- **`apps/web/src/components/landing/MitreStrip.tsx` is deleted.** Its twelve
+ ATT&CK tactic tiles carried unsourced coverage counts — 27 of 42 for Defense
+ Evasion, 9 of 11 for Initial Access — that correspond to nothing in the
+ tree. The component was imported nowhere and rendered on no route: the
+ landing page composes fifteen sections and this was not one of them, so the
+ caveat in its body copy was the only thing between those numbers and a
+ reader, and it would have stopped being so the moment somebody mounted the
+ section. Improving the disclaimer would have left the numbers in place for
+ the next person to inherit. Its entry in `ALLOWED_ILLUSTRATIVE` goes with
+ it; `scripts/check_mock_data_gated.py` checks that allow-list in both
+ directions, so a stale exemption fails the build rather than accumulating as
+ cover. The one MITRE figure the project does publish — 97.0% in
+ `BenchmarkBand` — is labelled "substrate" and is unchanged.
-- **The claim-gate figures gate checked every restatement of the tally except
- the document making the claim.** `readme_gates.py` compared the README and
- `evidence-pack.md` against the matrix rows and never compared the matrix's
- own Summary block, so `docs/audit/CLAIM_TO_GATE_MATRIX.md` summarised
- "GATED: 108" against 109 counted rows and every check in the repository
- passed. The file even carries a counting note about this exact class of
- error; it recurred because the gate written afterwards pointed outward only.
- The matrix is now the first source checked, the bullet-list form its summary
- uses is matched (the previous pattern needed both figures on one line and
- silently matched nothing there), and three cases in
- `tests/test_readme_figures_gate.py` pin it. The stale count is corrected and
- the tally is **122 rows — 113 GATED, 9 PARTIAL, 0 NO GATE**, recomputed with
- the script rather than typed.
+- **The welcome banner quoted counts it had no source for and linked to a case
+ most deployments do not have.** It advertised "26 vendors" against a registry
+ of 84 and "25 named runbooks" with nothing holding either to the tree. The
+ connector figure now comes from `CONNECTOR_COUNT`, which is generated from
+ the connector registry and held to it by `scripts/generate_connector_count.py
+ --check`; the playbook figure is gone rather than guessed, because no
+ equivalent source exists. Its second call to action linked to
+ `/cases/INC-RT-001`, which exists only after the demo seed has run, so on
+ every other deployment the banner's own CTA was a dead link — that tip is now
+ gated on demo mode, takes its href from `demoDeeplink()` and says on its face
+ that it is sample data.
-- **`screencast.yml` could never get past its third step.** Its
- `cache-dependency-path` named `apps/web/pnpm-lock.yaml`, which does not
- exist — this is a pnpm workspace with one lockfile at the root — and
- `setup-node` hard-fails when the cache path matches nothing. Every other
- workflow in the repo already pointed at the root lockfile; being
- `workflow_dispatch`-only meant no scheduled run ever exercised it.
+- **The Live Feed labelled an empty panel "Demo".** The seeded events were
+ gated behind `canUseDemoData()` in an earlier pass but `statusToLabel` was
+ not, so outside the hosted demo the panel rendered nothing at all under a
+ "Demo" pill whose tooltip read "showing demo data" — asserting the presence
+ of sample data that had just been correctly withheld. The pill now describes
+ what is on screen (`Live`, `Connected`, `Connecting…`, `Reconnecting…`,
+ `Offline`) and can only say `Demo` when seeded events are actually rendered,
+ and the idle panel carries an empty state naming what would fill it.
-- **The same commit did not build the same way twice, and one of the ways it
- could build did not start.** `One real event through the real pipeline`
- failed and then passed on re-run with no code change. The API container had
- died at import with `AssertionError: Status code 204 must not have a
- response body` from `app/api/v1/endpoints/community.py:183` — a file
- byte-identical to `main`, on a branch that touched nothing under
- `services/api`.
+- **The fabricated-data gate could not see either of the two worst cases.**
+ `scripts/check_mock_data_gated.py` recognised only mock data that announces
+ itself: all three of its patterns required a `MOCK_` / `DEMO_` name *and* an
+ assignment through a state setter or SWR `fallbackData`, which models one
+ situation — a view that fetches and substitutes a sample when the fetch
+ fails. A dataset written inline under an ordinary name, in a component with
+ no fetch at all, matched nothing; and with no fetch there was no real path
+ for a fallback to fall back *from*, which is the worse defect, not the
+ lesser one. The gate reported "All sample-data fallbacks are gated behind
+ demo mode" while both fabricated tables shipped.
- Measured, from the failed run's own log: `poetry install` hit a transient
- error, the Dockerfile's pip fallback took over, and it pinned
- `fastapi>=0.111,<0.112` and installed 0.111.1. In the same build `fusion`'s
- poetry install succeeded and took 0.141.1. One commit, two resolvers, two
- answers.
+ A third check looks for what makes fabricated domain data harmful rather
+ than for what an author happened to call it: a module-scope array of records
+ that names an entity a customer would recognise — a company, a person, a
+ host, an IP, an address — *and* attaches numbers to it. Numeric keys that
+ describe how something is drawn are excluded, which is what keeps it quiet:
+ 93 module-scope object arrays in the console, 7 matched, and the 4 already
+ behind `demoFallback()` were the mocks. It newly caught
+ `MSSPDashboardView.tsx` and `TeamAnalyticsView.tsx`.
- `community.py` has `from __future__ import annotations`, so its `-> None`
- return annotation reaches FastAPI as the string `"None"`. FastAPI resolves
- it through `ForwardRef` to `NoneType` — truthy — rather than the falsy
- `None` singleton it gets without PEP 563, concludes the route returns a
- body, and asserts that a 204 does not. **Every release from 0.111.0 through
- 0.116.2 raises; 0.117.0 and later do not.** The declared range was
- `>=0.111,<0.142`, so 29 of the 120 releases it permitted could not import
- the service at all, and the fallback pinned squarely inside that band. The
- earlier triage read the shape as innocent because it was checked without
- PEP 563 active, which is the one condition that makes it fail.
+ The reviewed-exception list is checked in **both** directions: an entry that
+ no longer matches anything fails the gate, so an exemption cannot outlive the
+ code it excused and become cover for whatever is written next under that
+ name. One entry, `MitreStrip.tsx` — a public landing-page illustration whose
+ visible copy already tells the reader the tiles are illustrative.
+ `tests/test_mock_data_gate.py` covers both properties, including that filter
+ lists, graph stylesheets, decorative SVG coordinates and pricing copy stay
+ unflagged; a gate that cries wolf on every configuration array gets deleted,
+ which is worse than the gap.
- The fix is determinism rather than a widened assertion:
+- **`SavedViewsBar` updated its parent while rendering.** The auto-apply of a
+ default saved view ran in the render body and called `onApply`, which for
+ every caller is a setState on the page component. React rejects that outright
+ and makes no promise about processing the update, so the filters an analyst
+ expected restored were not reliably applied. The comment described a ref-flag
+ that did not exist; it was `useState` with no effect anywhere. The chip
+ highlight is now derived rather than stored, and the one genuine side effect
+ — handing the filters to the page — happens in an effect. The existing
+ "exactly once" test could not catch this because its `onApply` was a bare
+ spy that set no state; the new test uses a real parent.
- - **All thirteen Python services install from a committed `poetry.lock`.**
- Three had one; ten now do. Two `--no-cache` builds of one commit produce
- an identical `pip freeze` (129 packages, same SHA-256), and a new
- `Reproducible builds / twice` job asserts it on every change.
- - **The pip fallbacks are gone** from all eight Dockerfiles that had one.
- Each carried its own copy of the dependency list under a comment asking
- for lockstep, and each had drifted. A build that fails is recoverable; an
- image that boots on versions nothing tested is not.
- - **Four services that hand-listed their dependencies in the Dockerfile**
- (`honeytokens`, `purple-team`, `ueba`, `mesh`) now install from a lock
- too. All four had drifted from the manifest they mirrored — `mesh` shipped
- `cryptography<50` against a manifest reading `<51`.
- - **The FastAPI floor is 0.117** in all thirteen manifests and every
- workflow. `services/api/tests/test_fastapi_floor.py` pins the boundary and
- fails on 0.116.2 with the original error; a matrix leg imports the service
- at both ends of the declared range.
+- **A self-hosted install handed its own users links into the maintainers'
+ deployment.** Beyond the CORS and demo-credential entries above, ten shipped
+ defaults resolved to the hosted host when left unconfigured, so an operator
+ who never set an override sent their users somewhere else:
-- **CI installed `cryptography>=41,<46` for services whose manifests required
- `>=46,<51`** — two ranges with no overlap, so the version CI tested could
- never be the version the image shipped. `connectors` and `osquery-tls` also
- floored at `44.0.1` for CVE-2024-12797 while CI was free to install `41`.
- One range, `>=46,<51`, now covers every manifest, Dockerfile and workflow,
- including `packages/aisoc-cli`, whose Ed25519 plugin signatures the API
- verifies. `PyJWT` was unbounded in eight workflow install paths and is now
- `>=2.8,<3` everywhere; `poetry` itself was 1.7.1 in five images, 1.8.2 in
- three and unpinned in two workflows, and is now `2.4.1` everywhere.
+ - `getPublicSiteUrl()` (`apps/web/src/lib/site.ts`) fell back to the hosted
+ origin, which is the `metadataBase` for the whole app — every canonical
+ tag, Open Graph URL, JSON-LD block and sitemap entry on a self-hosted
+ console pointed at another deployment. Now `http://localhost:3000`. The
+ hosted brand was also carried in `DISCOVERY_KEYWORDS`, the root layout's
+ OG/Twitter descriptions, its `sameAs`, and the PWA manifest description.
+ - Published investigation replays built share links from a module constant
+ `https://tryaisoc.com/r`, and tenant invite links had **two** independent
+ hard-coded defaults — one on the endpoint, one on the provisioner — free to
+ drift apart, which is the part that makes this hard to notice. Both now
+ resolve through a single `console_base_url()` reading
+ `CONSOLE_PUBLIC_BASE_URL`, with a documented deployment-neutral fallback.
+ - Approval email defaulted to a `From:` on the hosted domain, which fails
+ SPF/DKIM for anyone else. It now prefers the operator's own
+ `MAILGUN_DOMAIN`.
+ - `plugins/aisoc-direct/plugin.yaml` pre-filled the hosted osquery endpoint
+ as the field default; `services/osquery-tls` declared a hosted
+ `public_hostname` (and had no reader at all — the comment claimed a use it
+ did not have).
+ - The simulation-mode call to action pointed operators at docs on the hosted
+ domain rather than the project's own documentation site.
+ - The GitHub Action's PR comment hotlinked a badge served by the hosted
+ deployment, in the same line that promises "no data leaves your CI"; the
+ report card's coverage footer linked the hosted tool while its sibling
+ already linked the repository.
+ - `playwright.config.ts` defaulted one project to the hosted host and the
+ adjacent project to `localhost` — same env var, two answers.
-- **`ruff` was declared six different ways while `ruff format --check` gated
- on one of them.** `services/api` permitted `<0.17.0`, `fusion` `^0.2.0`,
- the published packages `>=0.3`, and the devcontainer installed it unpinned —
- while CI enforces `>=0.4.4,<0.5`. A contributor installing their own service's
- dev group got a ruff that reformats the tree and reds their PR with no
- dependency change in the diff. All fourteen declarations now read
- `>=0.4.4,<0.5`. The devcontainer also told contributors to `uv sync` against
- `services/api/uv.lock`, which has never existed in this tree.
+ Two admin console pages also rendered the hosted hostname as body text, and
+ the demo-mode 403 told every operator "This is the public AiSOC demo at
+ tryaisoc.com" regardless of where it was running.
+
+ A new gate, `scripts/check_hosted_hostname.py`, keeps this from coming back.
+ It is deliberately not "the string must not appear": the hostname stays in
+ 99 files that genuinely describe the managed offering — the fly/cloudflare/
+ terraform deploy configs, the marketing pages, the changelog, and two
+ `uuid5` namespace seeds that are compatibility constants rather than URLs.
+ It pins an exact occurrence count per path and fails in **both** directions:
+ a new or grown occurrence, and an exemption that outlived the occurrence that
+ justified it. A stale exemption is a standing permit for a future leak, and a
+ one-directional allow-list never notices one. It resolves its repo root from
+ the working directory rather than its own file location (a sibling gate did
+ the latter and reported a confident OK about a tree it never opened), refuses
+ a tree that fails a sentinel check, treats a zero-match scan as broken rather
+ than clean, and ships a `--self-test` proving the detector separates a
+ known-bad from a known-good sample and that the comparison rejects a vacuous
+ pass. Covered by `tests/test_hosted_hostname_gate.py` (39 cases) and
+ `.github/workflows/hosted-hostname.yml`, which runs on pull requests *and* on
+ pushes to `main` so an edit made at merge time cannot slip past.
- **The capability contract was not consulted on the dispatch path an agent
uses.** `POST /actions` has run the contract through
@@ -3019,6 +2683,29 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
reader, so the honest answer — `unverified`, with the reason — is written
explicitly.
+- **A playbook step the engine could not run reported success.** Twelve of
+ the twenty-two `StepType` members had no entry in the engine's handler
+ table. The run loop answered those with `{"skipped": true}` and left the
+ step's status at its `SUCCESS` default, so a playbook containing them ran
+ to `COMPLETED` having done nothing it said it did. `approval` is the one
+ that mattered: it is a human decision point, it appears in 14 steps across
+ the shipped packs, and it passed on its own — the run continued straight
+ into the action an analyst was meant to authorise. An unimplemented step
+ type now fails closed with `unimplemented: true` and an error naming the
+ verb, is not retried (a missing handler will still be missing next
+ attempt), and halts the run under the default `on_failure: abort` while
+ still honouring an explicit `continue`. A dry run reports `would_fail` for
+ such a step instead of a bare `dry_run: true`.
+
+ `apps/docs/docs/concepts/playbooks.md` had documented the safe behaviour
+ all along — "recorded as `SKIPPED` … so unknown actions never silently
+ succeed" — and has been corrected to describe what the code now does. The
+ same page described a manual approval gate backed by
+ `POST /v1/playbook-runs/{id}/approve`, where "the engine pauses on the
+ condition until the field flips, then resumes". No such endpoint exists and
+ the engine has no pause or resume; that section now says so and points at
+ the actions service, which does hold an action for an analyst.
+
- **A playbook run reported COMPLETED when its steps had failed.**
`on_failure: continue` decides whether the run keeps going; it was also
deciding what the run was called afterwards. 346 of the 380 steps in the
@@ -3027,6 +2714,36 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
FAILED, with an error naming how many and which, and the author's
`continue` policy still governs whether the remaining steps are attempted.
+- **The NL drafter rewrote steps to make its own output pass lint.** Because
+ the schema declared 9 of 22 step types, `_collapse_step_types_for_schema`
+ mapped the other 13 onto a "nearest neighbour" — `run_av_scan` and
+ `revoke_session` both became `investigate`, `approval` became `condition` —
+ keeping the original in `params.original_type`. The playbook that shipped
+ said it would investigate when the author had asked to disable an account,
+ and an approval gate came out as an ungated branch. The projection and its
+ second validation pass are removed; the schema covers the full range, so a
+ validation failure is now a real failure. The drafter's system prompt also
+ restated the vocabulary by hand and had drifted from it — offering
+ `webhook` as a trigger, which no validator in the repo accepts, and capping
+ `retry_max` at 5 against a model allowing 25 — and is now generated from
+ `StepType` and `bounds.py`.
+
+- **The playbook lint job validated two files and reported "2/2 passed".**
+ `scripts/lint_playbooks.py` claimed in its own docstring to check "any
+ `*.playbook.json` files anywhere in the repo" and only ever scanned the two
+ under `services/agents/data/playbooks/`. Pointed at the whole tree, 32 of
+ the 62 playbooks in `playbooks/packs/v1/` did not match the published
+ schema. It now scans recursively, treats finding no files as a broken scan
+ rather than a clean bill of health, and no longer crashes in its own error
+ path when a file passed on argv sits outside the repo.
+
+- **20 shipped playbooks carried a duplicated tag.**
+ `scripts/generate_playbooks.py` emitted `[category, *tags]` where several
+ categories already lead their tag list with the category name, producing
+ e.g. `["supply-chain", "supply-chain", "npm", …]`. Fixed in the generator,
+ which is what the reproducibility gate diffs, rather than in the 20
+ generated files.
+
- **`packages/types/src/playbook.ts` published a fourth playbook
vocabulary.** A 28-member `ActionType` union (`notify_email`,
`create_ticket_jira`, `collect_forensics`, `run_query_splunk`, …) matching
@@ -3040,1152 +2757,1350 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
`RunStatus` and the dispatch report, and `check_playbook_schema_parity.py`
now compares the published union against the engine's in both directions.
-- **`false_positive_rate` and `escalation_rate` published `0.0` on an empty
- denominator.** `x / n if n > 0 else 0.0` renders an undefined ratio as a
- measured zero, and for these two the zero is flattering: "0% false
- positives" and "0% escalated" are the two best numbers on the SOC metrics
- page, and a tenant that had resolved nothing and gated nothing scored both.
- This is the same defect as the MTTD/MTTR/MTTC means that shipped a NULL
- average as a confident `0.0`, one step removed, and it takes the same fix:
- `/metrics/soc` now reports `false_positive_rate_sample_count` and
- `escalation_rate_sample_count` alongside the rates, and the console renders
- "not measured · no resolved alerts in 7d" rather than a percentage. The
- fields are additive and optional, so a console running against an older API
- keeps its previous behaviour rather than blanking the tiles — a paired test
- holds that direction too, because "blank everything" would be a worse
- regression than the bug.
-
-- **An event ingested under a connector type with no profile became an alert
- with no host, no user and no source IP.** `_canonicalAliases` in
- `services/ingest/internal/normalizer/normalizer.go` already resolved
- `actor.user.name`, `device.name` and `src_endpoint.ip` from the spellings
- connectors actually use, but the pass that applied it was gated behind the
- canonical-envelope branch. Anything reaching the generic fallback — every
- push through `/v1/ingest/batch`, which sends a flat payload and so never
- matches that branch — resolved `title` and `external_id` and nothing else.
-
- Identity is what the rest of the platform is built on: entity extraction,
- the Investigation Rail's pivots, the `{tenant}:{entity}:{tactic}`
- correlation key, the entity graph and UEBA all key off those three fields.
- An alert still appeared, which is what made it look like it had worked, but
- it carried no entity chips and correlated into the `unknown` bucket. The
- alias pass now runs for every profile and fills only destinations the field
- map left empty, so a vendor profile's own mapping still wins.
-
- Three defects in the same family, found while fixing it:
-
- - **A nested vendor object was written whole into a scalar identity slot.**
- Fourteen of the 84 registered connectors emit `actor`, and four emit it as
- the vendor's object rather than a name, so `actor.user.name` became a map
- — an entity that renders as a map and correlates as garbage. Identity
- resolution now takes strings only and digs one level (`actor.name`,
- `actor.displayName`, `user.name`) for the scalar underneath.
- - **A vendor profile's severity ladder is spelled the way its vendor spells
- it.** `crowdstrike_falcon`'s is capitalised, and a pushed payload is
- whatever the caller wrote, so a lowercase `high` scored 0 and rendered as
- Unknown. The shared five-tier ladder is consulted when the profile's own
- has no entry for the value; `critical` stays its own tier.
- - **A vendor profile left `message` empty for a pushed payload**, because it
- maps its vendor's field name, so the promoter generated
- "Security Finding from <product>" over the title the caller actually sent.
+- **The agent recommended evidence acquisition the platform could not
+ perform.** `capture_forensics` was an `ActionType` with no executor
+ anywhere, and `services/agents/app/agents/investigation_agent.py` proposes
+ it by name whenever an investigation reaches the C2 or exfiltration stage,
+ with `requires_approval=True`. So on the most serious class of incident the
+ product raised an approval for forensic acquisition and failed with `No
+ executor found for action type` when an analyst approved it — a control
+ reachable from a recommendation and dead on approval.
-- **`connector_type` values the product advertises did not reach their
- profile.** The connectors service declares `crowdstrike` and `okta`; the
- ingest profiles were keyed `crowdstrike_falcon` and `okta_system_log`. The
- README's own push example used `crowdstrike`, so the example a new user
- copies missed the lookup, fell to the generic fallback and lost its vendor
- attribution — 80 of the 84 declared connectors had no profile entry under
- the name they are registered with.
+ It now has a Microsoft Defender arm that collects an investigation package,
+ declared LOW impact and **analyst-gated**. Analyst rather than automatic for
+ the reason that separated `suppress_alert` from `update_alert_disposition`
+ at an identical impact tier: the question is what bounds the verb. The
+ writeback is bounded by a disposition mapping that refuses to close a
+ confirmed true positive; this verb has no bound at all — it collects
+ whatever the vendor package contains from whichever host it is pointed at,
+ and the result is a copy of somebody's endpoint in a vendor cloud behind a
+ download URI. Pointed at the wrong host that is a data-handling event nobody
+ can take back.
- A `connectorTypeAliases` map resolves both spellings rather than renaming
- either. The longer names are load-bearing elsewhere — `packages/types`'
- `ConnectorType` union, the CLI's default, the graph extractor, the actions
- credential resolver — so renaming one side would have broken the other.
+ Only Defender, deliberately. MDE's investigation package is a whole-host
+ artefact bundle whose completion state and download URI are both readable.
+ CrowdStrike RTR's `get` retrieves one *named file*, which is a different
+ verb with a different blast radius; wiring it here would make one capability
+ mean two things depending on the tenant's vendor. Without MDE credentials
+ the executor simulates and says which credentials would enable it, rather
+ than reporting an acquisition that did not happen.
-- **Two registered connectors reached no normalization path at all.**
- `auditd` emitted `source` without `raw_event`, so it failed the canonical
- envelope check and lost the host it carries; it now emits both.
- `email_inbox` returns the message envelope shaped for
- `email-forwarded.yaml`, so it matched neither branch and strict mode
- rejected it outright; it now has a profile mirroring that template, the way
- `ai_runtime` mirrors `ai-runtime.yaml`.
-- **The dashboard published two numbers that contradicted the database, and a
- third surface that contradicted both.** A live acceptance pass measured a
- tenant with two cases closed in the last seven days at a 90-minute mean.
- `/mssp/portfolio` reported that correctly as 1.5h. The dashboard reported
- **CASES CLOSED (7D) 0** and **MTTR 0.0 hrs**, directly above its own
- "CASES OPENED (7D) 2".
+ The probe is real and reads the package back. Acquisition is asynchronous,
+ so the vendor's response is emphatically not the confirmation — it says a
+ machine action was queued. `Succeeded` **and** a retrievable download URI is
+ VERIFIED; `Failed` / `TimeOut` / `Cancelled` is FAILED, which is the alarm
+ worth having; `Pending` / `InProgress` is UNVERIFIED, because not finished
+ is not the same fact as not happening. Reading the status alone would
+ certify a collection that finished with nothing to download.
- The portfolio was right and the dashboard was wrong, for two separate
- reasons that each read plausibly in isolation:
+- **A delivered ChatOps prompt nobody had answered would have reported as a
+ completed action.** `ChatOpsVerifyExecutor` worked, sat in
+ `EXECUTOR_REGISTRY`, and no live-action adapter reached it, so the only
+ route to it was the legacy `ActionType` endpoint — which has no capability
+ contract, no approval matrix and no autonomy policy in front of it. Leaving
+ it unreachable was the right call at the time: it returns
+ `ActionStatus.RUNNING` to mean "the prompt went out, nobody has answered",
+ and `_to_live_status` folded everything that was not `FAILED` or a
+ simulation into `SUCCEEDED`.
- - `cases_closed_7d` filtered `status = 'resolved' AND updated_at >= …`.
- `resolved` is an *intermediate* state — the lifecycle's terminal one is
- `closed`, and it is that transition which writes `closed_at` — so a case
- that completed its lifecycle was invisible to the count. `updated_at` was
- also the wrong clock: it moves whenever anyone edits the case, so an old
- case gets a comment and re-enters the window while a closed one
- eventually leaves it.
- - `mttr_hours` averaged `alerts.resolved_at - alerts.created_at`, a
- different lifecycle on a different table from the one the portfolio
- measures. Nothing in ordinary case work writes that column, so the
- average was over zero rows and `float(None or 0.0)` published the empty
- result as a confident `0.0`.
+ The fix is the missing state rather than the exemption.
+ `LiveActionStatus.AWAITING_COMPLETION` means the vendor was touched and the
+ outcome is not yet known — the opposite of `PENDING_APPROVAL`, which means
+ nothing ran because policy wants a human first. Post-action verification
+ does not run against it, since there is no effect to read back yet. The same
+ state is what evidence acquisition needed, so one addition covers both
+ executors that had no honest result to return.
- Both now come from `app/services/resolution_time.py`, which owns the window
- and the SQL expression that *both* surfaces use, so the two cannot quote
- different MTTRs for one tenant again. The portfolio keeps computing its
- figure inside one bound cross-tenant statement — one round trip for the
- whole portfolio — and interpolates the shared fragments rather than calling
- the shared function; a test asserts it still does, and the load-bearing
- assertion is that the two surfaces produce the *same* number from the same
- rows rather than that each matches a hardcoded 90.0.
+ Registered against Slack and Teams, LOW impact and analyst-gated. Not
+ automatic like `notify` at the same impact, because `notify` addresses a SOC
+ channel and this addresses the account under investigation: sent
+ automatically on a true positive it tells an attacker they have been
+ detected, and the verb cannot know whether the person it is asking is the
+ suspect. Its dry run simulates in the adapter rather than stripping
+ credentials — the base adapter's strip-and-fall-through would turn a preview
+ into a failure for an executor that has no simulation branch by design.
- The third surface was the `MTTR` tile in the Security Operations Center
- strip, reading `alerts.mttr` off `/metrics/dashboard` — the same dead column,
- and rendered with an `m` suffix although the field is hours, so a real
- 1.5-hour MTTR would have displayed as "1.5m" had it ever been non-zero.
+- **Nothing compared what the agent recommends against what the platform can
+ execute.** `scripts/check_action_contract.py` compared four registries
+ inside the actions service and could not see a fifth: the verbs
+ `services/agents` puts in front of an analyst. It now parses every
+ `ProposedAction(action_type=...)` call site — including the conditional form
+ `attack_path_agent` uses — and fails when a proposed verb has no executor,
+ naming the file and line. Run against the tree before this change it reports
+ `capture_forensics` at `investigation_agent.py:157`.
- Rather than make the three means nullable — a breaking response change for
- every existing client — each now travels with the number of rows it was
- averaged over (`mttd_sample_count`, `mttr_sample_count`,
- `mttc_sample_count`, and `alerts.mttr_sample_count`). A mean over zero rows
- is unmeasured, not zero, and the tiles say "not measured" instead of
- claiming an unbeatable response time for a tenant that has resolved nothing.
- The fields are additive: a client that ignores them sees what it saw before.
+ Both exemption lists in that gate are now empty, and
+ `test_capability_reachability.py` asserts that emptiness directly, so a verb
+ can only be exempted by editing a reviewable assertion rather than appending
+ to a list.
-- **The entity-risk queue sent a tenant slug where a UUID was required, then
- blamed a healthy service for the rejection.** `apps/web/next.config.js`
- inlined `NEXT_PUBLIC_TENANT_ID` with a fallback of the literal string
- `'default'`. Three tenant-scoped surfaces pass that value as a query
- parameter to routes typed `tenant_id: UUID` — `/fusion/entity-risk/*`,
- `/honeytokens/*` and `/business-context/*` — and all three answered 422.
+- **Two response verbs worked and could not be reached.** `ack_alert` and
+ `suppress_alert` have had Splunk, Elastic and Defender arms since Phase 3.3
+ and were wired into `EXECUTOR_REGISTRY` — and appeared in none of the three
+ registries that make a verb dispatchable: the live-action adapters, the
+ capability contracts, or the capability vocabulary. Governed dispatch
+ answered `executor_not_found` for code that ran, which reads as a
+ misconfigured integration rather than a capability nobody connected. This is
+ the mirror image of a defect already fixed here in the other direction,
+ where eleven capabilities had a contract and no executor at all — including
+ `unisolate_host`, so the rollback for the most disruptive action in the
+ product resolved to nothing.
- Two things kept it hidden. `lib/api.ts` carries the correct canonical UUID
- as its own fallback, so reading it suggested the console was already doing
- the right thing; Next's inlining runs first, which made that fallback
- unreachable code. And the slug is not a dependable handle either: migration
- 001 seeds tenant `…0001` with slug `default` and the demo seed renames that
- slug to `demo`, so on a seeded install the literal matched neither the id
- nor the slug. The entity-risk client also read the build-time constant
- rather than `getActiveTenantId()`, so the queue ignored both the logged-in
- user and the tenant switcher; it now resolves the tenant the same way
- `request()` resolves the `X-Tenant-Id` header.
+ Both verbs now have six vendor adapters (one per vendor arm), a capability
+ contract, an entry in the vocabulary on both sides of the mirror, and a
+ verification probe. Each adapter pins `alert_vendor` so a tenant with two
+ SIEMs configured does not have the target chosen by credential ordering; the
+ pin is still checked against the credentials, so pinning a vendor the tenant
+ has not configured simulates rather than claiming an arm that could not have
+ run.
- `components/fim/FimDashboard.tsx` is deliberately pinned to `'default'`
- instead: `services/osquery-tls` types its tenant as a plain string and
- enrols nodes under that literal, so this one surface is keyed on that
- service's own convention and reading the shared UUID here would match no
- enrolled node. The two tenancy models still need reconciling in
- osquery-tls.
+ `ack_alert` is LOW impact and automatic: it marks a finding in-progress and
+ owned by AiSOC, removing nothing from anyone's view, and two analysts
+ working the same notable is the cost of not doing it. `suppress_alert` is
+ LOW impact and **analyst-gated**, because what makes the disposition
+ writeback safe to automate is the mapping that refuses to close a confirmed
+ true positive, and this verb has no such bound — it closes whatever it is
+ pointed at, on the caller's say-so.
- With the request failing, the queue rendered five invented entities —
- `[email protected]`, `updates.evil-cdn.xyz`, "last seen 5 months ago" on a
- stack that had been up for an hour. Above them, and *outside* the amber
- banner that disclosed them, four cards read "Contributing alerts 26" and
- "Alert → Incident 13.0:1" off the same sample payload. A disclosure that
- covers the rows and not the headline numbers is decorative, so the banner
- moved above the cards and every card now reads "not measured" when its
- request fails; the derived alert-to-incident ratio is withheld whenever the
- counts it divides are unknown.
+- **A one-directional gate would have missed all of it.** `check_action_contract.py`
+ now compares the four registries a verb needs in **both** directions, and
+ `test_capability_reachability.py` injects drift in each direction and asserts
+ the gate names it. The dominant failure shape in this repository is a check
+ that compares A against B and never B against A, so drift in the direction
+ things actually change passes while the check prints OK — the graph-schema
+ check reported OK with 17 node labels declared and 28 implemented.
- The banner also named the wrong subsystem. It said "Fusion service
- unreachable" while fusion was healthy and the actual failure was a 422 —
- a diagnosis that sends an operator to debug something that is not broken,
- which is worse than none. It is now derived from the response status: a 422
- reads as a console bug rather than an outage, a 5xx names fusion because
- that is when fusion is genuinely at fault, status 0 reads as unreachable,
- and an unrecognised failure stays vague rather than guessing. Every variant
- says the queue is *unknown* rather than empty.
+ Running it found three more mismatches beyond the two above. `create_ticket`
+ and `notify` were registered, contracted and dispatchable across four
+ vendors while absent from `KNOWN_CAPABILITIES` and the connectors
+ `Capability` enum, so the registry logged `capability_unknown` for them at
+ every startup; both are now in the vocabulary. `chatops_verify` has an
+ executor no adapter reaches, and three `ActionType` members
+ (`capture_forensics`, `add_ioc_to_blocklist`, `run_playbook`) have no
+ executor at all — the first of those is proposed by name by the
+ investigation agent on the C2/exfiltration path, so the product recommends
+ evidence acquisition it cannot perform. Those four are recorded in the
+ gate's exemption lists with the reason each is open. Both lists are
+ ratchets: an entry that gains an implementation and is not removed fails the
+ build.
-- **`/alerts/[id]` rendered a confidence of 21 as `2100%`, and negative
- evidence as `+-0.30`.** Confidence reaches the console on two keys at two
- scales: the API surfaces `confidence` as an integer 0-100, while fusion's
- `confidence_score` is the raw [0.0, 1.0] float the band was derived from.
- `normalizeAlert` accepted whichever key appeared first and passed it
- through unchanged, so `Alert.confidenceScore` meant one thing or the other
- depending on the payload — and its consumers guessed differently.
- `AlertDetailView` multiplied by 100; `AttackStory`, on the same page,
- divided and rendered "21/100". Each was right for one payload shape, and
- each had a passing test because its own mock used the scale it assumed.
+- **The disposition writeback now verifies itself against the vendor.** It
+ shipped declaring no verification probe, which was honest — a declared probe
+ that does not run is the defect the contract gate exists to catch — but the
+ standing rule is that an unverifiable action is not an autonomous one, and
+ this action is automatic. The probe re-reads the finding and compares its
+ state against the plan re-derived from the same verdict through the same
+ `plan_writeback` the executor used, rather than being told separately what
+ to expect. Splunk ES reads back the `incident_review` collection (a new
+ `SplunkClient.get_notable_event_state`) and confirms status `5` on a close,
+ or status `1` *and* the owner AiSOC set on an escalation, since status alone
+ cannot distinguish a notable an analyst had already picked up. QRadar
+ confirms `CLOSED` through `get_offense`, whose docstring had claimed to be
+ the writeback probe since it was written and had no caller.
- Normalised once at the boundary to the canonical 0-100 integer, deciding the
- scale from the *key* rather than the magnitude: a genuine confidence of 1 is
- indistinguishable from a raw score of 1.0 by value alone, so the tempting
- `v <= 1 ? v * 100 : v` would render the least-confident alert in the estate
- as the most confident. The two mocks that encoded the old scale were
- corrected with it, since sample data on a different scale than the real
- payload is what hid the bug.
+ A QRadar **escalation** deliberately reports `unverified`: escalating leaves
+ the offense `OPEN`, which is also its prior state, so confirming "still
+ OPEN" would certify a write that never happened — the same shape as the
+ isolation probe that returned `bool(device_id)` and would have certified an
+ uncontained host. Elastic, Sentinel and Defender expose no read of a
+ finding's state and report `unverified` too. `ack_alert` and
+ `suppress_alert` are verified against the same read-back.
- The rationale rows hardcoded a `+` prefix on a signed contribution, so a
- factor that argued *against* the verdict read `+-0.30`; they now carry one
- sign, matching the glyphs the narrative builder uses. Those rows also
- clamped `contribution / weight` into [0, 1], which rendered every negative
- factor at zero width — an invisible bar beside a nonsense label. Width now
- follows the magnitude and colour follows the sign.
+- **A dry run called the customer's production SIEM.** The live-action dry-run
+ path works by stripping credentials so the executor falls through to
+ simulation, and the strip list did not match what the client factory reads:
+ the Splunk adapters stripped `splunk_host` / `splunk_token` / `splunk_index`
+ while `_splunk_client` reads `splunk_url` *first* and also accepts basic
+ auth. `splunk_url` is exactly what the credential resolver writes for a
+ connector-configured tenant, so a "preview" built a real client and called
+ Splunk. Elastic had the identical mismatch (`elastic_host` stripped,
+ `elastic_url` read). The strip lists are now the factories' own exported key
+ tuples, and a test re-derives each read set from the factory source so the
+ two cannot drift again.
-- **The Investigation Rail's Details tab showed analysts the markup.**
- `build_narrative` documents its output as markdown-light — `**bold**`,
- backtick code spans, `- ` bullets, blank-line paragraphs — and the rail put
- the string in a `whitespace-pre-wrap` paragraph, which preserves the
- newlines and the asterisks alike. The panel read `**Medium** alert: … on
- **Finance & Legal #2**`.
+- **An `alert_vendor` pin was honoured without checking that vendor's
+ credentials existed.** `_ack_vendor` returned the pinned vendor
+ unconditionally. A dry run strips credentials, so the pin survived the strip,
+ resolved to "splunk" with no client, and hit an `assert` — crashing an action
+ that was meant to be a harmless preview. A pin naming a vendor the tenant had
+ never configured reported a vendor arm that could not run. A pin is now
+ checked against the credentials, and an unusable pin resolves to *nothing*
+ rather than falling through to whichever other SIEM happens to be configured:
+ "write this to Splunk" must not become "write this to Elastic".
- `components/alerts/NarrativeMarkdown.tsx` renders exactly that dialect and
- nothing more; an unrecognised construct falls through as literal text rather
- than being dropped. It builds React elements, never markup: the narrative
- embeds the alert title and entity names, which originate in connector
- payloads, so an HTML path here would make anyone who can name a host an XSS
- author. It introduces no heading, leaving the page's heading order intact.
+- **`CreateNotableEventExecutor` raised `TypeError` on every live call.** It
+ passed `title=` / `description=` / `fields=` to a client whose signature is
+ `(rule_name, event_data, severity, owner, status)`. Invisible because
+ simulation mode never constructs the client — the same defect class as the
+ `max_results` / `max_count` drift fixed earlier on this module. Every SIEM
+ executor-to-client call is now pinned by an autospec'd signature test, which
+ a hand-written fake with `**kwargs` could never have caught.
-- **The connector fleet badge claimed every source was reporting when none
- was.** `ConnectorFleetPanel` rendered the badge whenever the endpoint
- answered, and with zero connectors `failed + degraded` is zero — so a green
- "All sources reporting" sat directly above the panel's own "No connectors
- configured". Zero sources reporting is not the same statement as all of
- them reporting, and the green is the part an operator scans for. There is
- now no badge until there is a fleet, and when there is one it names the
- count it is vouching for.
+- **Every live Okta password reset raised `TypeError` and came back FAILED.**
+ `ResetPasswordExecutor` documents `parameters.send_email` as "used only for
+ the Okta path", reads it, and passed `send_email=` to
+ `OktaClient.reset_password(self, login_or_id)` — which did not accept it. The
+ executor's `except Exception` turned the `TypeError` into a FAILED
+ `ActionResult`, so the verb did not crash; it simply could never succeed
+ against Okta, and the failure read as an Okta problem. Simulation mode never
+ constructs a vendor client, which is why no test saw it: the one that covers
+ this executor stubs the client as `async def reset_password(self, *_args,
+ **_kw)`, and a fake that accepts anything proves the dispatch order and
+ nothing about the call. The client now takes `send_email` and maps it to
+ Okta's `sendEmail` parameter, and `test_identity_client_signatures.py`
+ autospecs all three IdP clients across every executor arm — the companion to
+ `test_siem_client_signatures.py`, for the third instance of the same defect.
-- **`apps/web/src/components/landing/MitreStrip.tsx` is deleted.** Its twelve
- ATT&CK tactic tiles carried unsourced coverage counts — 27 of 42 for Defense
- Evasion, 9 of 11 for Initial Access — that correspond to nothing in the
- tree. The component was imported nowhere and rendered on no route: the
- landing page composes fifteen sections and this was not one of them, so the
- caveat in its body copy was the only thing between those numbers and a
- reader, and it would have stopped being so the moment somebody mounted the
- section. Improving the disclaimer would have left the numbers in place for
- the next person to inherit. Its entry in `ALLOWED_ILLUSTRATIVE` goes with
- it; `scripts/check_mock_data_gated.py` checks that allow-list in both
- directions, so a stale exemption fails the build rather than accumulating as
- cover. The one MITRE figure the project does publish — 97.0% in
- `BenchmarkBand` — is labelled "substrate" and is unchanged.
+- **The lake's tenant isolation could be switched off by resolving a
+ dependency one major version higher.** `lake_sql.rewrite_for_tenant` is the
+ only thing separating one tenant's events from another's in ClickHouse: it
+ parses untrusted operator SQL with sqlglot, enforces the table allowlist
+ against the parse tree, bans ClickHouse table functions, and injects the
+ `tenant_id` predicate. sqlglot 27 moved the SELECT's FROM clause from
+ `args["from"]` to `args["from_"]`. The table walk read the old key, got
+ nothing, and took the branch written for `SELECT 1` — "no FROM, so no tenant
+ data, nothing to do". Every single-table query then came back with no
+ allowlist check, no table-function ban and no tenant predicate, reported as
+ a successful rewrite.
-- **A playbook step the engine could not run reported success.** Twelve of
- the twenty-two `StepType` members had no entry in the engine's handler
- table. The run loop answered those with `{"skipped": true}` and left the
- step's status at its `SUCCESS` default, so a playbook containing them ran
- to `COMPLETED` having done nothing it said it did. `approval` is the one
- that mattered: it is a human decision point, it appears in 14 steps across
- the shipped packs, and it passed on its own — the run continued straight
- into the action an analyst was meant to authorise. An unimplemented step
- type now fails closed with `unimplemented: true` and an error naming the
- verb, is not retried (a missing handler will still be missing next
- attempt), and halts the run under the default `on_failure: abort` while
- still honouring an explicit `continue`. A dry run reports `would_fail` for
- such a step instead of a bare `dry_run: true`.
+ `services/api/pyproject.toml` declared `sqlglot >=23.0.0,<31.0.0` while the
+ Dockerfile and every CI workflow declared `>=23,<27`, so this was reachable
+ by installing the service exactly as declared, and CI could not see it
+ because CI installed the narrow range. Verified on sqlglot 30.19.0 against
+ the pre-fix rewriter: `SELECT user_name FROM aisoc.raw_events` returned
+ unscoped, `SELECT * FROM system.tables` was accepted, and
+ `url('https://attacker.example/x', JSONEachRow)` was accepted — the last of
+ which makes the warehouse issue outbound HTTP with its own network identity.
- `apps/docs/docs/concepts/playbooks.md` had documented the safe behaviour
- all along — "recorded as `SKIPPED` … so unknown actions never silently
- succeed" — and has been corrected to describe what the code now does. The
- same page described a manual approval gate backed by
- `POST /v1/playbook-runs/{id}/approve`, where "the engine pauses on the
- condition until the field flips, then resumes". No such endpoint exists and
- the engine has no pause or resume; that section now says so and points at
- the actions service, which does hold an action for an analyst.
+ Three changes rather than one, because pinning alone would leave the trap
+ armed for the next bump. The FROM clause now resolves by node type instead
+ of by key name. `rewrite_for_tenant` ends with an audit that takes its own
+ independent census of the statement's tables and raises the new
+ `LakeSqlIsolationError` unless every one of them was scoped, the tenant
+ survived into the rendered string, and every rendered SELECT that reads a
+ lake table carries the tenant in its own WHERE — so a partially scoped
+ UNION fails too. And all seven install paths now declare one identical
+ range, enforced by `scripts/check_sqlglot_pin.py`, with
+ `.github/workflows/lake-isolation.yml` running the rewriter suites against
+ both the shipped range and the next major so a future bump fails loudly
+ instead of quietly downgrading isolation.
-- **The playbook lint job validated two files and reported "2/2 passed".**
- `scripts/lint_playbooks.py` claimed in its own docstring to check "any
- `*.playbook.json` files anywhere in the repo" and only ever scanned the two
- under `services/agents/data/playbooks/`. Pointed at the whole tree, 32 of
- the 62 playbooks in `playbooks/packs/v1/` did not match the published
- schema. It now scans recursively, treats finding no files as a broken scan
- rather than a clean bill of health, and no longer crashes in its own error
- path when a file passed on argv sits outside the repo.
+- **Scheduled hunts ran against credentials that could not exist, so every one
+ of them returned zero hits.** The event-warehouse drivers resolved their
+ endpoint and secret from `settings.ES_URL` / `ES_API_KEY` / `SPLUNK_URL` /
+ `SPLUNK_HMAC_TOKEN` / `CHRONICLE_PROJECT_ID`. None of those were declared
+ fields on `Settings`; each was read through `getattr(settings, name, None)`,
+ so the miss was silent, and `Settings` sets `extra="ignore"`, so an operator
+ who followed the resulting "set them in environment variables" message and
+ exported `ES_URL` got the same message back. The scheduler treats the
+ resulting `HuntNotConfigured` as a soft skip, so the failure was invisible.
-- **20 shipped playbooks carried a duplicated tag.**
- `scripts/generate_playbooks.py` emitted `[category, *tags]` where several
- categories already lead their tag list with the category name, producing
- e.g. `["supply-chain", "supply-chain", "npm", …]`. Fixed in the generator,
- which is what the reproducibility gate diffs, rather than in the 20
- generated files.
+ Credentials now resolve from the tenant's own `connectors` row — the one the
+ console wizard writes, encrypted by the credential vault — which is where
+ `federated.py` and `case_fanout.py` already read theirs. The warehouse is
+ per-tenant by construction as a result: a managed provider can point each
+ customer at their own cluster, which resolving from process settings made
+ impossible even in principle. `ES_URL` / `ES_API_KEY` are now declared
+ fields and remain as a deployment-wide fallback for single-cluster installs.
-- **The NL drafter rewrote steps to make its own output pass lint.** Because
- the schema declared 9 of 22 step types, `_collapse_step_types_for_schema`
- mapped the other 13 onto a "nearest neighbour" — `run_av_scan` and
- `revoke_session` both became `investigate`, `approval` became `condition` —
- keeping the original in `params.original_type`. The playbook that shipped
- said it would investigate when the author had asked to disable an account,
- and an approval gate came out as an ungated branch. The projection and its
- second validation pass are removed; the schema covers the full range, so a
- validation failure is now a real failure. The drafter's system prompt also
- restated the vocabulary by hand and had drifted from it — offering
- `webhook` as a trigger, which no validator in the repo accepts, and capping
- `retry_max` at 5 against a model allowing 25 — and is now generated from
- `StepType` and `bounds.py`.
+- **Provider selection ignored which SIEM the tenant had connected.**
+ `resolve_provider` returned the first driver whose `translated_query_key`
+ appeared in the hunt, and the natural-language translator emits ES|QL, SPL
+ *and* KQL for every question — so `esql` was always present and
+ Elasticsearch was always chosen, including for tenants who run only Splunk.
+ Selection is now driven by the tenant's enabled connectors first and the
+ available translation second.
-- **A new user who followed only the README could not log in.** `make up`
- worked, `make smoke` passed 8/8, and then authentication was impossible for
- three independent reasons at once. The only account was `[email protected]`,
- and `LoginRequest.email` is a pydantic `EmailStr`, which rejects RFC 6761
- special-use domains — so the address returned `HTTP 422 "value is not a
- valid email address: The part after the @-sign is a special-use or reserved
- name"` *before the password was ever compared*. The bcrypt hash migration
- 001 seeded matched neither the `changeme` that four documentation pages
- published nor the `admin` its own inline comment claimed; checked with this
- service's `verify_password`, every candidate returned `False`, so it
- corresponded to no known secret. And nothing existed to create a user with.
- `make demo` repaired it incidentally by writing a valid address, which left
- the demo path working and the path for running AiSOC on your own data
- broken.
+- **`POST /nl-query/execute` documented two request fields it ignored.**
+ `es_url` and `es_api_key` were described as overrides and silently dropped.
+ `es_url` now selects among the caller's own Elasticsearch connectors by
+ host, matched against connectors they own and never used as an outbound
+ target; `es_api_key` is refused with 400, because credentials belong in the
+ vault-encrypted connector rather than in a request body.
- The first administrator is now created by the deployment rather than
- committed to the repository. `services/api/app/scripts/bootstrap_admin.py`
- creates `[email protected]`, generates a password, and prints it once —
- it is stored nowhere. `make up` calls it, so the documented quick start ends
- with a credential on screen; `make bootstrap` runs it on its own and
- `make bootstrap ARGS=--reset-password` mints a new one. It is idempotent: a
- second run reports the existing account and changes nothing, which is what
- makes it safe for `make up` to call unconditionally. The address is
- validated with the same library the login route uses, so an address that
- cannot sign in is refused here with an explanation instead of becoming an
- account that fails at a login form with a schema error.
+- **`wet-eval.yml` reported success on eight consecutive weekly runs while
+ evaluating nothing.** The preflight step exited 0, every subsequent step was
+ `if: should_run == 'True'` and skipped, the notice step succeeded, and the
+ job went green. A green check that means "I did not run" is worse than a red
+ one: it is the only signal a reader gets and it says the opposite of the
+ truth. Preflight is now its own job and the run is gated on
+ `needs.preflight.outputs.should_run`, so with no funded key the benchmark
+ job reports as **skipped** — visibly distinct from success in the checks
+ list — and the preflight writes to the run summary that the published
+ numbers were not refreshed. Failing outright was the alternative and is
+ wrong here: a fork cannot configure the secret, and a weekly red cross
+ nobody can clear trains people to ignore the page. Staleness of the
+ published numbers is separately gated and does fail closed
+ (`scripts/check_scoreboard.py`, 45 days).
- Migration 001 no longer seeds a user at all, and `059` deactivates the
- orphaned row on databases that already ran it. A gate
- (`tests/test_first_run_gate.py`) now asserts that no migration ships a
- password hash and that every login example in the documentation uses an
- address the API accepts — the four pages were each self-consistent with the
- broken seed, which is why reading them found nothing.
- `golden-pipeline.yml` runs `make bootstrap` against the live stack and
- authenticates with the credential it printed, so the quick start's last step
- is covered by the same job that covers its first.
+- **The claim-gate figures gate checked every restatement of the tally except
+ the document making the claim.** `readme_gates.py` compared the README and
+ `evidence-pack.md` against the matrix rows and never compared the matrix's
+ own Summary block, so `docs/audit/CLAIM_TO_GATE_MATRIX.md` summarised
+ "GATED: 108" against 109 counted rows and every check in the repository
+ passed. The file even carries a counting note about this exact class of
+ error; it recurred because the gate written afterwards pointed outward only.
+ The matrix is now the first source checked, the bullet-list form its summary
+ uses is matched (the previous pattern needed both figures on one line and
+ silently matched nothing there), and three cases in
+ `tests/test_readme_figures_gate.py` pin it. The stale count is corrected and
+ the tally is **122 rows — 113 GATED, 9 PARTIAL, 0 NO GATE**, recomputed with
+ the script rather than typed.
-- **The published `aisoc-web:latest` image shipped demo mode baked on**, and
- it is the image `docker-compose.yml` pulls for `make up`. Next inlines
- `NEXT_PUBLIC_*` at build time, so there was no runtime escape: a
- self-hoster's console announced "Demo data resets daily at 00:00 UTC. All
- write actions are disabled." over their own real alerts, disabled every
- write control, and offered a "Self-host AiSOC" link inside an already
- self-hosted install. The only remedy was to rebuild the image.
+- **The OpenAPI gate's documented escape hatch did not exist.** The workflow
+ header told a maintainer to "re-run with `--allow-breaking`" and
+ `scripts/openapi_diff.py` claimed the flag was what "the release flow uses" —
+ but the workflow triggered on `pull_request` only, with no dispatch, no input
+ and no label check, and the diff step never passed the flag. `--allow-breaking`
+ had **no caller anywhere in the tree**. A maintainer facing a correct,
+ deliberate break had no action that worked, and the two statements describing
+ the procedure were both false.
- The demo is now its own build under its own tag. `latest`, `main` and
- `vX.Y.Z` are the product with demo mode off; `demo` and `vX.Y.Z-demo` carry
- the demo bundle, and `infra/compose/docker-compose.demo.yml` pulls those.
- Nothing has to choose between a working demo and a usable self-host image.
+ The hatch is now a PR label, `breaking-change-approved`, chosen over a
+ `workflow_dispatch` boolean because applying a label leaves an attributable
+ record of who authorised the break and when on the PR timeline. The job reads
+ that timeline and names the approver in its output. It re-runs on `labeled`
+ and `unlabeled`, so the label is a live control rather than one that waits for
+ the next push.
- Demo credentials are also no longer compiled into a non-demo bundle at all.
- `apps/web/Dockerfile` defaulted the autologin address and password to real
- values, and `login/page.tsx` and `DemoAutoLogin.tsx` each declared them as
- module-level literals — gating the *render* on `isDemoMode()` hid the panel
- but left the strings in every chunk the project ships. All three now read
- them from the build environment, and only the demo build supplies them.
+ Approval is not a skip, and the flag is no longer usable as a silent bypass:
+ `--allow-breaking` now *requires* `--changelog` and `--changelog-base`, so it
+ cannot be wired up without also wiring up the thing that records what was
+ approved. The detector still runs, and the job summary lists every breaking
+ change being permitted next to the CHANGELOG note that justified it. The
+ approval is refused if there is no `### BREAKING` section under
+ `## [Unreleased]`, or if that section is byte-identical to the base branch's —
+ checked in both directions, because "a BREAKING section exists" alone would
+ let the first note in a release cycle excuse every later break in that cycle.
-- **`make up` printed an API docs URL that 404s.** The README was corrected on
- its own; the tooling was not, so `make up`, `install.sh`, `install.ps1`,
- `scripts/lab.sh` and three documentation pages went on telling every new
- user to open `http://localhost:8000/docs` while the app mounts `/api/docs`.
- `test_readme_api_docs_url.py` now walks the tool output as well as the
- README.
+ The version-bump half of the old promise was dropped rather than implemented:
+ this repository accumulates under `[Unreleased]` and bumps `VERSION` at
+ release-cut, so a per-PR version check would demand something no PR can
+ correctly do. The comment now says what the control does.
-- **The README never told anyone to create `.env`.** `make doctor` correctly
- reported it missing and printed the fix; the quick start it was meant to
- support skipped the step. It is now the second line of the quick start.
+ Thirteen new tests, all of which fail against the pre-change tree. Four are
+ wiring assertions over the workflow YAML itself — that some step passes
+ `--allow-breaking`, that the step passing it is guarded by the label and
+ presents its evidence, that the unapproved path still blocks, and that
+ `labeled` is in the trigger types. A unit-tested function with no caller is
+ indistinguishable from a working feature until something asserts the call.
-- **`make doctor` reported a port held by a foreign process as held by us.**
- The check was `docker compose ps -q <service>`, which lists containers in
- any state — so a Postgres that exited *because* the port was taken still
- counted, and the doctor printed "port 5432 in use by aisoc postgres" while
- an unrelated process had it. The two remedies are opposites. It now asks
- which host port our own container actually publishes, names the container or
- process that holds the port otherwise, and reports a service already
- remapped to a different port as fine rather than as a conflict.
+- **`screencast.yml` could never get past its third step.** Its
+ `cache-dependency-path` named `apps/web/pnpm-lock.yaml`, which does not
+ exist — this is a pnpm workspace with one lockfile at the root — and
+ `setup-node` hard-fails when the cache path matches nothing. Every other
+ workflow in the repo already pointed at the root lockfile; being
+ `workflow_dispatch`-only meant no scheduled run ever exercised it.
-- **`make up` hit port conflicts with no explanation.** `docker compose up`
- reports `port is already allocated` against whichever container lost the
- race, after a minute of unrelated output and without naming what holds it.
- `make up` now runs the port check first (`doctor.sh --ports-only`) and stops
- before starting anything.
+- **Four more known-red CI entries, and the reason each could sit on `main`.**
+ None was anyone's recent breakage; all four reproduced on a pristine
+ checkout. What they had in common is more useful than what broke: in every
+ case the thing that should have objected either did not run, was not
+ required, or could not fail.
-- **The compose security note recommended a remedy that does nothing.** Both
- compose files told operators to change a host binding with a
- `docker-compose.override.yml`. Compose *appends* sequences when it merges,
- so an override without `!override` publishes the new binding alongside the
- old one and leaves the conflict in place. Both notes now show the tag, and
- `ports: !reset []` for removing a publishing entirely.
+ - **`Backup → destroy → restore` had lost its object store for the second
+ time.** `minio/minio` was withdrawn from Docker Hub (the repository 404s),
+ the job was moved to the quay.io mirror pinned by digest, and quay.io now
+ answers **401 UNAUTHORIZED for every tag and for that digest** — the whole
+ repository, not one tag. A third MinIO coordinate would be the same bet a
+ third time, so the fixture is now Versity's Apache-2.0 S3 gateway
+ (`ghcr.io/versity/versitygw`, pinned to the v1.8.0 multi-arch index
+ digest). It is a real S3 server rather than a mock, so `backup.sh` and
+ `restore.sh` still drive `aws s3 cp/ls/rm` and `s3api head-object` for
+ real; and it is on GHCR, which this workflow's own images already use and
+ which cannot be withdrawn independently of the CI it runs on. The job also
+ gained a readiness probe: `docker run -d` succeeding only means the
+ container was created, and the previous shape would have spent the bucket
+ loop's retries before failing with a connection error that said nothing
+ about why.
+ - **`docker compose up — full stack` failed building `services/realtime`,
+ and not for the reason it looked like.** The lock and the manifest do
+ agree (`npm install --package-lock-only` is a no-op against the committed
+ lock), the Dockerfile does copy `package-lock.json` into both stages, and
+ Node 22 is consistent across the Dockerfile, CI and `@types/node`. The
+ failure was `esbuild`: a transitive of `tsx` that the image never runs,
+ whose install script hardlinks the platform binary into place and then
+ immediately execs it to read its version, with no retry — under BuildKit
+ that exec loses a race with the writer still holding the inode and fails
+ `ETXTBSY`. The builder stage now installs with `--ignore-scripts`, which
+ it can do because the only thing it needs from devDependencies is `tsc`.
+ The emitted `dist/` is byte-identical. The runtime stage deliberately
+ keeps its install scripts: what it installs is what ships.
+ - **`test_business_context_hotpath.py` reached a real database.**
+ `FusedAlertTriageWorker.triage()` opens asyncpg connections through
+ `business_context._load_tenant_rules` and `ledger.persist_auto_triage`, so
+ with an inherited `DATABASE_URL` two tests failed with `InterfaceError:
+ cannot perform operation: another operation is in progress` — pytest-asyncio
+ gives each test its own event loop, and the module-level pool one test
+ opens is unusable by the next. The agents unit suite now declares that it
+ runs with no database, in `tests/conftest.py`, and **enforces it**: the
+ inherited DSN is removed and a real asyncpg connection raises. The guard
+ is a `BaseException` on purpose — every call site it covers is wrapped in
+ a fail-soft `except Exception`, which is correct in production and is
+ exactly why the bug was invisible, so an `Exception` here would be
+ swallowed by the handlers it exists to police. Two tests that drive the
+ real client against a DSN they set themselves are marked
+ `touches_database`. Replaces two earlier workarounds that did not work:
+ `os.environ.setdefault("DATABASE_URL", "")` is a no-op precisely when the
+ variable is set.
+ - **`test_a_missing_handler_is_not_retried` asserted nothing that could
+ fail.** It named `RUN_AV_SCAN` as its missing handler; that stopped being
+ true when the response verbs were wired into `_HANDLERS` via
+ `RESPONSE_STEP_TYPES`, so the step took the *retry* branch and was
+ attempted four times over fourteen seconds — the behaviour the test's own
+ docstring forbids — while reporting green. Its one behavioural assertion,
+ `_elapsed_ms == 0`, could not notice: `t0` resets at the top of every
+ attempt, so the field is the duration of the final attempt alone, and it
+ reads 0 both when the engine skips the retry loop (which writes a literal
+ `0`) and when a handler fails in under half a millisecond. The test now
+ takes its step type from the registry instead of naming one, so it cannot
+ silently rot the same way again, and asserts the property directly by
+ recording backoff sleeps. The file went from 14.13s to 0.16s. The
+ `patch_handlers` fixture now saves originals with `setdefault`, so a
+ second install of one key cannot write a test double permanently into the
+ process-wide handler registry.
-- **The agent recommended evidence acquisition the platform could not
- perform.** `capture_forensics` was an `ActionType` with no executor
- anywhere, and `services/agents/app/agents/investigation_agent.py` proposes
- it by name whenever an investigation reaches the C2 or exfiltration stage,
- with `requires_approval=True`. So on the most serious class of incident the
- product raised an approval for forensic acquisition and failed with `No
- executor found for action type` when an analyst approved it — a control
- reachable from a recommendation and dead on approval.
+ Why `main` carried them: `backup-restore` and `docker compose up — full
+ stack` fail hard but are **not** among the nine required checks, and compose
+ smoke is pull-by-default — it only builds a service when that service's
+ build context changed, so it is usually green without building the thing
+ that was broken. `test_business_context_hotpath.py` runs inside `Python —
+ Tests`, which **is** required, and passed only because GitHub's runners have
+ nothing on 5432 and the agents step sets no `DATABASE_URL` — green by
+ property of the runner, not of the code. `test_playbook_engine_correctness.py`
+ was named by no workflow at all; it is one of **62 of the 82** test files
+ under `services/agents/tests` in that position. The playbook and per-tenant
+ business-context files are added to the agents test list (gated agents tests
+ 529, up from 303); the remaining gap is real and is not closed here.
- It now has a Microsoft Defender arm that collects an investigation package,
- declared LOW impact and **analyst-gated**. Analyst rather than automatic for
- the reason that separated `suppress_alert` from `update_alert_disposition`
- at an identical impact tier: the question is what bounds the verb. The
- writeback is bounded by a disposition mapping that refuses to close a
- confirmed true positive; this verb has no bound at all — it collects
- whatever the vendor package contains from whichever host it is pointed at,
- and the result is a copy of somebody's endpoint in a vendor cloud behind a
- download URI. Pointed at the wrong host that is a data-handling event nobody
- can take back.
+- **Four tests failed on a clean checkout and belonged to nobody.** A suite
+ with known-failing tests teaches everyone to skim past red, so each is now
+ either fixed or skipped with a reason that says what to do about it.
- Only Defender, deliberately. MDE's investigation package is a whole-host
- artefact bundle whose completion state and download URI are both readable.
- CrowdStrike RTR's `get` retrieves one *named file*, which is a different
- verb with a different blast radius; wiring it here would make one capability
- mean two things depending on the tenant's vendor. Without MDE credentials
- the executor simulates and says which credentials would enable it, rather
- than reporting an acquisition that did not happen.
+ - `test_hunt_scheduler_cron.py::TestNextFireAt::test_basic_hourly` asserted
+ croniter's answer (the top of the next hour) from a class that, unlike its
+ neighbour, carried no `_CRONITER_AVAILABLE` guard — so without the
+ dependency it failed rather than skipped. Not environmental: the fallback
+ is an interval stepper with defined behaviour that nothing was checking.
+ It now asserts the right answer on each path, so both are tested.
+ - `test_router_report.py::test_findings_with_html_tags_are_escaped` required
+ the literal `&lt;script&gt;`, which only the `markdown` branch produces —
+ the `<pre>` fallback escapes the already-escaped body again and emits
+ `&amp;lt;script&amp;gt;`. That is more escaped, not less, so a correct
+ branch was reporting a security failure. It now asserts the property that
+ matters (nothing reaches the document as an executable tag, and the
+ payload is still present rather than dropped) on either branch.
+ - `test_router_report.py::test_html_wraps_markdown_with_document_chrome`
+ genuinely needs `markdown` — a declared dependency of the service — and
+ now skips with that stated, while CI installs it and runs the file, which
+ it had never done.
+ - `test_graph_freshness.py::test_graph_writer_does_not_block_fusion_on_failure`
+ treated "something answered on port 8080" as "the ingest service
+ answered". Its only escape hatch was a connection error, so an unrelated
+ local service returning 401 became a failure reading "fusion should never
+ block on graph". It now checks `GET /health` reports `service: ingest`
+ before asserting anything, and the skip names what did answer.
- The probe is real and reads the package back. Acquisition is asynchronous,
- so the vendor's response is emphatically not the confirmation — it says a
- machine action was queued. `Succeeded` **and** a retrievable download URI is
- VERIFIED; `Failed` / `TimeOut` / `Cancelled` is FAILED, which is the alarm
- worth having; `Pending` / `InProgress` is UNVERIFIED, because not finished
- is not the same fact as not happening. Reading the status alone would
- certify a collection that finished with nothing to download.
+- **The same commit did not build the same way twice, and one of the ways it
+ could build did not start.** `One real event through the real pipeline`
+ failed and then passed on re-run with no code change. The API container had
+ died at import with `AssertionError: Status code 204 must not have a
+ response body` from `app/api/v1/endpoints/community.py:183` — a file
+ byte-identical to `main`, on a branch that touched nothing under
+ `services/api`.
-- **A delivered ChatOps prompt nobody had answered would have reported as a
- completed action.** `ChatOpsVerifyExecutor` worked, sat in
- `EXECUTOR_REGISTRY`, and no live-action adapter reached it, so the only
- route to it was the legacy `ActionType` endpoint — which has no capability
- contract, no approval matrix and no autonomy policy in front of it. Leaving
- it unreachable was the right call at the time: it returns
- `ActionStatus.RUNNING` to mean "the prompt went out, nobody has answered",
- and `_to_live_status` folded everything that was not `FAILED` or a
- simulation into `SUCCEEDED`.
+ Measured, from the failed run's own log: `poetry install` hit a transient
+ error, the Dockerfile's pip fallback took over, and it pinned
+ `fastapi>=0.111,<0.112` and installed 0.111.1. In the same build `fusion`'s
+ poetry install succeeded and took 0.141.1. One commit, two resolvers, two
+ answers.
- The fix is the missing state rather than the exemption.
- `LiveActionStatus.AWAITING_COMPLETION` means the vendor was touched and the
- outcome is not yet known — the opposite of `PENDING_APPROVAL`, which means
- nothing ran because policy wants a human first. Post-action verification
- does not run against it, since there is no effect to read back yet. The same
- state is what evidence acquisition needed, so one addition covers both
- executors that had no honest result to return.
+ `community.py` has `from __future__ import annotations`, so its `-> None`
+ return annotation reaches FastAPI as the string `"None"`. FastAPI resolves
+ it through `ForwardRef` to `NoneType` — truthy — rather than the falsy
+ `None` singleton it gets without PEP 563, concludes the route returns a
+ body, and asserts that a 204 does not. **Every release from 0.111.0 through
+ 0.116.2 raises; 0.117.0 and later do not.** The declared range was
+ `>=0.111,<0.142`, so 29 of the 120 releases it permitted could not import
+ the service at all, and the fallback pinned squarely inside that band. The
+ earlier triage read the shape as innocent because it was checked without
+ PEP 563 active, which is the one condition that makes it fail.
- Registered against Slack and Teams, LOW impact and analyst-gated. Not
- automatic like `notify` at the same impact, because `notify` addresses a SOC
- channel and this addresses the account under investigation: sent
- automatically on a true positive it tells an attacker they have been
- detected, and the verb cannot know whether the person it is asking is the
- suspect. Its dry run simulates in the adapter rather than stripping
- credentials — the base adapter's strip-and-fall-through would turn a preview
- into a failure for an executor that has no simulation branch by design.
+ The fix is determinism rather than a widened assertion:
-- **Nothing compared what the agent recommends against what the platform can
- execute.** `scripts/check_action_contract.py` compared four registries
- inside the actions service and could not see a fifth: the verbs
- `services/agents` puts in front of an analyst. It now parses every
- `ProposedAction(action_type=...)` call site — including the conditional form
- `attack_path_agent` uses — and fails when a proposed verb has no executor,
- naming the file and line. Run against the tree before this change it reports
- `capture_forensics` at `investigation_agent.py:157`.
+ - **All thirteen Python services install from a committed `poetry.lock`.**
+ Three had one; ten now do. Two `--no-cache` builds of one commit produce
+ an identical `pip freeze` (129 packages, same SHA-256), and a new
+ `Reproducible builds / twice` job asserts it on every change.
+ - **The pip fallbacks are gone** from all eight Dockerfiles that had one.
+ Each carried its own copy of the dependency list under a comment asking
+ for lockstep, and each had drifted. A build that fails is recoverable; an
+ image that boots on versions nothing tested is not.
+ - **Four services that hand-listed their dependencies in the Dockerfile**
+ (`honeytokens`, `purple-team`, `ueba`, `mesh`) now install from a lock
+ too. All four had drifted from the manifest they mirrored — `mesh` shipped
+ `cryptography<50` against a manifest reading `<51`.
+ - **The FastAPI floor is 0.117** in all thirteen manifests and every
+ workflow. `services/api/tests/test_fastapi_floor.py` pins the boundary and
+ fails on 0.116.2 with the original error; a matrix leg imports the service
+ at both ends of the declared range.
- Both exemption lists in that gate are now empty, and
- `test_capability_reachability.py` asserts that emptiness directly, so a verb
- can only be exempted by editing a reviewable assertion rather than appending
- to a list.
+- **`Event spine (real containers)` had a genuine race, not a flaky
+ environment.** The graph-writer assertion was
+ `docker compose logs … 2>/dev/null | grep -q "<early string>"` inside a step
+ running under `set -euo pipefail`. `grep -q` exits the instant it matches —
+ and that string is line 4 of the service's output — which closes the pipe
+ while Compose is still writing; Compose exits 255 on the broken pipe and
+ `pipefail` promotes that over grep's 0. The pipeline reported failure over a
+ line that was present, which the failing run proves: the log dump its own
+ error handler printed 115 ms later contains the exact string.
-- **A self-hosted install handed its own users links into the maintainers'
- deployment.** Beyond the CORS and demo-credential entries above, ten shipped
- defaults resolved to the hosted host when left unconfigured, so an operator
- who never set an override sent their users somewhere else:
+ Reproduced deterministically rather than inferred — with a large log the
+ pipeline returns 255 every time, with a small one that fits the pipe buffer
+ it returns 0 every time, and the match is in both. That size dependence is
+ why it read as a flake. `2>/dev/null` made it worse by discarding the only
+ message that distinguished "Compose failed" from "the line is absent", so
+ the run reported the wrong cause. Fixed by capturing the logs once and
+ grepping the capture, and by checking Compose's own exit status separately —
+ not by a retry or a longer sleep, which would have converted a real race
+ into a slower real race. `scripts/doctor.sh` has the same shape twice and is
+ unaffected: it does not set `pipefail`.
- - `getPublicSiteUrl()` (`apps/web/src/lib/site.ts`) fell back to the hosted
- origin, which is the `metadataBase` for the whole app — every canonical
- tag, Open Graph URL, JSON-LD block and sitemap entry on a self-hosted
- console pointed at another deployment. Now `http://localhost:3000`. The
- hosted brand was also carried in `DISCOVERY_KEYWORDS`, the root layout's
- OG/Twitter descriptions, its `sameAs`, and the PWA manifest description.
- - Published investigation replays built share links from a module constant
- `https://tryaisoc.com/r`, and tenant invite links had **two** independent
- hard-coded defaults — one on the endpoint, one on the provisioner — free to
- drift apart, which is the part that makes this hard to notice. Both now
- resolve through a single `console_base_url()` reading
- `CONSOLE_PUBLIC_BASE_URL`, with a documented deployment-neutral fallback.
- - Approval email defaulted to a `From:` on the hosted domain, which fails
- SPF/DKIM for anyone else. It now prefers the operator's own
- `MAILGUN_DOMAIN`.
- - `plugins/aisoc-direct/plugin.yaml` pre-filled the hosted osquery endpoint
- as the field default; `services/osquery-tls` declared a hosted
- `public_hostname` (and had no reader at all — the comment claimed a use it
- did not have).
- - The simulation-mode call to action pointed operators at docs on the hosted
- domain rather than the project's own documentation site.
- - The GitHub Action's PR comment hotlinked a badge served by the hosted
- deployment, in the same line that promises "no data leaves your CI"; the
- report card's coverage footer linked the hosted tool while its sibling
- already linked the repository.
- - `playwright.config.ts` defaulted one project to the hosted host and the
- adjacent project to `localhost` — same env var, two answers.
+- **Two more gates were exempt from the empty-corpus rule only by accident.**
+ `check_route_auth.py` and `check_tenant_query_predicates.py` did not exit 0
+ over an empty tree, but neither had a corpus floor: each happened to fail
+ first on its own stale-exemption ratchet, because every entry stopped
+ matching at once. That is a true statement about the wrong thing, and it
+ disappears the moment somebody empties the table. Both now refuse the
+ corpus itself, ahead of every output mode including `--inventory`, which CI
+ runs as its own step — a green step printing `TOTAL 0` is the same defect
+ one level out. The route gates share one floor, in
+ `check_route_tenant_scope.py`, because they share one collector; the
+ predicate gate refuses each of the three ways its scan can empty
+ separately, since files, tenant-scoped tables and statements reaching zero
+ are three different losses and the middle one is the quiet one. Proven both
+ ways: with every exemption table emptied the floor is still what refuses,
+ and the real repository is unaffected.
- Two admin console pages also rendered the hosted hostname as body text, and
- the demo-mode 403 told every operator "This is the public AiSOC demo at
- tryaisoc.com" regardless of where it was running.
+ `check_route_auth.py` now imports its scanner as a module rather than by
+ name. It took `REPO_ROOT` and `SERVICES_DIR` by value at import time, so the
+ shared self-test rebinding them would have left this gate scanning the real
+ checkout while believing it was pointed at an empty one — a self-test that
+ proves nothing, which is the shape the whole exercise is about.
- A new gate, `scripts/check_hosted_hostname.py`, keeps this from coming back.
- It is deliberately not "the string must not appear": the hostname stays in
- 99 files that genuinely describe the managed offering — the fly/cloudflare/
- terraform deploy configs, the marketing pages, the changelog, and two
- `uuid5` namespace seeds that are compatibility constants rather than URLs.
- It pins an exact occurrence count per path and fails in **both** directions:
- a new or grown occurrence, and an exemption that outlived the occurrence that
- justified it. A stale exemption is a standing permit for a future leak, and a
- one-directional allow-list never notices one. It resolves its repo root from
- the working directory rather than its own file location (a sibling gate did
- the latter and reported a confident OK about a tree it never opened), refuses
- a tree that fails a sentinel check, treats a zero-match scan as broken rather
- than clean, and ships a `--self-test` proving the detector separates a
- known-bad from a known-good sample and that the comparison rejects a vacuous
- pass. Covered by `tests/test_hosted_hostname_gate.py` (39 cases) and
- `.github/workflows/hosted-hostname.yml`, which runs on pull requests *and* on
- pushes to `main` so an edit made at merge time cannot slip past.
+- **`check_gate_contract.py` can now ask "the directory exists and is empty",
+ not only "the repository is not there".** Its scratch tree omitted
+ `services/` and `detections/` entirely, so a gate opening with
+ `if not X.is_dir(): return 2` refused it for a reason that says nothing
+ about its corpus — and the way a corpus is actually lost is a renamed
+ package or a glob that stopped matching, neither of which removes the
+ directory. A second **skeleton** shape creates them empty and every check is
+ probed against both. The cost was measured rather than assumed before
+ committing to it: 64 of the 71 checks already refused the skeleton, and of
+ the five that did not, three were defects now fixed and two were already
+ recorded exceptions. The exception table is keyed by shape and has five
+ entries. Probe runtime went from ~8 s to ~16 s.
-- **The MSSP console showed six invented tenants and made no API call at all.**
- `MSSPDashboardView.tsx` declared `const TENANTS = [...]` — "Acme Financial",
- "GlobalRetail Corp", "MedSecure Health" and three more, with invented alert
- counts, MTTD/MTTR figures, risk scores, analyst headcounts and ARR — handed
- it to `useState`, and never fetched anything. Every operator on every
- deployment saw the same six rows, permanently, with no state in which they
- would not. The "Export Report" button raised a success toast and did nothing.
+- **`check_route_tenant_scope.py` reported OK having scanned zero routes.** It
+ refused a tree with no `services/` directory, which is not how a scan loses
+ its corpus: a renamed package, a changed decorator spelling or a walk that
+ stops descending all leave the directory in place. Against a `services/`
+ tree with no routes in it the gate printed `scanned 0 routes across 0 files`
+ and then `OK: every route taking a tenant identifier authenticates` — the
+ same sentence CI shows on a real pass. It now exits 2, and `--self-test`
+ carries the case. Naming the count was only half the fix: the number was
+ already printed, directly above the clean verdict.
- It now reads `GET /api/v1/mssp/portfolio` and
- `GET /api/v1/mssp/portfolio/alerts` with no sample-data fallback and no SWR
- `fallbackData` (supplying it disables revalidation, so a placeholder becomes
- what the view permanently shows). Four states, each saying which it is:
- loading; a `403` explained as "you do not manage any tenants" rather than an
- outage; a portfolio failure surfaced with its message and a retry; and an
- empty portfolio that distinguishes "this organisation manages no tenants"
- from "you were granted none" using `portfolio_wide` and `scoped_tenants`,
- because those have different fixes. The alert feed has its own states so a
- failure there does not claim the portfolio is down. Export now writes a CSV
- of the rows on screen — a real action rather than a toast.
+- **The dependency audit reported success over a tree with no manifests.**
+ `security_audit.py`'s pnpm, python and go arms each discovered zero targets
+ and printed `0 findings`, which is the sentence a clean audit of twenty
+ services prints. The file already treated an unscanned service as a failure
+ rather than a warning — the gap was that a corpus of zero was never
+ *unscanned*, just empty. All three now record a coverage gap when discovery
+ finds nothing, and `validate-ignores` refuses a missing policy file instead
+ of reporting `Validated 0 ignore entries` and exiting 0. `get_repo_root()`
+ also stopped falling back to `Path.cwd()`, which meant a run from anywhere
+ else audited whatever manifests happened to be under it.
- **ARR, risk score and analyst allocation are gone, not sourced.** There is no
- revenue, composite-risk or analyst-allocation data anywhere in this product.
- A column of nulls would still imply the measurement exists.
+- **The dependency audit only ever looked at one of the repository's two pnpm
+ install roots.** `run_pnpm_audit` ran `pnpm audit` in the repo root and
+ nowhere else, so `apps/mobile` — a deliberately separate install root, with
+ its own `pnpm-workspace.yaml` so its installs stop rewriting the root lock —
+ was never scanned. Two high-severity advisories stood open there while the
+ `security-audit` job reported a clean workspace, which is the same shape as
+ the stale `poetry.lock` that once dropped `services/slack-bot` and hid eleven
+ advisories. Install roots are now discovered from the tree (every directory
+ holding a `pnpm-lock.yaml`), each is audited, and a finding names the root it
+ came from rather than a generic "pnpm workspace". A tree with no lockfile
+ anywhere is recorded as a coverage gap instead of reported as clean.
-- **The fabricated-data gate could not see either of the two worst cases.**
- `scripts/check_mock_data_gated.py` recognised only mock data that announces
- itself: all three of its patterns required a `MOCK_` / `DEMO_` name *and* an
- assignment through a state setter or SWR `fallbackData`, which models one
- situation — a view that fetches and substitutes a sample when the fetch
- fails. A dataset written inline under an ordinary name, in a component with
- no fetch at all, matched nothing; and with no fetch there was no real path
- for a fallback to fall back *from*, which is the worse defect, not the
- lesser one. The gate reported "All sample-data fallbacks are gated behind
- demo mode" while both fabricated tables shipped.
+ The two advisories it surfaced (`GHSA-5p2g-fcmc-qvqq`, `GHSA-w3rx-r6r6-pgpr`
+ in `image-size`) are suppressed with an expiry rather than fixed, because no
+ fix is reachable: `apps/mobile` resolves `image-size 1.2.1` solely through
+ `metro 0.83.3`, no patched 1.x exists, every metro release through 0.87.1
+ still requires `^1.0.2`, and `image-size` 2.x drops the callable default
+ export `metro/src/Assets.js` calls — so an override would break asset
+ resolution at bundle time rather than bump a version. Dependabot's updater
+ reaches the same verdict independently (`security_update_not_possible`).
+ Both are denial-of-service only and build-time, reachable only from an image
+ already committed to this repository.
- A third check looks for what makes fabricated domain data harmful rather
- than for what an author happened to call it: a module-scope array of records
- that names an entity a customer would recognise — a company, a person, a
- host, an IP, an address — *and* attaches numbers to it. Numeric keys that
- describe how something is drawn are excluded, which is what keeps it quiet:
- 93 module-scope object arrays in the console, 7 matched, and the 4 already
- behind `demoFallback()` were the mocks. It newly caught
- `MSSPDashboardView.tsx` and `TeamAnalyticsView.tsx`.
+- **Twenty-two dead paths in comments and docstrings**, found by the new
+ gate on its first run. Among them: cost provenance pointing at migration
+ `055` when the file is `063`; the Wazuh severity table pointing at
+ `apps/docs/connectors/` instead of `apps/docs/docs/connectors/`; eight
+ copies of the vendored `tenant_scope.py` pointing at a
+ `check_vendored_tenant_scope.py` that is spelled `sync_`; eleven files
+ pointing at a root-level `tests/test_security_defaults.py` that lives
+ under `services/api/`; and three separate pointers at CI checks that had
+ never been written, one of which `check_action_contract.py` already
+ documented as fictional in its own docstring.
- The reviewed-exception list is checked in **both** directions: an entry that
- no longer matches anything fails the gate, so an exemption cannot outlive the
- code it excused and become cover for whatever is written next under that
- name. One entry, `MitreStrip.tsx` — a public landing-page illustration whose
- visible copy already tells the reader the tiles are illustrative.
- `tests/test_mock_data_gate.py` covers both properties, including that filter
- lists, graph stylesheets, decorative SVG coordinates and pricing copy stay
- unflagged; a gate that cries wolf on every configuration array gets deleted,
- which is worse than the gap.
+- **`main` went red because an install list depended on an extra it never
+ declared, and an upstream release stopped supplying it by accident.** The
+ `Python — Service unit tests (fusion, honeytokens, purple-team)` job failed
+ on `purple-team`'s `test_every_api_route_requires_auth` with
+ `ModuleNotFoundError: No module named 'greenlet'` — 99 other tests in the
+ job passed. Nothing about `purple-team` had changed: the test has imported
+ `app.api.routes` since it was written, that module has always imported
+ `sqlalchemy.ext.asyncio`, its `pyproject.toml` has always declared
+ `sqlalchemy[asyncio]`, and its `poetry.lock` resolves `greenlet 3.5.6`. The
+ declaration was right and the install path was wrong: this job consults
+ neither the manifest nor the lock, it pip-installs a hand-curated list, and
+ that list named a bare, unbounded `sqlalchemy`.
-- **`/analytics/team` ranked six invented analysts by invented accuracy.**
- "Sarah Chen, 47 cases closed, 96.2% accuracy, score 945" and five more, plus
- a highlights feed of things that never happened. Nothing in the platform
- measures per-analyst performance — no route, no table, no column — so the
- sample is confined to the hosted demo via `canUseDemoData()` and everyone
- else is told plainly that the measurement does not exist yet. The aggregate
- tiles read `—` rather than dividing by zero.
+ It passed for months anyway. Through SQLAlchemy 2.0.x, `greenlet` was
+ required *outside* the `asyncio` extra whenever `platform_machine` matched
+ one of `aarch64 | ppc64le | x86_64 | amd64 | AMD64 | win32 | WIN32` — true
+ on `ubuntu-latest` — so a bare `sqlalchemy` installed it incidentally and
+ the extra was load-bearing and unnamed at the same time. SQLAlchemy 2.1.0
+ removed that clause, leaving `greenlet>=1; extra == "asyncio"` as the only
+ requirement. 2.1.0 was published at 20:12:49 UTC on 2026-09-24; the last
+ green commit on `main` (`ba0c6429`) is timestamped eleven minutes before it
+ and the first red one (`ad775e8d`, #829) ten minutes after. #829 was an
+ LLM-routing change that touched no file under `services/purple-team` and
+ nothing in that import chain — it is the commit whose run happened to
+ re-resolve first, not the cause. So this was neither a dependency that went
+ missing nor an import path newly reached; it was an unpinned install
+ re-resolving across an upstream minor.
-- **The OpenAPI gate's documented escape hatch did not exist.** The workflow
- header told a maintainer to "re-run with `--allow-breaking`" and
- `scripts/openapi_diff.py` claimed the flag was what "the release flow uses" —
- but the workflow triggered on `pull_request` only, with no dispatch, no input
- and no label check, and the diff step never passed the flag. `--allow-breaking`
- had **no caller anywhere in the tree**. A maintainer facing a correct,
- deliberate break had no action that worked, and the two statements describing
- the procedure were both false.
+ The job now installs `"sqlalchemy[asyncio]>=2,<3"`, matching both the
+ manifest and the wave-2 matrix job, which had it right all along.
+ `greenlet` is deliberately *not* added as a top-level pin: the extra exists
+ to pull it, and naming the transitive package instead would record the
+ workaround rather than the dependency. `isolation.yml`, which had done
+ exactly that — bare `sqlalchemy` plus an explicit `greenlet` — now declares
+ the extra and drops the compensating entry.
- The hatch is now a PR label, `breaking-change-approved`, chosen over a
- `workflow_dispatch` boolean because applying a label leaves an attributable
- record of who authorised the break and when on the PR timeline. The job reads
- that timeline and names the approver in its output. It re-runs on `labeled`
- and `unlabeled`, so the label is a live control rather than one that waits for
- the next push.
+- **The wave-1 service-test job installed twelve packages with no version
+ bound, which is the outage that already happened, still loaded.** A CI job
+ pip-installing a hand-curated list named a bare, unbounded `sqlalchemy` and
+ passed for months only because a platform marker happened to pull
+ `greenlet`; when 2.1.0 deleted that clause, every import of
+ `sqlalchemy.ext.asyncio` began failing at collection. That one was bounded.
+ The other twelve in the same list were not, and the job's own comment said
+ so — *"Bounding the range is the second half: unbounded, any upstream
+ release lands here before anyone reads it."* `pydantic`,
+ `pydantic-settings`, `pytest`, `pytest-asyncio`, `structlog`, `redis`,
+ `aioredis`, `httpx`, `aiokafka`, `asyncpg`, `clickhouse-driver` and `pyyaml`
+ now carry the range the three services under test declare, so the job
+ installs what they ship rather than whatever PyPI is serving this morning.
- Approval is not a skip, and the flag is no longer usable as a silent bypass:
- `--allow-breaking` now *requires* `--changelog` and `--changelog-base`, so it
- cannot be wired up without also wiring up the thing that records what was
- approved. The detector still runs, and the job summary lists every breaking
- change being permitted next to the CHANGELOG note that justified it. The
- approval is refused if there is no `### BREAKING` section under
- `## [Unreleased]`, or if that section is byte-identical to the base branch's —
- checked in both directions, because "a BREAKING section exists" alone would
- let the first note in a release cycle excuse every later break in that cycle.
+ Two things fell out of doing it, both pre-existing and both invisible while
+ the install was unbounded. `redis` was installed bare while
+ `services/fusion` declares `redis[hiredis]` — the same dropped-extra shape
+ as the `sqlalchemy` break, an install path quietly installing a smaller set
+ than ships. And the three manifests **could not be satisfied at once**:
+ `services/fusion` declared `httpx = "^0.26.0"` while `honeytokens` and
+ `purple-team` declare `>=0.27.0`. The unbounded install resolved 0.28.1, so
+ fusion was being tested on a version its own manifest forbade and shipping
+ 0.26.0 — tested and shipped were different software, which is the exact
+ thing `check_dependency_pins.py` exists to prevent. fusion moves to
+ `>=0.27,<0.29`, matching api / actions / mesh / osquery-tls / slack-bot /
+ teams-bot, and its lock re-resolves to 0.28.1 so the three now agree and the
+ job tests what fusion ships. All three suites pass on the bounded set
+ (fusion 308, honeytokens 66, purple-team 100, every coverage floor met).
- The version-bump half of the old promise was dropped rather than implemented:
- this repository accumulates under `[Unreleased]` and bumps `VERSION` at
- release-cut, so a per-PR version check would demand something no PR can
- correctly do. The comment now says what the control does.
+ Still unbounded and reported rather than silently fixed: `services/connectors`
+ and `services/threatintel` also declare `httpx = "^0.26.0"`, and neither is
+ in this job.
- Thirteen new tests, all of which fail against the pre-change tree. Four are
- wiring assertions over the workflow YAML itself — that some step passes
- `--allow-breaking`, that the step passing it is guarded by the label and
- presents its evidence, that the unapproved path still blocks, and that
- `labeled` is in the trigger types. A unit-tested function with no caller is
- indistinguishable from a working feature until something asserts the call.
+- **`services/connectors` reached `sqlalchemy.ext.asyncio` without declaring
+ the extra that makes it importable.** `app/db/engine.py` calls
+ `create_async_engine`, and the manifest declared `sqlalchemy = "^2.0.0"`.
+ It resolved only because of the same pre-2.1.0 accident, which means the
+ next SQLAlchemy bump would have dropped `greenlet` from the lock the image
+ installs from and broken the poller in production rather than in CI. It was
+ the only service in this position — every other service that imports the
+ module already declared `[asyncio]`. Re-locking with the extra changes two
+ lines and no resolved version.
-- **Two response verbs worked and could not be reached.** `ack_alert` and
- `suppress_alert` have had Splunk, Elastic and Defender arms since Phase 3.3
- and were wired into `EXECUTOR_REGISTRY` — and appeared in none of the three
- registries that make a verb dispatchable: the live-action adapters, the
- capability contracts, or the capability vocabulary. Governed dispatch
- answered `executor_not_found` for code that ran, which reads as a
- misconfigured integration rather than a capability nobody connected. This is
- the mirror image of a defect already fixed here in the other direction,
- where eleven capabilities had a contract and no executor at all — including
- `unisolate_host`, so the rollback for the most disruptive action in the
- product resolved to nothing.
+- **`scripts/check_dependency_pins.py` now compares extras, not just
+ version ranges.** `sqlalchemy` and `sqlalchemy[asyncio]` are two different
+ dependency sets, and the gate that exists to assert every install path
+ agrees was reading the name and the range and discarding the extras — so it
+ reported agreement between a manifest and a workflow installing strictly
+ less software. Extras are now part of a declaration's identity in both
+ syntaxes (PEP 621 brackets and Poetry's `extras = [...]` table, the latter
+ being the one that was silently dropped), with three directions checked
+ because the direction nobody aims a gate at is the one that rots:
+ `manifest -> install path` (a workflow or image dropping an extra a
+ manifest declares), `source -> manifest` (code importing the module an
+ extra enables under a manifest that does not declare it, read out of the
+ source so the manifest is not asked to vouch for itself), and
+ `extra -> lock` (an extra declared while the lock resolved nothing it
+ provides — "the extra is declared and the library is absent" is now a
+ sentence this gate can say). Run against the pre-fix tree it names all
+ three defects above and the files holding them; `--self-test` injects each
+ direction separately, and six tests in
+ `tests/test_dependency_pin_gate.py` pin the parsing and the directions.
- Both verbs now have six vendor adapters (one per vendor arm), a capability
- contract, an entry in the vocabulary on both sides of the mirror, and a
- verification probe. Each adapter pins `alert_vendor` so a tenant with two
- SIEMs configured does not have the target chosen by credential ordering; the
- pin is still checked against the credentials, so pinning a vendor the tenant
- has not configured simulates rather than claiming an arm that could not have
- run.
+- **`scripts/check_toolchain_pins.py` compares every Node install root
+ against every other.** There are four — the workspace root, `apps/mobile`,
+ `services/realtime` and `services/mcp/cursor-extension` — and the gate read
+ dependency resolution out of the first one only, so the `image-size` split
+ above was invisible to it. It now discovers install roots structurally,
+ reads both pnpm and npm override spellings and both lockfile formats, and
+ checks in both directions: an override declared in one root against what
+ every other root resolved, and each root's own lockfile against its own
+ manifest. Exemptions are keyed on the exact versions they were verified
+ against, so a bump re-opens the question rather than inheriting the
+ clearance. Running it against the pre-fix tree names the defect and the
+ file; running it over a lockfile it cannot parse, or a tree with no
+ install root, fails rather than reporting a clean comparison it never
+ performed. The gate found a third instance of the same class on first run
+ (`ws` 6.2.6/7.5.13 in `apps/mobile`, verified clean against OSV and
+ recorded).
- `ack_alert` is LOW impact and automatic: it marks a finding in-progress and
- owned by AiSOC, removing nothing from anyone's view, and two analysts
- working the same notable is the cost of not doing it. `suppress_alert` is
- LOW impact and **analyst-gated**, because what makes the disposition
- writeback safe to automate is the mapping that refuses to close a confirmed
- true positive, and this verb has no such bound — it closes whatever it is
- pointed at, on the caller's say-so.
+- **CI installed `cryptography>=41,<46` for services whose manifests required
+ `>=46,<51`** — two ranges with no overlap, so the version CI tested could
+ never be the version the image shipped. `connectors` and `osquery-tls` also
+ floored at `44.0.1` for CVE-2024-12797 while CI was free to install `41`.
+ One range, `>=46,<51`, now covers every manifest, Dockerfile and workflow,
+ including `packages/aisoc-cli`, whose Ed25519 plugin signatures the API
+ verifies. `PyJWT` was unbounded in eight workflow install paths and is now
+ `>=2.8,<3` everywhere; `poetry` itself was 1.7.1 in five images, 1.8.2 in
+ three and unpinned in two workflows, and is now `2.4.1` everywhere.
+
+- **`ruff` was declared six different ways while `ruff format --check` gated
+ on one of them.** `services/api` permitted `<0.17.0`, `fusion` `^0.2.0`,
+ the published packages `>=0.3`, and the devcontainer installed it unpinned —
+ while CI enforces `>=0.4.4,<0.5`. A contributor installing their own service's
+ dev group got a ruff that reformats the tree and reds their PR with no
+ dependency change in the diff. All fourteen declarations now read
+ `>=0.4.4,<0.5`. The devcontainer also told contributors to `uv sync` against
+ `services/api/uv.lock`, which has never existed in this tree.
-- **A one-directional gate would have missed all of it.** `check_action_contract.py`
- now compares the four registries a verb needs in **both** directions, and
- `test_capability_reachability.py` injects drift in each direction and asserts
- the gate names it. The dominant failure shape in this repository is a check
- that compares A against B and never B against A, so drift in the direction
- things actually change passes while the check prints OK — the graph-schema
- check reported OK with 17 node labels declared and 28 implemented.
+- The workspace-wide `esbuild` override ban now applies to every install
+ root instead of the repository root alone. `apps/mobile` was the one place
+ the mistake that broke Turbopack's font import map could have been
+ reintroduced without anything noticing, because its bundler is Metro and
+ the damage would surface in a different workspace.
- Running it found three more mismatches beyond the two above. `create_ticket`
- and `notify` were registered, contracted and dispatchable across four
- vendors while absent from `KNOWN_CAPABILITIES` and the connectors
- `Capability` enum, so the registry logged `capability_unknown` for them at
- every startup; both are now in the vocabulary. `chatops_verify` has an
- executor no adapter reaches, and three `ActionType` members
- (`capture_forensics`, `add_ioc_to_blocklist`, `run_playbook`) have no
- executor at all — the first of those is proposed by name by the
- investigation agent on the C2/exfiltration path, so the product recommends
- evidence acquisition it cannot perform. Those four are recorded in the
- gate's exemption lists with the reason each is open. Both lists are
- ratchets: an entry that gains an implementation and is not removed fails the
- build.
+- **Dependabot proposed Expo SDK 57 packages for an SDK 54 app, from an entry
+ that was never meant to see them.** `pnpm-workspace.yaml` excludes
+ `apps/mobile` with a `!apps/mobile` negation and pnpm honours it — the root
+ lockfile has no `apps/mobile` importer. Dependabot's manifest scan does
+ not: it expands `apps/*`, finds `apps/mobile/package.json` and proposes
+ updates for it from the root `/` entry, where none of the Expo holds
+ written into the dedicated `/apps/mobile` entry apply. The result was
+ `expo-constants 18.0.14 -> 57.0.19` — a release-train version, not a
+ package version — carrying no lockfile change at all, since from the root
+ entry's point of view the lockfile is the root one and the package is not
+ in it. `pnpm install` resolves it to `expo-router 6.0.24 unmet peer
+ expo-constants@^18.0.13`, while `tsc --noEmit` stays green, so the mobile
+ job would have passed. Fixed with `exclude-paths: ["apps/mobile/**"]` on
+ the root entry, which is version-updates-only and so leaves that
+ directory's security alerts with the entry that owns them.
-- **The disposition writeback now verifies itself against the vendor.** It
- shipped declaring no verification probe, which was honest — a declared probe
- that does not run is the defect the contract gate exists to catch — but the
- standing rule is that an unverifiable action is not an autonomous one, and
- this action is automatic. The probe re-reads the finding and compares its
- state against the plan re-derived from the same verdict through the same
- `plan_writeback` the executor used, rather than being told separately what
- to expect. Splunk ES reads back the `incident_review` collection (a new
- `SplunkClient.get_notable_event_state`) and confirms status `5` on a close,
- or status `1` *and* the owner AiSOC set on an escalation, since status alone
- cannot distinguish a notable an analyst had already picked up. QRadar
- confirms `CLOSED` through `get_offense`, whose docstring had claimed to be
- the writeback probe since it was written and had no caller.
+- **The root entry's holds are written per dependency name, so a breaking
+ major of anything not already on the list still arrives as an ordinary
+ weekly bump.** `vitest` 4.1.11 → 5.0.1 was the demonstration. Every
+ `ignore` in that entry names a specific package someone had already been
+ burned by — `eslint`, `typescript`, `storybook`, `@storybook/*` — and
+ nothing holds majors as a class, so "stop proposing breaking majors"
+ described four names rather than a rule. Measured on the proposal: all 59
+ test files and all 615 tests passed, then the run failed with 59 unhandled
+ rejections reading `TypeError: Expected string coverage payload, received
+ object` from `V8CoverageProvider.onAfterSuiteRun`, and a coverage report of
+ `All files | 0 | 0 | 0 | 0`. vitest 5 changed the V8 coverage payload
+ shape and `@vitest/coverage-v8` stayed on 4.1.10, because its own manifest
+ range was still satisfied and it peer-requires `vitest` at an *exact*
+ version rather than a range. Separately, the root `pnpm.overrides` pins
+ `@vitest/mocker` to `>=4.1.11 <5` while `[email protected]` depends on
+ `@vitest/[email protected]`, so the override wins silently and the proposal's
+ lockfile carries a vitest 5 runtime with a vitest 4 mocker. Held at the
+ major for `vitest` and `@vitest/*`, and grouped within the major so the
+ exact-version peer pair stops drifting — `pnpm install` on the current
+ lockfile already reports `unmet peer @vitest/[email protected]: found
+ 4.1.10`, which is survivable inside a major and is why nobody had noticed.
- A QRadar **escalation** deliberately reports `unverified`: escalating leaves
- the offense `OPEN`, which is also its prior state, so confirming "still
- OPEN" would certify a write that never happened — the same shape as the
- isolation probe that returned `bool(device_id)` and would have certified an
- uncontained host. Elastic, Sentinel and Defender expose no read of a
- finding's state and report `unverified` too. `ack_alert` and
- `suppress_alert` are verified against the same read-back.
+- **`update-types: ["version-update:semver-major"]` cannot hold a 0.x
+ dependency, so the Storybook hold covered every member of the family
+ except the one that needed it.** For a pre-1.0 package the breaking change
+ arrives in the minor slot, and `@storybook/test-runner` is the only
+ pre-1.0 member: 0.23.0 declares
+ `peerDependencies: { storybook: "^0.0.0-0 || ^8.2.0 || ^9.0.0" }` and
+ 0.24.5 declares `"^0.0.0-0 || ^10.0.0 || … || ^11.0.0-0"`. Its version line
+ is independent of Storybook's the way the Expo packages' are, but its peer
+ tracks the Storybook major exactly, so 0.24 is the Storybook 10 line of
+ the package and installs against this workspace's `[email protected]` as an
+ unsatisfied peer. Nothing in CI could catch it — no workflow runs
+ `test-storybook`, and the only Storybook job runs `build-storybook`, which
+ never loads the runner — so the proposal was green on all 22 required
+ checks while migrating the whole jest tree 29 → 30 underneath it. The
+ minor channel is now held for that package alongside the family's major.
-- **A dry run called the customer's production SIEM.** The live-action dry-run
- path works by stripping credentials so the executor falls through to
- simulation, and the strip list did not match what the client factory reads:
- the Splunk adapters stripped `splunk_host` / `splunk_token` / `splunk_index`
- while `_splunk_client` reads `splunk_url` *first* and also accepts basic
- auth. `splunk_url` is exactly what the credential resolver writes for a
- connector-configured tenant, so a "preview" built a real client and called
- Splunk. Elastic had the identical mismatch (`elastic_host` stripped,
- `elastic_url` read). The strip lists are now the factories' own exported key
- tuples, and a test re-derives each read set from the factory source so the
- two cannot drift again.
+- **The `services/realtime` Dependabot entry let the eslint family arrive one
+ package at a time.** `@eslint/[email protected]` declares
+ `peerDependencies: { eslint: "^10.0.0" }`, so bumping it alone against this
+ service's `eslint@^9` installs a tree `npm ls` reports as `invalid` in nine
+ places — and nothing else notices: `npm ci`, `tsc`, `eslint src test` and
+ the test run are all green on it, and the effective rule set grows by three
+ rules rather than shrinking. Unlike the root workspace this service does
+ not use `eslint-plugin-react`, and `[email protected]` already
+ accepts `eslint@^10`, so the family is grouped rather than held: it can
+ move whenever all of it arrives in one reviewable pull request.
-- **An `alert_vendor` pin was honoured without checking that vendor's
- credentials existed.** `_ack_vendor` returned the pinned vendor
- unconditionally. A dry run strips credentials, so the pin survived the strip,
- resolved to "splunk" with no client, and hit an `assert` — crashing an action
- that was meant to be a harmless preview. A pin naming a vendor the tenant had
- never configured reported a vendor arm that could not run. A pin is now
- checked against the credentials, and an unusable pin resolves to *nothing*
- rather than falling through to whichever other SIEM happens to be configured:
- "write this to Splunk" must not become "write this to Elastic".
+- **The `services/api` pip entry had no holds, against two ratchet gates.**
+ `ruff` is declared in fourteen files (a count
+ `scripts/check_dependency_pins.py` enforces agreement across) and `mypy` in
+ six, plus `.github/workflows/ci.yml` for both, and Dependabot can only edit
+ one of them.
+ Measured rather than assumed: `ruff` 0.16.8 reformats 90 files and reports
+ 3 lint errors where the pinned 0.4.10 reports `All checks passed!` and
+ `1229 files already formatted`; `mypy` 2.3.1 makes
+ `scripts/check_mypy_baseline.py` exit 1 with 35 `(tree, file, code)`
+ entries no longer matching. Both failures land on the required
+ `Python — Lint & Type-check` check, which is to say on every open pull
+ request rather than on the bump's own. Held at the majors (and, for `ruff`,
+ the minors, since 0.4 → 0.16 is a minor step under semver while 0.4.x
+ patches still flow). `sqlglot` is deliberately *not* held: an `ignore` rule
+ suppresses security updates as well as version updates, and a single-path
+ sqlglot bump already cannot merge because `scripts/check_sqlglot_pin.py`
+ fails closed when the seven install paths disagree.
-- **`CreateNotableEventExecutor` raised `TypeError` on every live call.** It
- passed `title=` / `description=` / `fields=` to a client whose signature is
- `(rule_name, event_data, severity, owner, status)`. Invisible because
- simulation mode never constructs the client — the same defect class as the
- `max_results` / `max_count` drift fixed earlier on this module. Every SIEM
- executor-to-client call is now pinned by an autospec'd signature test, which
- a hand-written fake with `**kwargs` could never have caught.
+- `.github/dependabot.yml`'s `apps/mobile` entry gains the `typescript`
+ semver-major hold that the `/` and `/services/realtime` entries already
+ carry. Its CI gate is `tsc --noEmit`, so it was the only Node install root
+ running `tsc` without the hold — the same shape as the `image-size`
+ finding, a decision taken for the workspace that never reached the root
+ installing separately.
-- **The lake's tenant isolation could be switched off by resolving a
- dependency one major version higher.** `lake_sql.rewrite_for_tenant` is the
- only thing separating one tenant's events from another's in ClickHouse: it
- parses untrusted operator SQL with sqlglot, enforces the table allowlist
- against the parse tree, bans ClickHouse table functions, and injects the
- `tenant_id` predicate. sqlglot 27 moved the SELECT's FROM clause from
- `args["from"]` to `args["from_"]`. The table walk read the old key, got
- nothing, and took the branch written for `SELECT 1` — "no FROM, so no tenant
- data, nothing to do". Every single-table query then came back with no
- allowlist check, no table-function ban and no tenant predicate, reported as
- a successful rewrite.
+- **`_SECRET_PATTERNS` was declared with a type its value cannot have, which
+ switched off checking of the secret-masking loop.** The annotation in the AI
+ SDK's redaction module read
+ `tuple[tuple[str, re.Pattern[str]], None | str] | tuple`, whose first member
+ is a two-element tuple — structurally impossible for the eight-pair value, so
+ it only ever matched through the bare `| tuple`. That erases the element type
+ to `Any`, which is why nothing objected to calling `.subn` on something typed
+ as possibly a `str`. In a path whose job is to stop secrets leaving the
+ process, a checker that has been silently switched off is worse than none.
+ Declared `tuple[tuple[str, re.Pattern[str]], ...]`, which is what it is.
- `services/api/pyproject.toml` declared `sqlglot >=23.0.0,<31.0.0` while the
- Dockerfile and every CI workflow declared `>=23,<27`, so this was reachable
- by installing the service exactly as declared, and CI could not see it
- because CI installed the narrow range. Verified on sqlglot 30.19.0 against
- the pre-fix rewriter: `SELECT user_name FROM aisoc.raw_events` returned
- unscoped, `SELECT * FROM system.tables` was accepted, and
- `url('https://attacker.example/x', JSONEachRow)` was accepted — the last of
- which makes the warehouse issue outbound HTTP with its own network identity.
+- **Five LLM input-contract handlers raised `TypeError` instead of degrading.**
+ `/translation`, `/phishing`, `/knowledge-base`, `/hunts` and `/detection-loop`
+ each catch `LLMContractViolation` — the untrusted-input boundary refusing a
+ prompt — and logged it as `logger.warning("<event>", reason=exc.reason)`.
+ `logger` is `logging.getLogger`, not structlog, and the stdlib `Logger`
+ rejects an unknown keyword with a `TypeError`. An exception raised inside an
+ `except` block is not caught by a sibling handler, so the `except Exception`
+ sitting directly beneath it never saw it: the one path whose job is to
+ degrade gracefully was the only path that raised. `phishing.py` carries a
+ comment explaining that the log line was added *because* the handler used to
+ swallow everything — the fix for the silent failure was itself throwing. Only
+ six of thirteen trees declare `[tool.mypy]`, so an AST scan swept the rest of
+ `services/` and `packages/`; those five were the only instances repo-wide.
- Three changes rather than one, because pinning alone would leave the trap
- armed for the next bump. The FROM clause now resolves by node type instead
- of by key name. `rewrite_for_tenant` ends with an audit that takes its own
- independent census of the statement's tables and raises the new
- `LakeSqlIsolationError` unless every one of them was scoped, the tenant
- survived into the rendered string, and every rendered SELECT that reads a
- lake table carries the tenant in its own WHERE — so a partially scoped
- UNION fails too. And all seven install paths now declare one identical
- range, enforced by `scripts/check_sqlglot_pin.py`, with
- `.github/workflows/lake-isolation.yml` running the rewriter suites against
- both the shipped range and the next major so a future bump fails loudly
- instead of quietly downgrading isolation.
+- **Six guards inspected a different call's result from the one they
+ guarded.** `x.get(k) if isinstance(x.get(k), dict) else {}` calls `get`
+ twice; it is safe for a plain dict and that is not a property anything
+ enforces, which is what the twenty `union-attr` findings underneath it were
+ reporting. Fetched once and then tested, in `playbook_step_dispatch`,
+ `siem_writeback` and `sla`.
-- **The alerts list was empty on every deployment, under a row count that was
- real.** `AlertListResponse` returns the rows under `items`; the web client
- read `raw.alerts`, which is never present, so `Array.isArray(undefined)` was
- false and each page resolved to `[]` while `total` carried the true figure.
- The queue therefore rendered "1,247 alerts" above an empty table with no
- error to explain it, and an operator's most reasonable reading of that screen
- was that their estate was quiet. The client now reads `items`, still accepts
- the legacy `alerts` key the responder routes emit, and a test asserts the two
- halves of the contract against each other so they cannot drift apart again.
+ Together with a `_compute_durations` return annotation that claimed
+ `dict[str, int | None]` while returning a `str` severity, these take the mypy
+ ratchet from 690 to 656. `packages/sdk-py` declared `[tool.mypy]` with no
+ `python_version`, so it was type-checked against whichever interpreter the
+ job ran and its share of the baseline moved with CI rather than with the
+ code; it is pinned to 3.11 like the other five, and the toolchain gate now
+ fails on a tree that asks to be type-checked without saying against which
+ Python. What remains on the ratchet is annotation hygiene and artefacts of
+ the deliberate no-dependencies environment the baseline is recorded in — the
+ 19 surviving `union-attr` are all `mock.call_args` in tests.
-- **Two dashboards published fabricated security data as tenant state.**
- `DashboardView` and `SOCMetricsDashboard` both wrapped their SWR
- `fallbackData` in `demoFallback()`, which is `undefined` outside the hosted
- demo — and both then defeated that gate a few lines later with an
- unconditional `const resolved = isValid ? data : MOCK`. The mock was
- therefore exactly what rendered during first paint and after any API error,
- which for a self-hoster with an empty or unreachable backend is the whole
- session.
+- **A cancelled context-graph walk crashed the context bundle.**
+ `_fetch_neighborhoods` and `_fetch_ueba_baselines` filter
+ `asyncio.gather(..., return_exceptions=True)` results with
+ `isinstance(r, Exception)`, then unpack the survivors as a tuple.
+ `asyncio.CancelledError` is a `BaseException` and not an `Exception` on 3.8+,
+ so a cancelled child task passed the filter and reached the unpack as
+ `TypeError: cannot unpack non-sequence CancelledError`. `services/slack-bot`
+ carries a comment spelling out this exact trap; these two sites got it wrong.
+ Both now filter on `BaseException`.
- What that put on screen as the reader's own numbers: a connector inventory
- they do not run (`CrowdStrike EDR`, 412 events), a MITRE tactic ranking, a
- 24-hour alert-volume curve, MTTD 1.4h / MTTR 6.2h, and an LLM spend line of
- $76.65 naming three models they had never configured. Three "vs yesterday"
- trend deltas sat beside the real alert counts as literals — `/metrics/dashboard`
- publishes no period-over-period comparison, so there was nothing to derive
- them from.
+- **A Google Workspace key that was valid JSON but not an object failed inside
+ the JWT signing path.** `json.loads` returns a `str`, `list` or `int` for a
+ document that is valid JSON and not an object, and the constructor accepted
+ it; the failure surfaced later as `TypeError: string indices must be integers`
+ at `self._key["client_email"]`, at the moment an operator triggered a live
+ action. The configuration is wrong either way — it now says so when the
+ credential is saved rather than when it is used.
- Every panel now renders one of three honest states: real figures, an empty
- state naming what would populate it, or an error state carrying the failure
- and a retry that re-issues the request. Sample data still populates the
- hosted demo, which is the only reason it exists.
+- **`DEFAULT_SLA_TARGETS` was declared twice with disagreeing bodies, and
+ neither matched the migration.** The second definition won by being later in
+ the file and put `info` at `(480, 1440, 2880)`; the first said
+ `(240, 960, 2880)`; migration 040 seeds `(240, 1440, 4320)`. A tenant's
+ info-tier deadline therefore depended on whether it had a seeded row
+ (240 min) or fell back to Python (480 min) — and `alert_queue` reads exactly
+ this row as the `sla_due_at` catch-all for severities outside the four-tier
+ ladder. One definition now, matching the migration, which is what is in the
+ database.
-- **The lake's tenant isolation could be switched off by resolving a
- dependency one major version higher.** `lake_sql.rewrite_for_tenant` is the
- only thing separating one tenant's events from another's in ClickHouse: it
- parses untrusted operator SQL with sqlglot, enforces the table allowlist
- against the parse tree, bans ClickHouse table functions, and injects the
- `tenant_id` predicate. sqlglot 27 moved the SELECT's FROM clause from
- `args["from"]` to `args["from_"]`. The table walk read the old key, got
- nothing, and took the branch written for `SELECT 1` — "no FROM, so no tenant
- data, nothing to do". Every single-table query then came back with no
- allowlist check, no table-function ban and no tenant predicate, reported as
- a successful rewrite.
+- **`diff_fingerprints` was declared to return `dict[str, list[str]]` and never
+ has.** It returns `unchanged_count` as an `int`, which the connector
+ scheduler stores and a test asserts on. Because the scheduler assigns the
+ result straight into a `dict[str, Any]` variable, the wrong declaration
+ narrowed that variable and made the three correct lines underneath it look
+ like type errors instead — one wrong annotation producing four findings across
+ two modules. Declared `dict[str, Any]`, which is what both consumers and the
+ JSONB column already expect.
- `services/api/pyproject.toml` declared `sqlglot >=23.0.0,<31.0.0` while the
- Dockerfile and every CI workflow declared `>=23,<27`, so this was reachable
- by installing the service exactly as declared, and CI could not see it
- because CI installed the narrow range. Verified on sqlglot 30.19.0 against
- the pre-fix rewriter: `SELECT user_name FROM aisoc.raw_events` returned
- unscoped, `SELECT * FROM system.tables` was accepted, and
- `url('https://attacker.example/x', JSONEachRow)` was accepted — the last of
- which makes the warehouse issue outbound HTTP with its own network identity.
+- **`posture_loader` returned `None` from a function declared to return a
+ dict** when a 200 response carried no `config` key, while the non-200 branch
+ immediately above already returned `{}` for the same "nothing to load"
+ outcome.
- Three changes rather than one, because pinning alone would leave the trap
- armed for the next bump. The FROM clause now resolves by node type instead
- of by key name. `rewrite_for_tenant` ends with an audit that takes its own
- independent census of the statement's tables and raises the new
- `LakeSqlIsolationError` unless every one of them was scoped, the tenant
- survived into the rendered string, and every rendered SELECT that reads a
- lake table carries the tenant in its own WHERE — so a partially scoped
- UNION fails too. And all seven install paths now declare one identical
- range, enforced by `scripts/check_sqlglot_pin.py`, with
- `.github/workflows/lake-isolation.yml` running the rewriter suites against
- both the shipped range and the next major so a future bump fails loudly
- instead of quietly downgrading isolation.
+- **A load generator that could tick forever and send nothing.**
+ `services/demo-producer` silently `continue`d when `http.NewRequestWithContext`
+ failed, which only happens for a malformed URL — the one error in that
+ loop no retry can clear. It now says so and stops.
-- **Every entity chip in the Investigation Rail was a 404.**
- `alert_rail.py` built its pivots as `/attack-graph?entity=…` and there has
- never been an `attack-graph` route, so host, user, asset, IP and domain chips
- all failed before the query parameter mattered. The rail's own docstring
- described the working behaviour ("the same `?entity=` query the
- AttackGraphView already parses"), and `pivot.ts` states as fact that the rail
- emits `/graph?entity=…`; the code agreed with neither. Pivots now target
- `/graph`, which reads the parameter and selects the node.
+### Removed
- The existing test pinned all six strings under a docstring claiming the pin
- "prevents an accidental rename". It cannot — it compares the producer against
- a copy of itself, which is how a route that never existed stayed asserted.
- `services/api/tests/test_pivot_routes_resolve.py` derives the route table
- from `apps/web/src/app` and checks it against what `build_related_entities`
- actually returns, so the comparison now runs in the direction that drifts.
+- **The identity timeline's phantom second source.** It queried a table named
+ `aisoc_events` that no migration creates, nothing writes to, and that appears
+ nowhere else in the repository — with its failure swallowed at `DEBUG`. It
+ read as a second source of evidence while returning nothing on every
+ deployment. The remaining source now reports itself on the response
+ (`sources_unavailable`) when it cannot be read, so an empty timeline caused
+ by a broken query is distinguishable from one caused by no matches.
- The same test caught a second defect in the producer: values went into the
- URL unencoded, so an asset named `Finance & Legal #2` pivoted to
- `Finance & Legal` and looked like it had worked. Values are now encoded the
- way `pivot.ts` encodes them.
+- **The Chronicle warehouse scaffold.** It read `hunt.translated_query["udm"]`,
+ nothing in the repository emits UDM, and it gated on two settings that were
+ never fields — so it could not be selected and could not run, while
+ `available_providers()` reported it as a supported warehouse. Adding a real
+ one is a `register_provider` call plus a UDM translator.
-- **Six funnel metrics were published as zero whatever the database held.**
- `_funnel_window` merges `_triage_quality`'s output into the dict it returns,
- but the `FunnelMetrics(...)` call never named `triaged_alerts`,
- `abstentions`, `abstention_rate`, `ungrounded_demotions`,
- `mean_groundedness` or `scored_verdicts`, so all six fell back to their
- field defaults on every response. A published zero is a stronger claim than
- silence: it reads as "this tenant never abstained, and nothing was ever
- demoted for being ungrounded". The existing test asserted the six names were
- present in `model_fields`, which they were — declaring a field and forwarding
- it are different things. The replacement drives the endpoint and fails for
- any field the window computes and the response drops, including ones added
- later.
+### Security
-- **The Live Feed labelled an empty panel "Demo".** The seeded events were
- gated behind `canUseDemoData()` in an earlier pass but `statusToLabel` was
- not, so outside the hosted demo the panel rendered nothing at all under a
- "Demo" pill whose tooltip read "showing demo data" — asserting the presence
- of sample data that had just been correctly withheld. The pill now describes
- what is on screen (`Live`, `Connected`, `Connecting…`, `Reconnecting…`,
- `Offline`) and can only say `Demo` when seeded events are actually rendered,
- and the idle panel carries an empty state naming what would fill it.
+- **Fifty-eight routes across four services carried no authentication at all,
+ and the previous gate could not see them.** `check_route_tenant_scope.py`
+ asks a conditional question — *if* a route takes a tenant, where did the
+ tenant come from — so a route that takes no tenant was never in its reach.
+ 37 `services/agents` routes took none. Reproduced against the real routers
+ with credential material configured, so the result is not "the service was
+ unconfigured": an anonymous caller with no `Authorization` header created a
+ playbook (201), listed all 64 (200), **executed one** (202) and deleted it
+ (204), then read copilot conversations and ran a threat hunt. All seven
+ refuse with 401 now, while a valid console session still gets a non-empty
+ response.
-- **The welcome banner quoted counts it had no source for and linked to a case
- most deployments do not have.** It advertised "26 vendors" against a registry
- of 84 and "25 named runbooks" with nothing holding either to the tree. The
- connector figure now comes from `CONNECTOR_COUNT`, which is generated from
- the connector registry and held to it by `scripts/generate_connector_count.py
- --check`; the playbook figure is gone rather than guessed, because no
- equivalent source exists. Its second call to action linked to
- `/cases/INC-RT-001`, which exists only after the demo seed has run, so on
- every other deployment the banner's own CTA was a dead link — that tip is now
- gated on demo mode, takes its href from `demoDeeplink()` and says on its face
- that it is sample data.
+ `services/agents` keeps **dual-mode** auth rather than a bearer-only
+ scheme, because the console reaches it directly through a Next rewrite on a
+ session cookie: the guard is #813's `require_console_or_service_auth`,
+ extended rather than replaced. Its WebSocket could not use the same
+ dependency — a browser cannot set an `Authorization` header on a handshake,
+ and an `HTTPException` has no defined rendering on a WebSocket scope — so
+ `_ws_principal` reads the credential from the header or `?token=`, verifies
+ it with the *same* vendored logic, and closes with 1008 before `accept()`.
+ That route previously took its tenant from a query parameter defaulting to
+ the literal `"default"`, which names no tenant anywhere in this schema, and
+ would start a fresh investigation on the connection.
-- **`SavedViewsBar` updated its parent while rendering.** The auto-apply of a
- default saved view ran in the render body and called `onApply`, which for
- every caller is a setState on the page component. React rejects that outright
- and makes no promise about processing the update, so the filters an analyst
- expected restored were not reliably applied. The comment described a ref-flag
- that did not exist; it was `useState` with no effect anywhere. The chip
- highlight is now derived rather than stored, and the one genuine side effect
- — handing the filters to the page — happens in an effect. The existing
- "exactly once" test could not catch this because its `onApply` was a bare
- spy that set no state; the new test uses a real parent.
+ Also closed: `connectors` (9 data routes, including the one that decrypts a
+ saved instance's credentials to test them and the two that push into a
+ customer's ITSM), `fusion` (5), `osquery-tls` (7), `threatintel` (1),
+ `actions` (1) and 20 in `services/api` — among them `PUT /deployment/config`,
+ `POST /deployment/airgap/bundle`, `POST /compliance/evidence/collect`, both
+ LLM-backed `/translate` routes and the seven STIX/TAXII routes.
-- **The alerts list was empty on every deployment, under a row count that was
- real.** `AlertListResponse` returns the rows under `items`; the web client
- read `raw.alerts`, which is never present, so `Array.isArray(undefined)` was
- false and each page resolved to `[]` while `total` carried the true figure.
- The queue therefore rendered "1,247 alerts" above an empty table with no
- error to explain it, and an operator's most reasonable reading of that screen
- was that their estate was quiet. The client now reads `items`, still accepts
- the legacy `alerts` key the responder routes emit, and a test asserts the two
- halves of the contract against each other so they cannot drift apart again.
+ Two counts in the original report were **wrong, in the safe direction**, and
+ the reason matters: `slack-bot` (5) and `teams-bot` (4) were never open. An
+ AST pass sees no `Depends` and calls a route unauthenticated, but Slack Bolt
+ verifies a request signature, `/approval-card` compares a shared internal
+ token in constant time, and the Teams webhook verifies an HMAC-signed card
+ payload with a replay window. `scripts/check_route_auth.py` models these as
+ **conditional** exemptions that name the verifier and lapse the moment the
+ handler stops calling it.
-- **Two dashboards published fabricated security data as tenant state.**
- `DashboardView` and `SOCMetricsDashboard` both wrapped their SWR
- `fallbackData` in `demoFallback()`, which is `undefined` outside the hosted
- demo — and both then defeated that gate a few lines later with an
- unconditional `const resolved = isValid ? data : MOCK`. The mock was
- therefore exactly what rendered during first paint and after any API error,
- which for a self-hoster with an empty or unreachable backend is the whole
- session.
+- **Thirty routes across six services let the caller name the tenant they were
+ reading.** `/fusion/entity-risk/*` was the reported instance and the worst
+ one: three routes on the API gateway and three on the fusion service took
+ `tenant_id` as a query parameter with no auth dependency at all. The console
+ reaches fusion *directly* through a Next rewrite when `FUSION_URL` is set, so
+ those routes were reachable from any browser on the internet, and an
+ anonymous request naming another tenant's UUID returned that tenant's
+ entity-risk queue, stats and per-entity detail. The engine's own docstring
+ asserted the missing control — "the tenant_id is part of the key prefix and
+ the API service re-checks tenant on read" — and the second half was not true
+ on either side. Key prefixing isolates whichever tenant it is handed;
+ validating the parameter's *value* fixes nothing, because a UUID that parses
+ is still a UUID the caller chose.
- What that put on screen as the reader's own numbers: a connector inventory
- they do not run (`CrowdStrike EDR`, 412 events), a MITRE tactic ranking, a
- 24-hour alert-volume curve, MTTD 1.4h / MTTR 6.2h, and an LLM spend line of
- $76.65 naming three models they had never configured. Three "vs yesterday"
- trend deltas sat beside the real alert counts as literals — `/metrics/dashboard`
- publishes no period-over-period comparison, so there was nothing to derive
- them from.
+ An AST pass over all 600 routes in `services/` found the same shape in five
+ more places, in two flavours. Taking a tenant with no authentication:
+ `services/agents` `/triage/{run_id}`, `/cases/{id}/triage`,
+ `/cases/{id}/investigate`, `/investigations` and `/explain`; and six
+ `services/osquery-tls` routes. Taking one *with* authentication but never
+ intersecting it with the caller's scope: `honeytokens`, `purple-team` and
+ `ueba`, where a router-level service token proved the caller was a trusted
+ service but said nothing about which tenant it was acting for.
- Every panel now renders one of three honest states: real figures, an empty
- state naming what would populate it, or an error state carrying the failure
- and a retry that re-issues the request. Sample data still populates the
- hosted demo, which is the only reason it exists.
+ The tenant now comes from the credential. A new
+ `app/security/tenant_scope.py`, vendored into the six services that need it,
+ resolves either a console session (the first-party HS256 access token, whose
+ verified `tenant_id` claim is authoritative) or a trusted service declaring
+ the tenant it acts for on `X-AiSOC-Tenant-ID`. A `tenant_id` on the request
+ survives only as a *filter*, intersected with that scope, so an MSSP
+ operator can still narrow to one managed customer while naming an outside
+ tenant narrows to nothing and returns 403. A service token that declares no
+ tenant resolves to an empty scope and is refused: absent is never all, which
+ is the shape every cross-tenant leak in this codebase has had. The HS256
+ verification is stdlib-only and mirrors `services/realtime/src/auth.ts`,
+ rejecting `alg: none`, a refresh token presented for access, and an expired
+ or wrongly-signed token.
+
+ Closing these surfaced a worse variant the parameter audit could not see:
+ eight routes matched on an id with **no tenant predicate at all**.
+ `honeytokens` `GET/PATCH/DELETE /{token_id}` and `/{token_id}/triggers`,
+ `purple-team` `PATCH /executions/{id}/detection` and the three
+ `/tabletop/{session_id}` routes, and `ueba`
+ `PATCH /anomalies/{id}/acknowledge`. Any caller could read, revoke or delete
+ another tenant's honeytoken, overwrite another tenant's detection outcome,
+ or acknowledge away another tenant's anomaly by naming its UUID. All eight
+ now filter on the caller's tenant as well as the id, so a foreign row is a
+ 404 — which is what it is, from that caller's point of view.
-- **The lake's tenant isolation could be switched off by resolving a
- dependency one major version higher.** `lake_sql.rewrite_for_tenant` is the
- only thing separating one tenant's events from another's in ClickHouse: it
- parses untrusted operator SQL with sqlglot, enforces the table allowlist
- against the parse tree, bans ClickHouse table functions, and injects the
- `tenant_id` predicate. sqlglot 27 moved the SELECT's FROM clause from
- `args["from"]` to `args["from_"]`. The table walk read the old key, got
- nothing, and took the branch written for `SELECT 1` — "no FROM, so no tenant
- data, nothing to do". Every single-table query then came back with no
- allowlist check, no table-function ban and no tenant predicate, reported as
- a successful rewrite.
+- **Any authenticated user could disable detection rules inside any other
+ tenant.** `_ensure_mssp_parent`, the guard on the MSSP write surface, had
+ `pass` for a body. Four routes took a caller-supplied child tenant id and
+ wrote it onto a row without checking whose child it was.
- `services/api/pyproject.toml` declared `sqlglot >=23.0.0,<31.0.0` while the
- Dockerfile and every CI workflow declared `>=23,<27`, so this was reachable
- by installing the service exactly as declared, and CI could not see it
- because CI installed the narrow range. Verified on sqlglot 30.19.0 against
- the pre-fix rewriter: `SELECT user_name FROM aisoc.raw_events` returned
- unscoped, `SELECT * FROM system.tables` was accepted, and
- `url('https://attacker.example/x', JSONEachRow)` was accepted — the last of
- which makes the warehouse issue outbound HTTP with its own network identity.
+ The consequential one was `POST /api/v1/mssp/overrides`. An override with
+ `action: "exclude"` is read back by `resolve_effective_rules`, filtered on
+ `child_tenant_id == <the reader's tenant>`, and the rule is popped out of the
+ set `POST /api/v1/rules/hunt` runs. So naming another tenant's id silently
+ deleted a named detection from their hunts, and the victim's only symptom was
+ a hunt that stopped matching. Reproduced against the previous commit: a
+ tenant's effective ruleset went from one critical cloud rule to zero on an
+ override written by an unrelated tenant.
- Three changes rather than one, because pinning alone would leave the trap
- armed for the next bump. The FROM clause now resolves by node type instead
- of by key name. `rewrite_for_tenant` ends with an audit that takes its own
- independent census of the statement's tables and raises the new
- `LakeSqlIsolationError` unless every one of them was scoped, the tenant
- survived into the rendered string, and every rendered SELECT that reads a
- lake table carries the tenant in its own WHERE — so a partially scoped
- UNION fails too. And all seven install paths now declare one identical
- range, enforced by `scripts/check_sqlglot_pin.py`, with
- `.github/workflows/lake-isolation.yml` running the rewriter suites against
- both the shipped range and the next major so a future bump fails loudly
- instead of quietly downgrading isolation.
+ Closing those four routes alone would not have been enough, because
+ `POST /api/v1/mssp/children/{id}/onboard` let anyone *become* the parent
+ first — its only check was a `409` when the target already had a parent, so
+ every standalone tenant on a deployment was adoptable by any authenticated
+ user. Adoption now requires the child to have invited that specific parent by
+ setting `settings.mssp_parent_invite` through `PATCH /api/v1/tenants/me/settings`,
+ which only ever writes the caller's own row and is gated on `settings:write`.
+ The invite is single-use. The child-scoped routes answer `404` rather than
+ `403` for a tenant that is not yours, so they cannot enumerate tenant UUIDs.
-- **Scheduled hunts ran against credentials that could not exist, so every one
- of them returned zero hits.** The event-warehouse drivers resolved their
- endpoint and secret from `settings.ES_URL` / `ES_API_KEY` / `SPLUNK_URL` /
- `SPLUNK_HMAC_TOKEN` / `CHRONICLE_PROJECT_ID`. None of those were declared
- fields on `Settings`; each was read through `getattr(settings, name, None)`,
- so the miss was silent, and `Settings` sets `extra="ignore"`, so an operator
- who followed the resulting "set them in environment variables" message and
- exported `ES_URL` got the same message back. The scheduler treats the
- resulting `HuntNotConfigured` as a soft skip, so the failure was invisible.
+- **`/api/v1/identity-timeline` read every tenant's alerts.** Both routes bound
+ an authenticated user and never used it: the SQL against `aisoc_alerts`
+ carried no `tenant_id` predicate, so any authenticated caller could pull any
+ tenant's alerts whose title or evidence matched a substring — and the
+ substring is the search term, so the match is caller-controlled. Both routes
+ are now scoped to the caller's tenant.
- Credentials now resolve from the tenant's own `connectors` row — the one the
- console wizard writes, encrypted by the credential vault — which is where
- `federated.py` and `case_fanout.py` already read theirs. The warehouse is
- per-tenant by construction as a result: a managed provider can point each
- customer at their own cluster, which resolving from process settings made
- impossible even in principle. `ES_URL` / `ES_API_KEY` are now declared
- fields and remain as a deployment-wide fallback for single-cluster installs.
+- **`/api/v1/playbooks` had no authentication at all.** The module declared no
+ `Depends` of any kind across eight routes and there is no global auth
+ middleware, so every route was reachable unauthenticated — including
+ `POST /playbooks/{id}/run`, which executes a playbook against the estate.
+ Each route now demands `playbooks:read`, `playbooks:write` or
+ `playbooks:execute`; all three permissions already existed in
+ `ROLE_PERMISSIONS` and had no reader. `:execute` stays distinct from
+ `:write` so an analyst can run a governed playbook without editing one.
-- **Provider selection ignored which SIEM the tenant had connected.**
- `resolve_provider` returned the first driver whose `translated_query_key`
- appeared in the hunt, and the natural-language translator emits ES|QL, SPL
- *and* KQL for every question — so `esql` was always present and
- Elasticsearch was always chosen, including for tenants who run only Splunk.
- Selection is now driven by the tenant's enabled connectors first and the
- available translation second.
+- **`scripts/check_tenant_query_predicates.py` — the predicate question.**
+ Closing the parameter shape surfaced eight routes matching on an id with no
+ tenant predicate; the sweep that found them was opportunistic. This gate
+ asks the question structurally over the whole tree, and the model and table
+ inventories are **derived from the tree** (a model's `tenant_id` column, the
+ migrations' DDL) rather than listed, so a new migration cannot slip past.
+ It matters most where nothing stands behind it: of 95 tenant-scoped tables
+ only 31 carry an RLS policy, and RLS engages only on a session that ran
+ `SET LOCAL app.current_tenant_id`.
-- **`POST /nl-query/execute` documented two request fields it ignored.**
- `es_url` and `es_api_key` were described as overrides and silently dropped.
- `es_url` now selects among the caller's own Elasticsearch connectors by
- host, matched against connectors they own and never used as an outbound
- target; `es_api_key` is refused with 400, because credentials belong in the
- vault-encrypted connector rather than in a request body.
+ Real leaks it found beyond the known eight:
-### Added
+ - `GET /graph/attack-path/{case_id}`'s **relational fallback** read
+ `aisoc_cases` by id with no tenant predicate. The primary Neo4j path is
+ scoped, but the fallback runs precisely when that path failed — and its
+ own docstring says it exists so deployments without a graph database keep
+ working, i.e. permanently for many of them. An authenticated caller naming
+ another tenant's case UUID got its title, severity, MITRE techniques and
+ alert ids.
+ - `GET /osquery/distributed/{query_id}` was **both** unauthenticated and
+ unscoped. `osquery_distributed_query` has no `tenant_id` of its own — it
+ is reached through its node — so the lookup now joins onto
+ `osquery_node.tenant_id`. Demonstrated with two seeded tenants: before,
+ tenant B naming tenant A's `query_id` got A's host telemetry back; after,
+ nothing.
+ - `alert_explain._resolve_rule_lineage` selected a detection rule by an id
+ lifted out of the alert's `raw_event` — vendor-supplied, so a crafted
+ event could name another tenant's rule and have its definition explained
+ back.
+ - The ITSM webhook inserted its system comment into `aisoc_case_comments`
+ **without `tenant_id` at all**, leaving rows belonging to no tenant since
+ migration 044 added the column.
+ - Thirteen by-id writes (`alerts` escalate/snooze/update, four `connectors`,
+ three detection-rule routes, `claim_alert`, `run_saved_hunt`) were scoped
+ only by a preceding read. Not exploitable as written, and one reorder from
+ being scoped by nothing.
-- **Two-way SIEM integration: an AiSOC verdict is written back onto the
- finding that produced the alert.** The integration only ever ran inbound. A
- Splunk notable became an alert, the agent triaged it, and the notable sat in
- the Splunk queue untouched — so an analyst re-read a finding AiSOC had
- already dismissed, and a finding AiSOC had confirmed waited its turn behind
- them. Nothing on `main` wrote a disposition back to any SIEM in any form.
+ What cannot be decided statically sits on a **shrink-only ratchet** — 34
+ entries, each with a reason, `MAX_RATCHET` asserted against the table's
+ length, and an entry whose statement is now scoped failing as *stale*. That
+ last property caught two of this change's own edits.
- New capability `update_alert_disposition` with five vendor arms (Splunk ES,
- Elastic Security, Microsoft Sentinel, IBM QRadar, Microsoft Defender), two
- new clients (`sentinel_client.py`, `qradar_client.py`), migration
- `057_alert_source_links.sql`, and `POST /api/v1/alerts/{id}/source-writeback`.
- The agents triage worker posts to that route after the verdict is durable and
- fails soft — the verdict outranks its writeback, and an unreachable Splunk
- must not dead-letter an alert that was triaged correctly. Docs:
- [Integrations → SIEM writeback](apps/docs/docs/integrations/siem-writeback.md).
+- **Row-level security covered 31 of 95 tenant-scoped tables; it now covers 92,
+ and the seven policies that already existed but could never work are
+ repaired.** On the other 64 tables the query predicate was the only thing
+ between two customers, so one missing `WHERE tenant_id` was a leak rather
+ than something a second layer caught — which is not the design the
+ repository documents. `060_rls_coverage.sql` adds a policy to every
+ remaining table in the API chain, and the four services that manage their
+ own schema (honeytokens, osquery-tls, purple-team, ueba) each carry a
+ matching alembic revision. The three that remain are named rather than
+ rounded away: `users` is excluded so authentication can resolve a principal
+ before a tenant exists, and `case_tasks` / `case_timeline` are ORM models
+ that no migration creates.
- **The disposition mapping is the safety argument, not the approval tier.** A
- confirmed true positive is *escalated and never closed*: it is the finding a
- human most needs to see, and closing it because the platform is confident is
- how an agent turns a real intrusion into a resolved ticket nobody read. An
- unknown verdict is *refused, never guessed* — which matters because
- `normalize_disposition` defaults an unrecognised string to `true_positive`,
- so a mapper that normalised first would convert "I do not recognise this"
- into a confident claim. Only benign and false-positive verdicts may close a
- finding, and `benign_true_positive` is classified as a correct detection of
- authorised activity rather than as a false positive, so it never inflates a
- rule's own FP rate.
+ Four things found along the way, each invisible for the same reason:
- **Governance ships dry-run by default.** `AISOC_SIEM_WRITEBACK_ENABLED`
- defaults on and `AISOC_SIEM_WRITEBACK_EXECUTE` defaults **off**, so an
- operator opts in to writing into their own SIEM. Anything that is not an
- explicit yes — including a typo — is read as a dry run. `executed` is the
- single field that means a vendor was touched; it is carried on the API
- response, on the worker's return value, and as a no-default column on
- `alert_source_links`, so a dry run, a refusal and a credential-less
- simulation can never be read as a write that happened. Projecting a closing
- verdict onto a linked Jira / ServiceNow ticket needs a third flag
- (`AISOC_SIEM_WRITEBACK_CLOSE_CASE`, off) because the ITSM connectors project
- a status transition rather than a note, so the only truthful way to reach the
- ticket is to resolve the case for real.
+ - **The application bypasses RLS entirely.** `docker-compose.yml` and the CI
+ service containers run every service as `POSTGRES_USER=aisoc`, which the
+ postgres image creates as a superuser, and a superuser ignores policies
+ even under `FORCE ROW LEVEL SECURITY`. Measured, not inferred: with two
+ alerts seeded one per tenant and the session bound to tenant A, that role
+ sees both and a `NOSUPERUSER NOBYPASSRLS` role sees one. The security doc
+ claimed the opposite — "there is no superuser escape hatch via the
+ application's DB role" — and now carries the grant that makes it true.
+ - **Seven policies were keyed on a session variable nothing sets.**
+ `alert_sla_events` and `tenant_sla_config` read `app.tenant_id`;
+ `custom_parsers` and `retention_policies` read `app.current_tenant`;
+ `compliance_evidence` had no unset-context arm. All five returned zero
+ rows once RLS engaged. `external_assets` and `external_asset_drift` called
+ `current_setting` without `missing_ok`, so an unbound session raised
+ `unrecognized configuration parameter` instead. All seven are normalised
+ to the canonical predicate.
+ - **The agents service had the mirror-image bug.** Its four
+ `_set_rls_context` helpers wrote `app.tenant_id` while every policy reads
+ `app.current_tenant_id`, so the scoping they exist to provide had never
+ been applied — the policies fell through their fail-open arm every time.
+ - **Five tables had RLS enabled without `FORCE`**, so the table owner —
+ which is the application — walked past the policy regardless.
-- **The join key survives ingest.** `finding.uid` — the vendor's own id for a
- finding — reached the OCSF envelope and was then discarded, so by the time a
- row hit `alerts` the notable's rule UID, the Elastic signal id and the QRadar
- offense id were gone and no verdict could be aimed at anything. Fusion now
- carries it onto `alerts.external_id` and writes an `alert_source_links` row,
- but only for a vendor with a writeback arm: a link to a system AiSOC cannot
- write to would read as a two-way integration that is not one.
+ `tests/isolation/test_postgres_rls.py` is the evidence, wired into
+ `integration.yml`'s migrations job where a real database with the full chain
+ already exists. It seeds an A row and a B row into every RLS-covered tenant
+ table (80 of 80 in the API chain), asserts both are visible unscoped before
+ asserting either absence, then binds the session to A and asserts B's row is
+ unreachable. It also asserts the shipped role's bypass, so a green run here
+ can never be read as "tenant isolation is on in production".
-- **A SOC operations dashboard at `/dashboards/operations`.** `/dashboard`
- answers what is happening in the estate; this answers whether the machine
- that reports it is working. The distinction matters because every failure
- mode of a detection pipeline makes it *quieter* — a connector stops polling,
- a schema changes and events bounce, a token expires — and an alert-centric
- view reports all three as good news.
+- **The attack-path relational fallback is verified against a real Postgres.**
+ `GET /graph/attack-path/{case_id}` reads `aisoc_cases` by id whenever the
+ Neo4j traversal fails — which for a deployment shipping no graph database is
+ always. It gained a tenant predicate in the previous release, but that fix
+ was covered by a gate and by review only: the statement uses
+ `CAST(:cid AS UUID)`, which SQLite mangles, so no offline suite could execute
+ it. Tenant B naming tenant A's case UUID is now demonstrated to be refused
+ against the database the query is written for, with tenant A's row asserted
+ present first.
- Six panels, each backed by one endpoint and owning its own fetch, loading,
- empty and error state: connector fleet staleness (`/health/fleet`), rejected
- events by reason (`/health/dead-letters`), alert severity and disposition
- (`/alerts/stats`), detection coverage counting only *enabled* rules
- (`/detection/coverage`), agent runs/tokens/spend (`/costs/dashboard`), and
- response actions paused for approval (`/approvals`). The first three had no
- web client at all.
+- The case-timeline linked-alert hydration in `cases.py` now binds a tenant as
+ defence in depth. Reaching it already required a tenant-scoped case, so this
+ was not a live read, but a poisoned `alert_ids` array would otherwise have
+ surfaced another tenant's alert title.
- Fleet staleness is reported against each connector's own cadence — "3.2
- intervals behind" rather than an absolute age, because a daily connector and
- a five-minute connector are not late at the same wall-clock time. One dead
- endpoint blanks one panel and names the failure; it does not take the page
- down and does not substitute plausible figures.
+- **`services/osquery-tls` enrolled every node under the literal `"default"`,
+ which resolves to no tenant on any seeded deployment.** The service keys
+ `tenant_id` as a `String(64)` while the platform keys UUIDs, and nothing
+ translated. Migration `001` seeds the canonical tenant with slug `default`,
+ but the demo seed renames that slug to `demo`, so the literal matched
+ neither the UUID nor the slug and silently matched nothing: FIM events were
+ written under a string the console could never ask for, and the FIM surface
+ returned an empty table that looked like "no file changes" rather than "the
+ read and the write disagree". `app/services/tenant_resolver.py` now resolves
+ the placeholder to the canonical seed tenant by its **stable UUID** —
+ ignoring whatever the slug has been renamed to — falling back to the sole
+ tenant of a single-tenant install, and refusing enrolment for a genuinely
+ unknown ref rather than filing the node under an unreadable tenancy. Skips
+ log at `warning` with the ref; a silent `debug` skip is how the original bug
+ survived.
-- **Federated SIEM search has a console surface.** `/api/v1/federated/backends`
- and `/api/v1/federated/search` have fanned one query out to Splunk, Microsoft
- Sentinel, Elastic and QRadar — in parallel, against each tenant's own
- vault-encrypted credentials — for some time. `apps/web` had no client for
- either, so the capability was reachable only from the SDK. `/federated-search`
- now exposes it: pick which connected SIEMs to query, describe the search once
- in free text plus optional `field operator value` filters, and read the merged
- rows.
+- **The maintainers' hosted origin shipped in the default CORS allow-list of
+ nine services.** `services/{api,agents,connectors,honeytokens,purple-team,ueba}`
+ (six byte-identical copies of the shared `cors.py`), `services/realtime`, and
+ the Go `ingest` and `enrichment` servers all listed `https://tryaisoc.com` and
+ `https://www.tryaisoc.com` among the origins they trust when
+ `AISOC_CORS_ORIGINS` is unset. That is one deployment's public origin baked
+ into every self-hosted install, trusted for credentialed cross-origin
+ requests its operator never opted into — and if that domain ever changed
+ hands, the grant travels with it. The default is now local development only;
+ the hosted deployment already sets `CORS_ORIGINS` explicitly
+ (`infra/fly/api/fly.toml`), so nothing legitimate depended on the default.
+ `apps/docs/docs/deployment/env-vars.md` documented the old list and was
+ corrected with it, and a new gate pins the six vendored `cors.py` copies
+ byte-identical — nothing enforced that before, and a single drifted copy is
+ exactly how one service would quietly keep the origin.
- The design constraint the page is built around is the endpoint's per-source
- isolation. It deliberately never fails the whole call because one backend is
- slow, 401s or 5xxs; it returns a verdict per backend instead. A UI that
- renders only the merged rows discards that, and the analyst cannot tell
- "Sentinel has nothing" from "Sentinel did not answer" — opposite conclusions
- mid-incident. So the per-backend strip renders above the rows, always, with
- each backend's own row count, latency and error message. A backend that
- failed shows no row count, because "0 rows" beside a timeout reads as
- "nothing matched". An empty result set is labelled "No matching events" when
- a backend answered and matched nothing, and "No backend returned results"
- when every backend failed.
+- **The seeded demo identity was a live, operator-owned domain.**
+ `[email protected]`, paired with a published password, appeared across
+ compose, fly, render, coolify, railway, two workflows, the web Dockerfile and
+ the docs. It told a self-hoster to type somebody else's hostname to sign in
+ to their own install, and published a well-known credential pair against a
+ real domain. Now `[email protected]` — RFC 2606 reserved, so it can never be
+ registered and can never receive mail, and it still satisfies the
+ `pydantic.EmailStr` check that rejected the earlier `[email protected]`. Every
+ config and doc moved with it. Safe to re-run: `seed_demo._ensure_user`
+ reconciles on `DEMO_USER_ID`, not on the address, and rewrites a stale email
+ in place.
- Recognised entities in a row deep-link to `/graph?entity=<type>:<value>`.
- Recognition is a fixed per-vendor field-name list rather than a heuristic:
- `src_ip`, `source.ip` and `SourceIP` all resolve to the same IP pivot, while
- `zip`, `recipient` and `description` resolve to none.
+- **`scripts/check_route_auth.py` — default-deny over all 601 routes.** Every
+ route must carry an auth dependency or appear in one of three tables, each
+ recording why it is public. It reuses the tenant-scope gate's AST collector
+ rather than adding a second parser: building it surfaced that
+ `playbooks.py`'s `ExecuteUser = Annotated[AuthUser, Depends(require_permission(...))]`
+ was reported as an unauthenticated playbook-run, because the vocabulary
+ listed `ReadUser` and `WriteUser` and nobody thought of the third. Auth
+ aliases are now resolved by **what they wrap**, which removes the naming
+ dependency in both directions — the reflex fix of appending the new name to
+ the list is how a list stops describing anything. State after: 517
+ authenticated, 73 public with a recorded reason, 11 verified in-band, 0
+ unexplained.
-- **`/graph` honours `?entity=` and selects the node it names.** Three callers
- already built that URL — the Investigation Rail's entity chips, `HuntView`'s
- "Pivot to graph" button, and now federated search — and nothing read it, so
- every pivot navigated to the graph and dropped the entity, leaving the
- analyst to find the node by eye. Both the typed form (`host:WIN-DC01`) and
- the bare form `HuntView` emits are accepted, matching on node id then label.
-- **A real operator organisation above tenants, so the MSSP console can be
- backed by data instead of gated behind demo mode.** `/mssp/overview`,
- `/mssp/tenants` and `/mssp/incidents` returned five hardcoded companies —
- "Acme Corp, health 92.4, 12 open alerts", "Wayne Enterprises", invented
- incidents with invented assignees — because the cross-tenant aggregation
- query behind them was never written. Gating the sample behind demo mode
- removed the lie but left the feature unbuilt: outside demo mode the
- console showed zeros forever. Making it real was a schema change, not an
- ungating.
+- **`approval_timers` is in a migration and carries a policy.**
+ `services/slack-bot`'s `PostgresTimerStore` created it at startup with
+ `CREATE TABLE IF NOT EXISTS`, outside every chain in the repository — so
+ `060_rls_coverage.sql` never saw it, it had no `tenant_id` for a policy to
+ filter on, and it decides whether a pending containment auto-rejects. Under
+ the DML-only runtime role the DDL itself fails, because Postgres checks the
+ schema ACL before the existence test, and `main.py` caught that into a
+ warning and fell back to the non-durable store: durable approval timers
+ would have quietly stopped being durable. `062_approval_timers.sql` adds the
+ table, a `tenant_id`, and the canonical policy; the store probes with
+ `to_regclass` and refuses with the migration's name rather than creating
+ anything, carries the tenant in every statement, and binds
+ `app.current_tenant_id` on each pooled connection so the policy engages too.
+ A deployment already carrying the runtime-created table converges on the
+ same shape, with its existing rows defaulting to an empty `tenant_id` that
+ the migration says how to backfill.
- Migration `058` adds `organizations`, `organization_members`,
- `organization_tenants` and `organization_member_tenants`, and backfills
- from `tenants.parent_tenant_id`, which keeps working. Three constraints
- hold the boundary in the database rather than in whichever code path
- writes a row: `organization_tenants` is unique on `tenant_id` so two
- providers cannot claim one customer; `organization_member_tenants` has
- composite foreign keys onto both the membership and the portfolio, so a
- grant cannot name an unmanaged tenant and releasing a tenant revokes every
- grant over it; and `organizations.home_tenant_id` cascades, so erasing a
- provider's own tenant removes the organisation while leaving its customers
- standing as unclaimed tenants.
+- **The zero-CodeQL-alert invariant was documented for four months with
+ nothing enforcing it.** `apps/docs/docs/operations/security.md` has said
+ since 2026-05-15 that "the Python alert count on `main` is zero, and we
+ treat that as a CI gate — a new alert breaks the security workflow". No such
+ gate existed anywhere in the repository. `codeql.yml` uploads SARIF and
+ `github/codeql-action/analyze` does not fail a build on findings; `main` has
+ no branch protection, so "Code scanning results" was not a required check
+ either; `security.yml`'s only hard job is the claim-to-gate matrix; and
+ nothing in the tree queried the code-scanning API. The scan itself was
+ healthy — `main` was analysed continuously, most recently minutes before
+ this was written — so the usual stale-green and mixed-`codeql-action`-pin
+ traps were both ruled out. The enforcement was simply imaginary, which is
+ why two `note`-severity alerts could sit open on `main` under a documented
+ count of zero.
- Roles carry two separate axes. `owner`/`admin` reach the whole portfolio;
- `operator`/`viewer` reach only tenants granted to them, and **nothing**
- when they have no grants — "no scope" degrading into "all scopes" is the
- shape every cross-tenant leak in this codebase has had.
+ `scripts/check_codeql_alerts.py` is that gate, wired into the new
+ `.github/workflows/codeql-alert-gate.yml` on push to `main`, on pull
+ requests, on a `workflow_run` after CodeQL finishes, and daily. It fails on
+ any open CodeQL alert at **any** severity — `note` included, since both
+ motivating alerts were `note` and carried no `security_severity_level`, so a
+ threshold anywhere would have reproduced the original silence exactly. It
+ also refuses the vacuous pass: an unanalysed ref, a declared language with
+ no analysis, an analysis older than ten days, or one belonging to a
+ different commit than the one that triggered the run are all failures rather
+ than a clean bill of health, and an input it cannot read exits 2 rather than
+ 0. Dismissed alerts stay excluded — that is GitHub's audited escape hatch
+ and the repository uses it for ~20 accepted-risk
+ `py/request-without-cert-validation` findings — but the count is printed on
+ every run so a silent mass-dismissal is visible. `--self-test` injects an
+ alert at each severity plus every shape of vacuous pass and requires the
+ gate to catch all ten; CI runs it immediately before the gate itself.
- Every cross-tenant read resolves its tenant list through
- `resolve_portfolio_scope` and nowhere else, then passes it to
- `require_scope`, which raises rather than letting an aggregate run
- unfiltered; the list is bound as a query parameter. A `?tenant_id=` filter
- is intersected with the portfolio, so naming an outside tenant narrows to
- nothing instead of reaching out. An AST gate fails the build if a new
- cross-tenant function is added that never calls `require_scope`, and
- `test_mssp_portfolio_isolation.py` replays two organisations plus an
- unmanaged tenant against live Postgres in `integration.yml`.
+- **`scripts/validate_playbooks.py` printed to stderr and called `sys.exit(2)`
+ while being imported** (CodeQL `py/print-during-import`, alert #896). Beyond
+ the note, this was a live defect:
+ `scripts/check_playbook_schema_parity.py` imports the module to read
+ `SUPPORTED_TRIGGERS` and wraps the import in `except Exception`, which
+ cannot catch `SystemExit` — a broken environment would have killed the
+ parity gate's interpreter instead of producing its diagnostic. The import
+ now raises `ImportError`, which that handler catches.
- The sample rows are deleted rather than gated. `health_score` is gone
- because it was an undefined composite; `mttr_minutes` is measured from
- cases a tenant actually closed and is null when it closed none; seeded
- rows are counted separately as `synthetic_alerts` and excluded from the
- headline figures.
+- **`python-jose` is gone, and `ecdsa` with it.** `ecdsa` carried
+ CVE-2024-23342 (Minerva timing attack on P-256) with no patched release —
+ OSV records the affected range as introduced at 0 with no fixed event,
+ because upstream states python-ecdsa offers no side-channel resistance and
+ will not fix it. It was an unconditional requirement of `python-jose`, so
+ the suppression was renewed rather than resolved. `services/api` now signs
+ and verifies with `PyJWT`, which it already depended on for the OIDC and
+ SAML paths; `python-jose`, `ecdsa` and `rsa` all leave the dependency tree.
+ Six CI workflows installed `python-jose[cryptography]` and never installed
+ `PyJWT`, and reached `cryptography` — a declared direct dependency of
+ `services/api` — only through that extra; they now install both by name.
-- **Per-tenant limit headroom, so a cap cannot throttle a customer
- silently.** A tenant that hits a ceiling raises no error anyone sees —
- alerts keep arriving and stop being triaged, which reads to an evaluator
- as "the AI doesn't work". `app/services/entitlements.py` measures
- `connectors`, `seats`, `alerts_per_day` and `triages_per_month` from real
- rows, reports `ok` / `warning` / `exhausted` per tenant across a
- portfolio, and logs exhaustion at `warning`. AiSOC ships uncapped: a key
- with no configured ceiling reports `unlimited` rather than a default
- nobody set. Ceilings come from `tenants.limits` (per tenant, wins in
- either direction) or the new `AISOC_DEFAULT_TENANT_LIMITS` setting —
- declared as a real field, because an undeclared setting is dropped by
- `extra="ignore"` and the operator who exports it gets no explanation.
+- **`image-size` moved to a patched release instead of staying suppressed.**
+ Both advisories were held open on the reading that no fix existed. They
+ record a vulnerable range of `<= 2.0.2`, and npm has published 2.0.3 and
+ 2.0.4; a null `first_patched_version` is not the same claim as no fix
+ existing. A pnpm override pins `>=2.0.4 <3`, which
+ `@docusaurus/mdx-loader`'s `^2.0.2` range accepts. This clears the only two
+ high-severity advisories in the pnpm workspace.
-- **The Splunk warehouse driver executes.** It previously raised
- `HuntNotConfigured("provider scaffolded but live SPL execution not yet
- shipped")` on every call, so the SPL every hunt was translated into was
- discarded. `app/services/spl_runner.py` carries the same three guards as the
- ES|QL runner — per-tenant SSRF allow-list, air-gap policy, row cap — and
- prefixes the translator's bare `index=…` expression with `search`, without
- which Splunk's REST API answers 400 on every hunt.
+- **`apps/mobile` resolved a vulnerable `image-size` that the workspace had
+ already fixed.** Two high-severity advisories — GHSA-5p2g-fcmc-qvqq
+ (CVE-2025-71329, JXL/HEIF parsers) and GHSA-w3rx-r6r6-pgpr
+ (CVE-2025-71330, ICNS parser), both infinite-loop denial of service, both
+ fixed in 2.0.3 — stayed open against `apps/mobile/pnpm-lock.yaml` for a
+ structural reason rather than an unfixed one. The repository root pinned
+ `image-size` to `>=2.0.4 <3` through a pnpm override, and `apps/mobile` is
+ a deliberately independent install root with its own
+ `pnpm-workspace.yaml` and lockfile, created that way so its install would
+ stop rewriting the root lock. The override could not reach it and nothing
+ compared the two. Resolved by declaring the same override in
+ `apps/mobile/package.json`; the lockfile now resolves `[email protected]`
+ and drops its `queue` dependency with it.
-### Removed
+ Metro is the only consumer, and no version of Metro that depends on
+ `image-size` permits 2.x — every one declares `^1.0.2`, which is why
+ Dependabot recorded `security_update_not_possible` and failed its job
+ instead of opening a pull request. Metro 0.83.3's two call sites go
+ through `_interopRequireDefault(require("image-size")).default`, and
+ `[email protected]`'s CommonJS build still exports a callable `default`, so
+ `metro.getAssetSize` returns correct dimensions against it.
-- **The identity timeline's phantom second source.** It queried a table named
- `aisoc_events` that no migration creates, nothing writes to, and that appears
- nowhere else in the repository — with its failure swallowed at `DEBUG`. It
- read as a second source of evidence while returning nothing on every
- deployment. The remaining source now reports itself on the response
- (`sources_unavailable`) when it cannot be read, so an empty timeline caused
- by a broken query is distinguishable from one caused by no matches.
+- **The dependency suppression list is empty.** All 42 entries in
+ `scripts/security_audit_ignores.txt` were re-verified against OSV and
+ against the versions the lockfiles actually resolve. Every one was
+ resolvable, and most of the justifications had stopped being true: nine
+ starlette entries blamed a `fastapi<0.137` cap that exists in neither bot
+ service, three cryptography entries blamed `<50`/`<49` caps that exist
+ nowhere in the repository, and the langchain, weasyprint, anyio, aiohttp,
+ h2, idna and pydantic-settings entries each named a version older than the
+ one their lock resolves. The file now records what was measured, so the next
+ review starts from evidence rather than from the previous reason string.
-- **The Chronicle warehouse scaffold.** It read `hunt.translated_query["udm"]`,
- nothing in the repository emits UDM, and it gated on two settings that were
- never fields — so it could not be selected and could not run, while
- `available_providers()` reported it as a supported warehouse. Adding a real
- one is a `register_provider` call plus a UDM translator.
+- **The pnpm and Go arms of the audit could report success without scanning.**
+ A failed `govulncheck`, an unparseable `pnpm audit` response and a registry
+ that never answered were all recorded as warnings, which exit 0 — so the job
+ printed "0 findings" for ecosystems it had not read. All three now record a
+ coverage gap, which `exit_code_for` already failed on for the Python arm.
+ That property — an unscanned target fails the build — had no test; it does
+ now.
## [9.0.0] — 2026-09-23
@@ -8534,7 +8449,8 @@ demo profile. Details below.
- Helm chart for Kubernetes deployment (`infra/helm/aisoc/`)
- MIT License
-[Unreleased]: https://github.com/beenuar/AiSOC/compare/v9.0.0...HEAD
+[Unreleased]: https://github.com/beenuar/AiSOC/compare/v10.0.0...HEAD
+[10.0.0]: https://github.com/beenuar/AiSOC/compare/v9.0.0...v10.0.0
[9.0.0]: https://github.com/beenuar/AiSOC/compare/v8.1.1...v9.0.0
[8.1.1]: https://github.com/beenuar/AiSOC/compare/v8.1.0...v8.1.1
[8.1.0]: https://github.com/beenuar/AiSOC/compare/v8.0.0...v8.1.0
diff --git a/README.md b/README.md
index 07029d11c..d4e291f01 100644
--- a/README.md
+++ b/README.md
@@ -7,7 +7,7 @@
**An open-source, self-hostable AI Security Operations Center.** It ingests your security telemetry, detects and correlates threats, investigates them with AI agents whose reasoning is fully auditable, and proposes responses a human approves.
[![License: MIT](https://img.shields.io/badge/License-MIT-22c55e.svg?style=flat-square)](https://opensource.org/licenses/MIT)
-[![Version](https://img.shields.io/badge/version-9.0.0-f59e0b?style=flat-square)](CHANGELOG.md)
+[![Version](https://img.shields.io/badge/version-10.0.0-f59e0b?style=flat-square)](CHANGELOG.md)
[![CI](https://img.shields.io/github/actions/workflow/status/beenuar/AiSOC/ci.yml?branch=main&label=CI&style=flat-square)](https://github.com/beenuar/AiSOC/actions/workflows/ci.yml)
[![CodeQL](https://img.shields.io/github/actions/workflow/status/beenuar/AiSOC/codeql.yml?branch=main&label=CodeQL&style=flat-square)](https://github.com/beenuar/AiSOC/actions/workflows/codeql.yml)
[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/beenuar/AiSOC/badge)](https://securityscorecards.dev/viewer/?uri=github.com/beenuar/AiSOC)
@@ -69,7 +69,7 @@ Open **http://localhost:3000** and sign in with the credentials `make up`
printed (API docs at **http://localhost:8000/api/docs**).
Something wrong? `make doctor` checks every dependency and tells you what to
-run next. Requires Docker with ~6 GB of RAM.
+run next. Requires Docker with ~6.5 GB of RAM.
## Try it without connecting anything
@@ -119,8 +119,6 @@ response.
## Deployment profiles
-One architecture, three profiles of it.
-
| Profile | Command | Services | RAM | What you get |
|---|---|---|---|---|
| **core** | `make up` | 11 | ~6.5 GB | The full alerting pipeline: ingest → detect → correlate → alert → triage → console, plus the LLM gateway |
@@ -130,13 +128,15 @@ One architecture, three profiles of it.
CORE is not a cut-down toy — it is the smallest deployment that takes a real
event and produces a real alert.
-**What CORE can and cannot do about AI.** The LLM gateway ships in CORE, so
-AI triage works as soon as you put a provider key in `.env` — no profile
-change and no second command. Until you do, there is no AI: every alert is
-triaged by the deterministic path, the console says so, and no LLM call is
-made. That is the honest default, not a degraded one. The gateway is also
-what reports what a call cost, so a CORE install's cost figures are measured
-rather than estimated. Reasoning:
+**What CORE can and cannot do about AI.** The LLM gateway ships in CORE, so a
+provider key in `.env` reaches a model with no profile change and no second
+command. That route is proven against a **local** model — one real completion,
+measured, through Ollama on an operator's own hardware. **No hosted provider
+has ever been exercised**: this project has no funded key, so its published
+per-model rows read *not measured* rather than zero. Until you supply a key
+there is no AI — every alert takes the deterministic path and the console says
+so. The gateway is also the only thing that can report what a call cost, so an
+unmeasured cost renders as absent, never as `$0.00`. Reasoning:
[ADR-0006](docs/decisions/0006-llm-gateway-in-core.md).
## Real vs synthetic data
@@ -184,12 +184,12 @@ so. They do not fabricate a verdict.
| Detection engine (833 executable rules) | Stable | Fixture replay + unit | Yes |
| Alert correlation into incidents | Stable | Unit | Yes |
| REST API + web console | Stable | Unit + integration | Yes |
-| AI triage + Investigation Ledger | Beta | Unit + substrate eval | Yes, copilot mode |
+| AI triage + Investigation Ledger | Beta | Unit + substrate eval + local-model run | Yes, copilot mode |
| Event lake + hunting (ClickHouse) | Beta | Unit | Yes, `full` profile |
| Entity graph (Neo4j) | Beta | Unit | Yes, `full` profile |
| Governed response actions | Beta | Unit | Human-approved only |
| Scheduled connectors | Beta | Contract tests | `full` profile |
-| UEBA | Beta | Unit | `full` profile |
+| UEBA | Beta | Unit + live migration round-trip | `full` profile |
| Package distribution (npm/PyPI) | Ready, unpublished | `release.yml` builds and packs all eight on every tag | Install from source — the upload is blocked on registry credentials, which is an account action |
## What AiSOC is not
@@ -240,8 +240,8 @@ Guides: [add a connector](https://beenuar.github.io/AiSOC/docs/plugins/hello-plu
[plugin lifecycle](https://beenuar.github.io/AiSOC/docs/plugins/lifecycle) ·
[contributing](CONTRIBUTING.md)
-Every number in this README is produced by `scripts/project_stats.py` and
-checked in CI, so a figure cannot drift from the tree without a red build.
+The connector and detection-rule counts here are recounted from the tree by
+`scripts/project_stats.py`, which CI fails if this README disagrees with it.
## Roadmap · Contributing · License
diff --git a/RELEASES.md b/RELEASES.md
index e608b5d76..c3b7d29fd 100644
--- a/RELEASES.md
+++ b/RELEASES.md
@@ -2,13 +2,31 @@
This file mirrors what used to live in the "What's new" section of [`README.md`](README.md). The complete, machine-readable inventory (with file paths, env-var diffs, and per-release test counts) lives in [`CHANGELOG.md`](CHANGELOG.md).
-> **TL;DR for first-time visitors:** AiSOC is on `v9.0.0`, released 2026-09-23 — ten waves, and one finding under nearly all of them: the mechanism existed, was tested, and nothing called it. The one worth stating first is that **approving an action executed nothing** — `decide()` flipped a row and never reached the execution service, so every tap of Approve recorded a decision and ran nothing while telling the operator the opposite. Both ends of that loop are closed. Every product claim is backed by a failing CI test (claim-to-gate matrix: 99 GATED / 9 PARTIAL / 0 NO GATE). The latest GitHub release with notes and downloads: <https://github.com/beenuar/AiSOC/releases/latest>.
+> **TL;DR for first-time visitors:** AiSOC is on `v10.0.0`, released 2026-09-25 — a release about controls that were written and not reachable. Row-level security covered 92 tables and filtered nothing, because every service connected to Postgres as a superuser; fifty-eight routes carried no authentication at all; UEBA reported healthy and could not write an anomaly; AI triage was wired to a gateway both of its resolvers ignored. **It is a major because upgrading requires action** — services now connect as a DML-only `aisoc_app` role. Every product claim is backed by a failing CI test (claim-to-gate matrix: 139 rows — 131 GATED / 8 PARTIAL / 0 NO GATE). The latest GitHub release with notes and downloads: <https://github.com/beenuar/AiSOC/releases/latest>.
---
## What's new
-`VERSION` is `9.0.0`. The **v9.0.0** release (2026-09-23) is ten waves of one audit question: what in this tree exists, is tested, and has no caller?
+`VERSION` is `10.0.0`. The **v10.0.0** release (2026-09-25) is an audit of a single shape: a control that exists, is tested, and sits on a path nothing reaches.
+
+**Read the BREAKING section of the changelog before upgrading.** Every deployment must act on one item: services connect to Postgres as a DML-only `aisoc_app` role rather than as the schema owner, which is what makes the row-level-security policies filter. The bundled Postgres provisions it on a fresh volume; an existing volume or a managed database does not.
+
+**v10.0.0 highlights (September 25, 2026)**
+- **92 row-level-security policies filtered nothing.** Compose, CI, the Helm chart and the Terraform environment all ran every service as `POSTGRES_USER=aisoc`, which the postgres image creates as a superuser — and a superuser ignores policies even under `FORCE ROW LEVEL SECURITY`. Measured with one alert per tenant and the session bound to tenant A: the old role saw 2 rows, the new role sees 1.
+- **Fifty-eight routes across four services carried no authentication at all.** Reproduced rather than inferred: an anonymous caller with no `Authorization` header created a playbook, listed all 64, **executed one**, deleted it, then read copilot conversations and ran a threat hunt. A further thirty routes let the caller name the tenant they were reading.
+- **UEBA could never write a baseline or an anomaly, and reported healthy throughout.** Three schema defects on one code path, then a fourth: the baseline updater mutated a dict in place, so SQLAlchemy saw no change and left the column out of every `UPDATE`. Measured on a live stack — 36 events for one entity, all processed without error, the stored baseline frozen at `count: 1`.
+- **AI triage could not reach a model, and it was not a missing key.** Compose set `LLM_GATEWAY_URL` on both services; both resolvers had been deliberately written to ignore it, so every `aisoc-<role>` alias went to a provider that cannot resolve one and the caller rendered that as "no LLM available". The gateway also moved from `full` into CORE, so a key now works with no profile change.
+- **A new user who followed only the README could not log in** — the seeded address was rejected by the login route's own validator before the password was compared, and the committed hash matched no published credential. `make up` now creates an administrator and prints a generated password once.
+- **Costs stopped being invented.** The tracker priced the `aisoc-<role>` *alias* against a table of hosted list prices it does not appear in, so every call fell through a default and a local completion that cost nothing was booked at `$0.000999` — a figure that reached the dashboard, the ledger and the budget circuit breaker. Every cost is now measured, labelled an estimate, or absent.
+
+The full inventory — 188 entries, including what is knowingly still open — lives under `[10.0.0]` in [`CHANGELOG.md`](CHANGELOG.md).
+
+---
+
+## v9.0.0 (2026-09-23)
+
+`VERSION` was `9.0.0`. The **v9.0.0** release (2026-09-23) is ten waves of one audit question: what in this tree exists, is tested, and has no caller?
**v9.0.0 highlights (September 23, 2026)**
- **Approving an action executed nothing.** `decide()` flipped a row, notified the realtime service and returned 200 without ever touching `services/actions`. The other end was missing too — nothing in the repository ever created an approval, so the queue had no producer and was structurally empty on every deployment. A queue with no producer and a queue with no pending work look identical.
diff --git a/ROADMAP.md b/ROADMAP.md
index 4310bac5f..1ace56766 100644
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -677,6 +677,35 @@ Full inventory under `[8.1.1]` in [`CHANGELOG.md`](CHANGELOG.md).
---
+## v10.0 — Shipped (2026-09-25)
+
+An audit of one shape: a control that exists, is tested, and sits on a path
+nothing reaches. Row-level security covered 92 tables and filtered nothing,
+because every service connected to Postgres as a superuser. Fifty-eight routes
+across four services carried no authentication at all, and thirty more let the
+caller name the tenant they were reading. UEBA reported healthy and could not
+write a baseline or an anomaly. AI triage was wired to a gateway that both of
+its resolvers had been deliberately written to ignore.
+
+**A major because upgrading requires action.** Services now connect as a
+DML-only `aisoc_app` role rather than as the schema owner, which is what makes
+those 92 policies filter; an existing data volume or a managed database must be
+migrated deliberately. Also breaking: the `/mssp/*` payloads dropped fields
+that described fabricated data and now refuse a non-member, two executor-less
+`ActionType` members were removed, `CostTracker.total_cost_usd` is gone, and
+the published `aisoc-web:latest` image no longer carries demo mode — the demo
+build moved to its own tag.
+
+Two published figures were re-measured rather than restated: CORE is 11
+services (the LLM gateway moved into it) and `full` is 21, not 30. The quick
+start now ends by creating an administrator and printing a generated password
+once, because the credential pair the documentation published was wrong in
+three independent ways.
+
+Full inventory under `[10.0.0]` in [`CHANGELOG.md`](CHANGELOG.md).
+
+---
+
## v9.0 — Shipped (2026-09-23)
Ten waves of one audit question: what in this tree exists, is tested, and has
diff --git a/VERSION b/VERSION
index f7ee06693..a13e7b9c8 100644
--- a/VERSION
+++ b/VERSION
@@ -1 +1 @@
-9.0.0
+10.0.0
diff --git a/apps/web/package.json b/apps/web/package.json
index 3afc41f52..945261aa6 100644
--- a/apps/web/package.json
+++ b/apps/web/package.json
@@ -1,6 +1,6 @@
{
"name": "@aisoc/web",
- "version": "9.0.0",
+ "version": "10.0.0",
"private": true,
"description": "AiSOC SOC Console - Next.js 14 Frontend",
"license": "MIT",
diff --git a/docs/architecture/README.md b/docs/architecture/README.md
index c95000385..45b8b78c5 100644
--- a/docs/architecture/README.md
+++ b/docs/architecture/README.md
@@ -251,9 +251,14 @@ One architecture, three profiles of it — not three architectures.
| Profile | Command | Services | RAM | What you get |
|---|---|---|---|---|
-| **core** | `make up` | 10 | ~6 GB | Ingest → detect → correlate → alert → triage → console. The full alerting pipeline. |
-| **full** | `make up-full` | 30 | ~12 GB | Core plus event lake, entity graph, vector store, enrichment, connectors, LLM gateway. |
-| **demo** | `make up && make demo` | 10 | ~6 GB | Core plus clearly-labelled synthetic data. |
+| **core** | `make up` | 11 | ~6.5 GB | Ingest → detect → correlate → alert → triage → console, plus the LLM gateway. The full alerting pipeline. |
+| **full** | `make up-full` | 21 | ~12 GB | Core plus event lake, entity graph, vector store, enrichment, connectors. |
+| **demo** | `make up && make demo` | 11 | ~6.5 GB | Core plus clearly-labelled synthetic data. |
+
+The LLM gateway moved into CORE this release, so a provider key works without
+a profile change. `full` is 21 services, not the 30 published here previously:
+30 is `full` plus the `monitoring`, `chatops`, `extras` and `osquery`
+profiles, which `make up-full` does not start.
CORE is not a toy. It is the smallest deployment that can take a real event
and produce a real alert, which is the thing the product is for.
diff --git a/docs/roadmap/v8-progress.md b/docs/roadmap/v8-progress.md
index 1f6a889a0..30286ef8b 100644
--- a/docs/roadmap/v8-progress.md
+++ b/docs/roadmap/v8-progress.md
@@ -1,11 +1,12 @@
# AiSOC v8 — progress tracker
-**Last updated:** 2026-09-23
-**Current release:** `v9.0.0` (2026-09-23) · **Next:** unscheduled.
+**Last updated:** 2026-09-25
+**Current release:** `v10.0.0` (2026-09-25) · **Next:** unscheduled.
Packaging is no longer named against a version: it slipped v8.0 to v8.1 to
v8.2 for the same reason each time, and the blocker is registry credentials
-rather than code. See `[9.0.0]` in `CHANGELOG.md` for what v9.0 closed and
-what it knowingly left open.
+rather than code. See `[10.0.0]` in `CHANGELOG.md` for what v10.0 closed and
+what it knowingly left open — it is a major because upgrading requires an
+operator action, not because the v8 line is superseded.
This tracker is the at-a-glance view of what has landed across the v8 line and
what is still open. It is deliberately short and dated. When it disagrees with