Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
docs(agents): incremental continual-learning memory update (#445)
Beenu Aroraccd6c70194b5
diff --git a/AGENTS.md b/AGENTS.mdindex 498d02937..22d557770 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -70,6 +70,7 @@- **Static analysis hygiene (CodeQL):** Python alert count on `main` is zero as of 2026-05-14, enforced as a CI gate. Two patterns drove the cleanup:- **`py/log-injection` — sanitise inline at the call site.** The taint tracker doesn't follow `_log_safe(value)` through function boundaries reliably, but it does recognise an inline `str(value).replace("\r", "").replace("\n", " ")[:N]` chain. Canonical example: `services/api/app/api/v1/endpoints/waitlist.py`. Even `uuid.UUID`-typed values are sanitised explicitly so the property is visible to both CodeQL and future readers.- **`py/import-and-import-from` — pick one import style per module.** Tests that monkey-patch module-level constants should use `pytest.MonkeyPatch.setattr(module, "_NAME", value)` (with the standard from-import form), not `import app.foo as foo_module` _and_ a from-import for the same names.+ - **`py/request-without-cert-validation` — accepted-risk for on-prem appliance clients.** The Splunk / FortiGate / PAN-OS / osctrl / FleetDM / MISP clients default to `verify=True` and only disable TLS verification on an explicit operator opt-in (required for self-signed / internal-CA appliances), so these ~20 alerts are dismissed `won't fix` with a comment recommending CA-bundle pinning as the future alternative. Related one-liners from the Jul-2026 pass: a count var named `secret*` trips the alert heuristic (rename to e.g. `vaulted`); `py/ineffectual-statement` on Protocol/abstract `...` bodies → use a docstring body; `py/empty-except` → add an explanatory comment. Zero open alerts re-confirmed on `main` (Jul 2026), now across Python + Go (`go vet` / `go build`).- Docs anchor: `apps/docs/docs/operations/security.md#static-analysis-codeql`.- **UEBA env-var convention (PR #135, first community contribution, closes Issue #134):** `services/ueba/app/core/config.py` uses Pydantic `BaseSettings` with `populate_by_name=True` and `Field(default=..., validation_alias=AliasChoices("UNPREFIXED", "UEBA_PREFIXED"))` per field. Both forms work; unprefixed wins when both are set. `services/ueba/alembic/env.py` follows the same rule: `os.environ.get("DATABASE_URL") or os.environ.get("UEBA_DATABASE_URL", default)`. Same pattern as `services/fusion/app/core/config.py`. Test coverage: `services/ueba/tests/test_config.py` asserts four cases (unprefixed-only / prefixed-only / both-with-unprefixed-winning / default-fallback). Doc anchor: `apps/docs/docs/deployment/env-vars.md#ueba-service-servicesueba`.- **Release flow is tag-driven (CHANGELOG-extraction).** `.github/workflows/release.yml` watches for `v*` tags, extracts the matching `[X.Y.Z]` section from `CHANGELOG.md` (Keep-a-Changelog format) via an awk script, then publishes via `softprops/action-gh-release` and pushes 12 service images to GHCR with the version tag. To cut a release: (1) promote `[Unreleased]` → `[X.Y.Z]` in `CHANGELOG.md` (leaving a fresh empty `[Unreleased]`), (2) bump `VERSION` and `apps/web/package.json`, (3) refresh the README version badge / headline / roadmap entry, (4) commit `chore(release): vX.Y.Z`, (5) `git tag vX.Y.Z && git push --tags`. Do NOT create the GitHub release manually — the workflow does it. v7.5.0 (2026-06-29) is the canonical reference example.@@ -77,3 +78,7 @@- **`packages/aisoc-sandbox` — offline 30-second on-ramp.** Standalone simulator wrapping the agent graph with in-memory SQLite + deterministic LLM mode, designed so `pip install aisoc-sandbox && aisoc-sandbox demo` completes in <30 s with zero external deps. Ships five bundled scenarios (lateral-movement + four added in PR #368). Distinct from the full production LangGraph stack — keep it lightweight on purpose. Tested via `packages/aisoc-sandbox/tests/`.- **README + onramp CI gates.** `README.md` is capped at ≤250 lines (current target after PR #368 was 1,102 → 234). Heavy content lives in `RELEASES.md`, `ROADMAP.md`, `docs/architecture/overview.md`, and the docs portal. `.github/workflows/onramp-gates.yml` enforces four gates on every PR: (1) every CLI command shown in README either runs in CI or is explicitly marked "coming in v8.0", (2) `demo/hero.gif` + screencast assets exist on `main`, (3) README line count ≤250, (4) `aisoc-sandbox demo` cold-start. The matching local script lets contributors run the same checks. Advertised packages (`@aisoc/mcp`, `@aisoc/sdk`, `@aisoc/plugin-sdk`, `aisoc-cli`, `aisoc-sdk`, `aisoc-plugin-sdk`) are NOT yet published — README and docs use monorepo-local invocations and flag future commands as "coming in v8.0".- **Devcontainer prebuilt image.** `.devcontainer/devcontainer.json` prefers `ghcr.io/beenuar/aisoc-devcontainer:latest` (with local Dockerfile fallback). `.github/workflows/devcontainer-build.yml` publishes multi-arch (amd64+arm64) on changes to the Dockerfile; `.github/workflows/devcontainer-coldstart.yml` is `workflow_run`-triggered against the freshly-published `:latest` and probes the toolchain (image pull ≤60 s, `uv`/`ruff`/`docker compose`/etc. on PATH). Two durable gotchas the Dockerfile encodes: (a) `docker-compose-plugin` is NOT in Debian Bookworm's default apt repo — install Compose v2 by direct curl of the official binary into `/usr/local/lib/docker/cli-plugins/docker-compose` (pinned to v2.29.7); (b) `pipx`-installed tools land in `/root/.local/bin` which is invisible to the container's non-root `node` user — install `uv` and `ruff` system-wide into `/usr/local/bin` instead. Docs anchor: `apps/docs/docs/development/devcontainer.md`.+- **World-class hardening program (12-phase, from Jul 2026) + claim-to-gate governance.** A staged program drives the repo toward production readiness; phase status is tracked in `ROADMAP.md` + a local `PROGRESS.md`, and the program forbids stub implementations, fake/circular gates, and simulated results presented as functional. Its backbone is `CLAIM_TO_GATE_MATRIX.md` — a permanent CI artifact mapping every product claim → the CI gate that proves it (each row is `GATED` / `PARTIAL` / `NO GATE`). A checker script enforces a **ratchet**: CI fails if the `NO GATE` count exceeds a committed baseline (`MAX_NO_GATE`, currently 3; `--max-no-gate` configurable). `.github/workflows/security.yml` runs the matrix ratchet as the **sole hard gate**, plus gitleaks / semgrep / trivy / checkov / tfsec in **observe mode** (`continue-on-error`, SARIF upload via `github/codeql-action`); `.gitleaksignore` is the curated secret allowlist and Trivy is installed as a pinned binary via a `run:` step (marketplace action versions were unreliable). Phase 1 also shipped a per-run nonce `PromptInjectionGuard` (ties detections to ledger flags + L0 demotion) and memory-poisoning-resistant override provenance (author / confidence / trust_weight; `apply_redisposition` gated by a confirmation token).+- **Detection corpus honesty.** `detections/` carries ~6,975 rules on disk, but only ~939 are **executable** today (861 native + 77 Sigma-imported + 1 community); the remainder are quarantined/disabled. A truth-table documents the executable-vs-quarantined split. Keep these counts honest — never present quarantined/aspirational rules as executable (same synthetic-vs-real transparency rule as the eval harness).+- **Stateful stores + per-store tenant isolation.** Beyond Postgres/Redis the platform runs Kafka (Redpanda-compatible in CI: KRaft, no ZooKeeper), ClickHouse (event lake, table `aisoc.raw_events`), Neo4j (entity graph), and Qdrant (vector store in `services/threatintel` via `QdrantStore`, for IOC/actor embeddings). This complements the query-layer Postgres isolation on `/hunts` + `/cases`: ClickHouse scopes via `lake_sql.rewrite_for_tenant()` injecting a `tenant_id` predicate into user SQL; Neo4j via a `tenant_id` node-property filter; Redis via a `tenant:{tid}:` key-prefix; Kafka via the `X-Tenant-ID` header / `tenant_id` envelope with per-tenant downstream filtering; Qdrant via `tenant_id` in point payloads + a mandatory query filter (public feed intel is intentionally global, so scoping targets tenant-private data). Offline assertions live in `tests/isolation/`; a CI live-container replay seeds two tenants and asserts cross-tenant reads never leak (skips gracefully when containers are absent).+- **Production event-flow spine + response posture.** The CI-gated production path is: Connectors → `services/ingest` (OCSF normalize) → Kafka `raw_events` → `services/fusion` (promote + fuse) → Postgres alerts + `services/realtime` WS; the fusion consumer validates event schema and routes poison events to a dead-letter queue (they were previously dropped silently). Default response/autonomy posture is **copilot / dry-run** (human approval required); SOAR executors return `simulated` results unless real vendor credentials are plumbed. Populating the ClickHouse lake and running the detection corpus against the live stream are tracked roadmap items (a lake-writer / detect worker), so live-stream detection may not be wired in a given checkout — verify before assuming it runs.