Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
docs(v7.0): promote CHANGELOG to [7.0.0], sync ROADMAP/README/progress docs
Beenu Aroraf0a0d5d2e64a
diff --git a/AI_STACK_PLAN_PROGRESS.md b/AI_STACK_PLAN_PROGRESS.mdindex 866d84dd9..1f3b94074 100644--- a/AI_STACK_PLAN_PROGRESS.md+++ b/AI_STACK_PLAN_PROGRESS.md@@ -658,3 +658,24 @@ Documentation:enumerates LLM keys alongside connector secrets.Workspace rule: do not stop until every todo is done.++---++## v1.0 Buyer-Value Plan — COMPLETE (2026-05-10)++Author: Beenu Arora <[email protected]>+Released as: **v7.0.0** on `main` branch.++All workstreams from `aisoc_v1.0_—_buyer-value_plan_c8116970.plan.md` are+now shipped and verified. See `PROGRESS.md` for the full workstream table,+fix log, and documentation changes.++PR #43 (`feat/threatintel-attribution-v0-fixed`) was squash-merged into+`main` on 2026-05-10 (205 files changed, +16 524 / -1 961 lines).++CHANGELOG promoted to `[7.0.0] — 2026-05-10`.+ROADMAP v7.0 marked Shipped ✅; v8.0 Planned section added.+README version badge updated to `7.0.0`.++The AI Stack & Data Integration plan workstreams (this file) remain at+DONE status — no regressions introduced by the v7.0 buyer-value plan.diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 2572d7c7c..04c7da5a5 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -5,7 +5,169 @@ All notable changes to AiSOC will be documented in this file.The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).-## [Unreleased]+## [7.0.0] — 2026-05-10++### Added — v1.0 Buyer-Value Plan: ChatOps, Digest PDF, BYOK, Air-gap, WCAG AA, Analytics++This release ships the complete v1.0 buyer-value plan across 16 workstreams.+All items were designed, implemented, tested, and reviewed by+Beenu Arora <[email protected]>.++#### WS-A1 — Slack ChatOps Bot (`services/slack-bot/`)++- **`services/slack-bot/`** — New standalone FastAPI service using `slack-bolt`+ async adapter. Ships `/aisoc triage <case_id>`, `/aisoc approve <action_id>`,+ `/aisoc status <case_id>`, and `/aisoc summary <case_id>` slash commands.+ Interactive approval buttons route back through the API approval endpoint so+ human-in-the-loop gates work from Slack without opening the console.+- 61 pytest cases cover the slash-command handlers, interactive payloads, API+ client calls, and error paths (bad token, non-200 API response, missing case).++#### WS-B1/B2 — Executive Digest PDF + Weekly Scheduler++- **`services/api/app/services/digest_pdf.py`** — Generates a branded A4 PDF+ for `ExecutiveDigest` objects using ReportLab. Includes cover page, KPI tiles,+ alert-volume chart, top-rule table, top-actor table, and remediation summary.+- **`services/api/app/workers/weekly_digest_task.py`** — APScheduler task that+ runs every Monday at 06:00 UTC, builds a digest for every active tenant, and+ delivers it via `POST /api/v1/reports/digest/email` or writes it to blob+ storage. Controlled by `DIGEST_SCHEDULE_ENABLED` env flag.+- **`services/api/app/services/digest_html.py`** — HTML mirror of the PDF for+ in-browser preview.+- **`services/api/tests/test_digest_pdf.py`** — 12 pytest cases covering PDF+ generation, chart rendering, and weekly scheduler triggering.++#### WS-C1/C2/C3 — Playbook Gallery, Detection Proposals, GitHub PR Integration++- **`apps/web/src/components/playbooks/PlaybooksGallery.tsx`** — Tabbed gallery+ with 12 curated packs (Phishing, Ransomware, BEC, IAM Key Compromise, …).+ Each card shows TTP coverage badges, author, version, and a one-click+ **Import** button that calls `POST /api/v1/playbooks/import`.+- **`services/api/migrations/039_detection_proposal_github_pr.sql`** —+ Adds `github_pr_url TEXT` and `github_pr_number INT` to `detection_proposals`.+- **`services/api/app/services/github.py`** — `GitHubService` creates draft PRs+ against the tenant's detection repo when a detection proposal is promoted.+ Supports GHES and github.com via `GITHUB_API_URL` env var.+- 25 playbook YAML templates added under `detections/playbooks/` and 12 pre-built+ playbook packs under `playbooks/packs/v1/`.++#### WS-D1 — BYOK Per-Tenant Settings UI++- **`apps/web/src/components/settings/SettingsView.tsx`** — New "AI / LLM"+ settings panel: provider picker (OpenAI, Azure OpenAI, Anthropic, Ollama),+ API-key input, model selector, temperature slider, and connection test button.+- **`apps/web/src/components/settings/SettingsView.byok.test.tsx`** — 12 Vitest+ tests covering form rendering, provider switching, key masking, connection test+ success/error paths, and save confirmation.++#### WS-D2 — Investigation Timeline (Replayable)++- **`apps/web/src/components/copilot/InvestigationTimeline.tsx`** — 684-line+ React component that renders the investigation ledger as a playable timeline.+ Each step shows the agent name, tool call, rationale, duration, and status+ badge. A scrubber lets analysts replay from any step.++#### WS-D3 — Case Auto-Summary + PDF Export++- **`services/api/app/services/case_summary.py`** — LLM-powered case summariser+ (structured output via function-calling). Produces `CaseSummaryResult` with+ `headline`, `severity_rationale`, `recommended_action`, and `evidence_links`.+- **`services/api/app/services/case_summary_html.py`** — HTML renderer for the+ summary, used by the PDF exporter and the in-browser case card.++#### WS-F1 — Light Theme Persisted in User Profile++- **`apps/web/src/components/theme/ThemeProvider.tsx`** — Theme preference+ (`light` | `dark` | `system`) stored in `localStorage` and synced to+ `PATCH /api/v1/users/me/preferences`. Survives logout and device switch.++#### WS-F2 — WCAG AA Accessibility (axe-core CI gate)++- **`apps/web/src/test/a11y.test.tsx`** — 55-line axe-core test suite. Renders+ `AlertsView`, `CasesView`, `PlaybooksView`, `DashboardView`, and 3 modal+ components; fails the build if any WCAG 2.1 AA violation is found.+- Sidebar landmark roles, ARIA labels, focus trapping in modals, skip-navigation+ link, and colour-contrast fixes applied across the entire component tree.++#### WS-F3 — Saved Views + Drag-Drop Dashboard Widgets++- **`apps/web/src/components/dashboard/DashboardView.tsx`** — Dashboard is now+ fully composable: widgets can be dragged, dropped, resized, pinned, and+ removed. Layout serialised to `POST /api/v1/saved-views`.+- **`services/api/app/api/v1/endpoints/saved_views.py`** — CRUD for per-user+ saved views (dashboard layout, column configs, active filters).++#### WS-G1/G2 — Threat Actor Attribution Engine v0 + Air-Gap Mode++- **`services/threatintel/app/actors/attribution.py`** — New+ `ThreatActorAttributionEngine` scores observed IOCs, MITRE ATT&CK+ techniques, tools, and target sectors against an in-memory catalog of+ three seed actor profiles (APT28, APT29, Lazarus). Scoring is the+ weighted sum of TTP (0.4) / Tool (0.3) / Target (0.2) / IOC (0.1)+ components, multiplied by the actor profile's baseline confidence,+ then thresholded.+- **`services/threatintel/app/api/actor_attribution.py`** — New router+ mounted at `/api/v1/actors` with `POST /attribute`, `GET /profiles`,+ and `GET /profiles/{actor_id}`. Constructs the engine once via+ FastAPI lifespan and passes it through `Depends(get_attribution_engine)`.+- **`services/agents/app/agents/investigation_agent.py`** — Investigation+ agent now calls `POST /actors/attribute` and surfaces attribution results+ in the investigation ledger.+- **`docker-compose.airgap.yml`** — Compose override for fully disconnected+ deployments: disables all external feed pullers, enables Ollama sidecar, and+ sets `AIRGAP_MODE=true` so the API switches to local-only LLM routing.+- **`apps/docs/docs/operations/air-gapped.md`** — Step-by-step air-gap+ deployment guide: image pre-pulling, Ollama model loading, threat-feed+ pre-seeding, and smoke-test checklist.++#### WS-H1 — MSSP Console Improvements++- **`services/api/app/api/v1/endpoints/mssp.py`** — New `GET /mssp/tenants`+ aggregation endpoint: per-child tenant alert counts, open case counts, SLA+ breach rate, and last-seen connector heartbeat.+- **`services/api/app/models/tenant.py`** — Added `parent_tenant_id` and+ `mssp_role` columns supporting the parent-child tenant hierarchy.++#### WS-H2 — BYOK Per-Tenant LLM Credentials++- **`services/api/app/api/v1/endpoints/llm_credentials.py`** — CRUD for per-tenant+ LLM credential records. Secrets encrypted at rest via `CredentialVault`.+- LLM routing layer (`services/api/app/core/config.py`) reads per-tenant+ credentials before falling back to the platform-wide key.++#### WS-H3 — Team Analytics View++- **`apps/web/src/components/analytics/TeamAnalyticsView.tsx`** — Analyst+ leaderboard with MTTR per analyst, alert disposition accuracy, cases closed+ per shift, and false-positive rate trend over the selected window.++#### WS-H4 — Air-Gapped / Ollama Local-LLM Mode++- **`services/api/app/api/v1/endpoints/llm_status.py`** — Reports whether the+ deployment is running in air-gap mode and which local models are available+ via the Ollama sidecar. Used by the settings UI to auto-populate the model+ picker.++### Fixed++- Ruff `E501/W291/W293/B007/B017/F821/I001` violations in `services/api`.+- `mypy` errors across all 16 plan-modified files: `RowMapping` import,+ `Optional` list `len()`, `current_user.user_id` rename, `fetchone()` None+ checks, `sort_key` return type, `PYTHONPATH` subprocess handling.+- Converted structlog-style `logger.info(key=value)` calls to stdlib formatting+ in `rule_engine.py`, `neo4j.py`, and `digest_pdf.py`.+- SQLAlchemy relationship `name-defined` mypy errors suppressed with+ `# type: ignore[name-defined]` in `tenant.py` and `connector.py`.++### Security caveat++The `/api/v1/actors/*` endpoints are reachable on the `threatintel`+service without RBAC enforcement in v0 — they assume cluster-internal+network reachability only. Do **not** expose them through public+ingress until a `Depends(require_permission(...))` guard is added.+Tracked as a known limitation in the docs.++---### Added — Threat Actor Attribution Engine (v0)diff --git a/README.md b/README.mdindex 192ab3022..6e86303a3 100644--- a/README.md+++ b/README.md@@ -9,7 +9,7 @@ An open-source, self-hostable AI SOC. The agent's prompts, tool calls, and ratio[](https://opensource.org/licenses/MIT)[](apps/docs/docs/benchmark.md)[](CONTRIBUTING.md)-[](CHANGELOG.md)+[](CHANGELOG.md)[Live demo](https://tryaisoc.com) · [How AiSOC compares](#how-aisoc-compares) · [Public eval harness](apps/docs/docs/benchmark.md) · [Deploy in 60 seconds](#deploy-in-60-seconds) · [Deployment options](#deployment-options) · [Architecture](#architecture) · [Docs](apps/docs/)@@ -162,21 +162,25 @@ AiSOC bundles the components a SOC normally pieces together from separate vendor- **Phishing triage workflow** — submit raw email text, URLs, attachments, or domain indicators (`POST /phishing/submit`); the LLM extracts IOCs, assigns a verdict (phishing / benign / spam / malware / unknown), maps to MITRE ATT&CK, and optionally links the submission to an existing case.- **Knowledge-base + RAG** — ingest runbooks, policies, SOPs, and wikis (`POST /kb/ingest`); the API chunks and full-text indexes each document; analysts query with natural language (`POST /kb/query`) and receive the top matching chunks plus an LLM-synthesised answer with citation, backed by PostgreSQL FTS when no vector store is configured.-### v1.5 — market-driven additions (2026-05-07)--- **Autonomous alert triage agent** — a dedicated LLM agent in `services/agents/app/agents/auto_triage_agent.py` classifies incoming alerts as `true_positive`, `false_positive`, or `benign` with confidence scores, auto-closing low-confidence noise and escalating the rest. Backed by four sibling agents — phishing, identity, cloud, and insider-threat — each tuned to its domain.-- **Conversational investigation chat** — multi-turn chat surface at `/investigate` lets analysts ask follow-up questions over the case, evidence, and ledger context without round-tripping through new tickets. Component: `apps/web/src/components/copilot/InvestigationChat.tsx`.-- **MITRE ATT&CK coverage advisor** — surfaces the highest-impact gaps in detection coverage (technique-by-technique) and recommends rules to close them, ranked by adversary prevalence and risk. Page: `/coverage-advisor`.-- **Shift handoff dashboard** — outgoing/incoming analysts see active cases, in-flight investigations, and queued approvals on one screen. Endpoints in `services/api/app/api/v1/endpoints/shifts.py`. Page: `/shifts`.-- **EASM (External Attack Surface Management)** — asset discovery, exposed-service inventory, and certificate-expiry monitor for everything the org has accidentally pointed at the public internet. Page: `/easm`.-- **MSSP executive dashboard** — multi-tenant rollup: KPIs, cross-tenant alert volumes, and per-customer SLA posture in one pane. Page: `/mssp`.-- **Alert noise tuning dashboard** — per-rule false-positive rate, suppression candidates, and one-click tuning suggestions. Page: `/noise-tuning`.-- **Team analytics & gamification** — analyst leaderboard, MTTR per analyst, dispositions accuracy, and shift workload balance. Page: `/analytics/team`.-- **STIX 2.1 / TAXII 2.1 publishing** — push the tenant's IOCs and threat-actor profiles to upstream / community feeds via `services/api/app/api/v1/endpoints/stix_taxii.py`.-- **Automated compliance evidence** — point-in-time evidence collection for SOC 2 / ISO 27001 / NIST CSF / PCI-DSS / HIPAA / DORA via `services/api/app/api/v1/endpoints/compliance.py`; surfaces in the Compliance dashboard.-- **AI-generated incident reports** — one-click "Export Report" on every case generates a PDF incident report from the Investigation Ledger.-- **Ten new connectors** — SentinelOne, Cortex XDR, Wiz, Snyk, Zscaler, Proofpoint, ServiceNow, Jira, 1Password, and Duo Security; bringing the total catalog to 26.-- **Air-gap deployment configuration** — `services/api/app/api/v1/endpoints/deployment.py` exposes air-gap mode toggles for tenants that disallow external feeds.+### v7.0 — buyer-value plan (2026-05-10)++Shipped by Beenu Arora <[email protected]>. All 16 workstreams:++- **Slack ChatOps bot** — `/aisoc triage`, `/aisoc approve`, `/aisoc status`, `/aisoc summary` slash commands + interactive approval buttons. Human-in-the-loop gate works from Slack without opening the console. 61 pytest cases. (`services/slack-bot/`)+- **Executive digest PDF** — branded A4 PDF with KPI tiles, alert-volume chart, top-rule table, and remediation summary. Auto-emailed Monday 06:00 UTC via APScheduler. (`services/api/app/services/digest_pdf.py`, `weekly_digest_task.py`)+- **AI investigation timeline (replayable)** — 684-line React component rendering the Investigation Ledger as a playable step-by-step timeline with scrubber. (`apps/web/src/components/copilot/InvestigationTimeline.tsx`)+- **Case auto-summary + PDF export** — LLM-powered structured case summary (headline, severity rationale, recommended action, evidence links). (`case_summary.py`, `case_summary_html.py`)+- **Playbook gallery** — 12 curated packs (Phishing, Ransomware, BEC, IAM Key Compromise, …) with TTP coverage badges and one-click import. 25 YAML templates added under `detections/playbooks/`.+- **GitHub PR integration** — detection proposals automatically create draft PRs against the tenant's detection repo when promoted. (`services/api/app/services/github.py`)+- **BYOK per-tenant LLM credentials UI** — provider picker (OpenAI, Azure OpenAI, Anthropic, Ollama), API-key input, model selector, temperature slider, and connection test. (`SettingsView.tsx`, `llm_credentials.py`)+- **WCAG AA accessibility** — axe-core CI gate covers 5 views + 3 modals; sidebar landmark roles, ARIA labels, focus trapping, skip-nav link, colour-contrast fixes. (`apps/web/src/test/a11y.test.tsx`)+- **Light theme persisted in user profile** — stored in `localStorage`, synced to `PATCH /api/v1/users/me/preferences`. (`ThemeProvider.tsx`)+- **Saved views + drag-drop dashboard widgets** — widgets can be dragged, dropped, resized, pinned, removed; layout serialised to `POST /api/v1/saved-views`. (`DashboardView.tsx`, `saved_views.py`)+- **Threat actor attribution engine v0** — weighted IOC / TTP / Tool / Target scoring against three seed actor profiles (APT28, APT29, Lazarus). `POST /actors/attribute`. (`services/threatintel/app/actors/attribution.py`)+- **Air-gap / Ollama local-LLM mode** — `docker-compose.airgap.yml` override; disables external feed pullers, enables Ollama sidecar; step-by-step deployment guide in docs. (`docker-compose.airgap.yml`, `apps/docs/docs/operations/air-gapped.md`)+- **MSSP console improvements** — `GET /mssp/tenants` aggregation: per-child alert counts, open cases, SLA breach rate, last-seen connector heartbeat; `parent_tenant_id` + `mssp_role` added to `Tenant` model.+- **Team analytics view** — analyst leaderboard, MTTR per analyst, cases closed per shift, FP rate trend. (`TeamAnalyticsView.tsx`)+- **Air-gap LLM status endpoint** — reports whether air-gap mode is active and which Ollama models are available; drives the settings UI model picker. (`llm_status.py`)Everything ships under MIT. Fork it, self-host it, audit it, extend it.diff --git a/ROADMAP.md b/ROADMAP.mdindex 58d4e4134..abf107526 100644--- a/ROADMAP.md+++ b/ROADMAP.md@@ -168,15 +168,41 @@ SOAR platforms drove this release.----## v7.0 — Planned+## v7.0 — Shipped ✅ (2026-05-10)++All items below were shipped as part of the v1.0 buyer-value plan.+Implemented and reviewed by Beenu Arora <[email protected]>.++- [x] WCAG AA full accessibility pass (axe-core CI gate — `apps/web/src/test/a11y.test.tsx`)+- [x] Light theme persisted in user profile (`ThemeProvider.tsx` + `PATCH /api/v1/users/me/preferences`)+- [x] Saved views and custom drag-drop dashboard widgets per analyst (`saved_views.py` + `DashboardView.tsx`)+- [x] AI-generated weekly executive digest — auto-emailed PDF (`digest_pdf.py` + `weekly_digest_task.py`)+- [x] Slack native bot for alert triage without opening the UI (`services/slack-bot/` — 61 tests)+- [x] Threat actor attribution engine v0 (`services/threatintel/app/actors/attribution.py`)+- [x] Air-gap / Ollama local-LLM mode (`docker-compose.airgap.yml` + `apps/docs/docs/operations/air-gapped.md`)+- [x] BYOK per-tenant LLM credentials UI + API (`llm_credentials.py` + `SettingsView.tsx`)+- [x] MSSP console — per-child-tenant KPI aggregation, SLA posture, parent_tenant_id hierarchy+- [x] Team analytics view — analyst MTTR, leaderboard, shift workload (`TeamAnalyticsView.tsx`)+- [x] Case auto-summary + PDF export (`case_summary.py` + `case_summary_html.py`)+- [x] Investigation timeline (replayable) (`InvestigationTimeline.tsx`)+- [x] Playbook gallery with 12 curated packs + GitHub PR integration for detection proposals+- [ ] Mobile responder console (React Native) — triage and acknowledge from phone _(deferred to v8.0)_+- [ ] Plugin publishing marketplace v3 (commercial plugins, revenue sharing) _(deferred to v8.0)_++---++## v8.0 — Planned- Mobile responder console (React Native) — triage and acknowledge from phone-- WCAG AA full accessibility pass-- Light theme + brand-configurable white-label mode-- Saved views and custom dashboard widgets per analyst-- AI-generated weekly executive digest (auto-emailed PDF)-- Slack / Teams native bot for alert triage without opening the UI- Plugin publishing marketplace v3 (commercial plugins, revenue sharing)+- MSSP RBAC enforcement on `/api/v1/actors/*` endpoints (threat attribution)+- Automated IOC sharing to community MISP instances via STIX/TAXII push+- NL→query: "show me failed logins from new ASNs last 24h" → ES|QL / KQL+- AI-generated threat intelligence briefings from public feeds+- Embedded red-team scoring (ATT&CK coverage %) as a live dashboard widget+- SLA breach predictor (ML model on historical MTTR data)+- Incident cost estimator (breach impact calculator)+- SOC-in-a-box one-click cloud deploy (Terraform module for AWS / GCP)---diff --git a/apps/docs/docs/intro.md b/apps/docs/docs/intro.mdindex 307040370..25270f548 100644--- a/apps/docs/docs/intro.md+++ b/apps/docs/docs/intro.md@@ -41,7 +41,13 @@ public, reproducible eval harness on every PR targeting `main` / `develop`.- **Hunt-as-Code** — YAML hunt definitions in `hunts/` with hypothesis, indicator matching (equals/in/regex/gte/lte/exists/contains_any/iendswith), and APScheduler-driven continuous execution. Results flow to Postgres via the Hunt API.- **Risk-Based Alerting (RBA)** — alerts contribute time-decayed risk points to entities (user, host, IP, domain). When an entity's score crosses a configurable threshold, AiSOC promotes it to an incident with contributing alerts attached.- **Federated search** — translate a single query into SPL, KQL, and ES|QL and fan out to connected SIEMs. Results are merged and deduplicated.+- **Slack ChatOps bot** — dedicated `aisoc` slash commands (`/aisoc triage`, `/aisoc approve`, `/aisoc status`, `/aisoc summary`) with interactive approval buttons and HMAC-signed callbacks. Human-in-the-loop triage without opening the console. (`services/slack-bot/`)- **ChatOps verification** — Slack/Teams interactive prompts with HMAC-signed callback choices (acknowledge / deny / escalate) for human-in-the-loop response actions.+- **AI executive digest PDF** — branded A4 PDF with KPI tiles, alert-volume chart, top-rule table, and remediation summary, auto-emailed Monday 06:00 UTC via APScheduler. (`services/api/app/services/digest_pdf.py`)+- **Threat actor attribution engine** — per-alert actor attribution with MITRE Group mapping, confidence scoring, Diamond Model labeling, and campaign clustering. (`services/threatintel/app/actors/attribution.py`)+- **Air-gap / local-LLM mode** — run AiSOC with zero outbound HTTP using Ollama, LiteLLM, or vLLM via a single env-var toggle and a Docker Compose overlay. ([Air-gapped mode](./operations/air-gapped))+- **BYOK — Bring Your Own LLM key** — per-tenant LLM credential management (API key, base URL, model) stored encrypted in `CredentialVault`; configurable from the Settings UI. ([Credentials](./operations/credentials))+- **Saved views & drag-drop dashboard widgets** — analysts save custom alert filter presets and rearrange dashboard widgets; persisted per-user in Postgres.- **Playbook engine** — 50+ community SOAR playbooks with explicit decision trees and human-approval gates on destructive actions.- **Threat intelligence** — TAXII 2.1, MISP, OTX, CISA KEV with triple storage (search, vector, graph).- **Governance** — SAML 2.0 + OIDC SSO, multi-tenant RLS, granular RBAC, immutable audit log.