Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
fix: address maintainer review blockers
Alexandru Mateescu240ecbf484c5
diff --git a/.env.bak-202610021531 b/.env.bak-202610021531deleted file mode 100644index 3f6df55c1..000000000--- a/.env.bak-202610021531+++ /dev/null@@ -1,455 +0,0 @@-# AiSOC Environment Variables-# Copy this file to .env and fill in your values-#-# ╔════════════════════════════════════════════════════════════════════════════╗-# ║ YOU DO NOT HAVE TO EDIT THIS FILE TO GET A WORKING STACK. ║-# ║ ║-# ║ make up creates .env from this template and fills the four ║-# ║ generated secrets — AISOC_CREDENTIAL_KEY, SECRET_KEY, ║-# ║ AISOC_SERVICE_TOKEN, AISOC_ACTIONS_SERVICE_TOKEN — with ║-# ║ real random values. ║-# ║ make env does only that, if you want to look before you start. ║-# ║ ║-# ║ Those three ship EMPTY here rather than as `replace-me-…` prose. A ║-# ║ placeholder is not a neutral default: a non-empty AISOC_CREDENTIAL_KEY ║-# ║ that is not a valid Fernet key made every connector save answer HTTP 500, ║-# ║ so copying this template used to break the product while skipping the copy ║-# ║ left it working. `scripts/check_env_placeholders.py` now fails on any ║-# ║ placeholder shipped here, and `make doctor` reports them in your .env. ║-# ║ ║-# ║ Optional, and worth setting: ║-# ║ OPENAI_API_KEY a hosted provider key for better AI triage. CORE ║-# ║ ships a local model, so this is an upgrade, not a ║-# ║ requirement — see "AI / LLM" below. ║-# ║ POSTGRES_PASSWORD the dev default works out of the box; change it for ║-# ║ anything shared. ║-# ╚════════════════════════════════════════════════════════════════════════════╝--# ─── Image Pinning ────────────────────────────────────────────────────────────-# Tag pulled by every `image: ghcr.io/beenuar/aisoc-*:${AISOC_VERSION:-latest}`-# in docker-compose.yml. Docker pulls this tag on first boot and skips the-# local build (~90 s vs ~15 min). Pin to a release tag (e.g. v12.0.0) for-# reproducible deploys; leave as `latest` to track the newest release.-AISOC_VERSION=latest--# What `pull_policy:` resolves to for every first-party image. `missing` means-# "use what is local if there is one", which is right for an immutable release-# tag and wrong for `latest`, which is republished on every merge — so-# `make up` refreshes the images itself whenever AISOC_VERSION names a moving-# tag, and skips it when the tag is pinned.-#-# Set this to `never` on an air-gapped host: Compose then refuses to contact a-# registry, and `make up` skips its refresh for the same reason. `always` makes-# Compose itself re-check on every container start.-# AISOC_PULL_POLICY=never--# ─── Core Configuration ───────────────────────────────────────────────────────-# `development` enables the dev auth bypass in services/api/app/api/v1/dev_auth.py.-# Set `production` for anything network-reachable; docker-compose.yml reads this-# value, so changing it here is enough.-# Production is the default because the value of this variable used to-# decide whether an uncredentialed request was served as an administrator.-# For the developer conveniences, layer the overlay rather than editing-# this line: docker compose -f docker-compose.yml -f infra/compose/docker-compose.dev.yml up -d-ENVIRONMENT=production-LOG_LEVEL=info--# HS256 signing key for console sessions, shared by the API and the connectors-# service. Left empty here and generated by `scripts/ensure_env.py` — see the-# note on AISOC_CREDENTIAL_KEY below for why a placeholder is not used.-SECRET_KEY=1fd76079684f5b19cb24294429815bcbd570131b30f65d89edf6368b07e8b7d0--# Base URL `make bootstrap` prints as the sign-in address, and the origin the-# console is reached on. Leave it at the default for a laptop; set it to the-# address operators actually browse to for anything else, or the printed-# credential arrives with the wrong URL beside it.-AISOC_CONSOLE_URL=http://localhost:3000--# ─── Where the stack is reachable from ────────────────────────────────────────-# Every host port is published to 127.0.0.1 by default, which is right for a-# laptop and fatal for a server: nothing outside the machine can reach it,-# including the browser you meant to use.-#-# Set this to publish the console on all interfaces. It is the only port a-# browser needs — the console proxies to the API, the agents service and the-# realtime gateway over the internal network — so the datastores and the-# development passwords in this file stay on loopback.-#-# AISOC_CONSOLE_BIND_ADDR=0.0.0.0-#-# Put a TLS-terminating reverse proxy in front before exposing it to a network-# you do not control. See apps/docs/docs/deployment/single-host.md.-AISOC_CONSOLE_BIND_ADDR=0.0.0.0--# Moves EVERY published port, datastores included. Only for an isolated-# network: it hands Postgres, Redis, Kafka and Neo4j — with the development-# passwords in this file — to anything that can route to the host.-# AISOC_BIND_ADDR=0.0.0.0--# Whether this deployment is a demo. Read at run time by both the API and the-# console, and reported at GET /api/runtime-config so it can be checked from-# outside rather than inferred from the page. Leave false unless you are-# running a public demo; `make demo` is the opt-in that loads synthetic data.-AISOC_DEMO_MODE=false--# Shared HS256 secret used ONLY for short-lived realtime WS/SSE tickets-# (Issue #239). The API mints tickets at `POST /api/v1/realtime/ticket` and the-# Node realtime service (`services/realtime`) verifies them on every WebSocket-# upgrade and SSE request — both sides MUST read the same value. This is-# intentionally separate from SECRET_KEY so the realtime edge can be rotated-# independently and a leaked ticket can never forge a full API session.-#-# `make up` generates this. There is no development fallback any more: both-# sides used to fall back to a constant committed to this repository, and no-# manifest passed either the real secret or the variable the production check-# read, so that constant was the effective key in every deployment. Unset, the-# ticket endpoint fails closed (503) and realtime rejects every connection.-AISOC_REALTIME_JWT_SECRET=U_XjLpDZdXAj8zOYhL5Hv8eN3UOeTkUD6Ws_0dPsDLk--# HS256 key for the session tokens the SAML and OIDC callbacks issue, and the-# bearer that guards /metrics. Both are generated by `make env`. The API-# refuses to boot without them now that ENVIRONMENT defaults to production:-# in a development-class environment `enforce_secure_defaults` only warns,-# which is why neither was generated before.-JWT_SECRET=6zbpUL_SA81S17OmYi56IWMCyUBtNJKBigUqsCmyC9E-METRICS_TOKEN=7cUq7f5gLU-8rrVkd8b0bY8cZsMT2a4hu0qExcwe1HI-# Time-to-live (seconds) for minted realtime tickets. Kept short so a leaked-# ticket is only briefly useful; the frontend re-mints on every (re)connect.-# Clamped to a 60s ceiling by the ticket endpoint regardless of this value.-AISOC_REALTIME_TICKET_TTL_SECONDS=60--# Application-layer encryption key for connector credentials and other-# secret payloads stored in Postgres.-#-# LEAVE THIS EMPTY. `make up`, `make env` and `./install.sh` all run-# `scripts/ensure_env.py`, which writes a freshly generated key here the first-# time. Generating it by hand works too:-# python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"-#-# It ships empty rather than as a `replace-me-…` string on purpose. A non-empty-# value that is not a valid Fernet key is *worse* than no value: the vault takes-# its development path only when the key is empty, so a placeholder reached-# `Fernet()`, raised, and turned every connector save into an HTTP 500 —-# minutes after `cp .env.example .env`, with nothing linking the two. Empty is a-# value the vault understands.-#-# Treat a real value like a database password: never commit it, never reuse it-# across environments, never share it.-AISOC_CREDENTIAL_KEY=n-NAjH2x0NgryRcZrvSrQH-rp66MsI1HxDQr09j6L5w=-# Comma-separated list of previous credential keys to support zero-downtime-# rotation. Leave blank unless you are mid-rotation.-AISOC_CREDENTIAL_KEY_ROTATION_FROM=--# Internal URL the API uses to reach the connectors microservice for-# catalog lookups and "Test connection" actions in the click-and-connect-# wizard. ``http://connectors:8003`` is correct for docker-compose;-# use ``http://localhost:8088`` when running the API on the host while-# connectors is exposed via the docker-compose port forward.-CONNECTORS_SERVICE_URL=http://connectors:8003-CONNECTORS_SERVICE_TIMEOUT_SECONDS=15--# ─── Database ─────────────────────────────────────────────────────────────────-# Two roles, and the distinction is load-bearing rather than tidy.-#-# aisoc owner. Superuser (the postgres image makes POSTGRES_USER one),-# owns every table, applies the migration chain. No service-# connects as it.-# aisoc_app runtime. SELECT / INSERT / UPDATE / DELETE and nothing else, so-# that the schema's row-level-security policies apply to it. A-# superuser ignores those policies even under FORCE ROW LEVEL-# SECURITY, which is why running the services as `aisoc` left all-# 92 of them filtering nothing.-#-# POSTGRES_PASSWORD is the owner's; AISOC_APP_DB_PASSWORD is the runtime role's.-# Both are interpolated by docker-compose.yml, so changing them here rotates-# them across the stack. They are deliberately different values: they are-# different privilege levels.-#-# Empty here and generated by `scripts/ensure_env.py`, like the secrets-# above. This file used to ship `POSTGRES_PASSWORD=aisoc_dev_secret`, and-# `docker-compose.prod.yml` guarded it with-# `${POSTGRES_PASSWORD:?… the development default is a published literal}` —-# a guard that rejects an *empty* variable and cannot look at a value, so the-# one input its message named was the one input it accepted. A production-# stack booted on a password printed in this repository while the file said-# it could not. `preflight-secrets` in that file now refuses the value, and-# shipping these empty means a fresh install never has one to refuse.-POSTGRES_PASSWORD=440c43cf18f78e66a2885f12480c7551c934331b5a88eedc-AISOC_APP_DB_PASSWORD=03b5366c74c98fde62b1656d433f367e7cff75035d82cf4c--# What every service connects as.-DATABASE_URL=postgresql+asyncpg://aisoc_app:${AISOC_APP_DB_PASSWORD}@localhost:5432/aisoc--# DDL only: `python -m app.scripts.run_migrations` uses this and nothing else-# does. Leave it unset and migrations fall back to DATABASE_URL, which is-# correct for a deployment that has not split the roles and fails loudly-# ("permission denied for schema public") on one that has.-# aisoc-db-role: owner — the migration chain applies DDL; the runtime role holds none-DATABASE_MIGRATION_URL=postgresql+asyncpg://aisoc:${POSTGRES_PASSWORD}@localhost:5432/aisoc--# Generated alongside the Postgres pair, and referenced rather than repeated:-# these two used to spell the password out inside the DSN, so rotating-# REDIS_PASSWORD left a stale copy here that still worked.-REDIS_PASSWORD=12b5ff3f51c1cbb8a68b887b73ff3368cc3b30a4fa9b693d-CLICKHOUSE_PASSWORD=61dd17f85342f5f3a4850dab1da5a13a4c1bd682dd3dc70b-REDIS_URL=redis://:${REDIS_PASSWORD}@localhost:6379/0-CLICKHOUSE_URL=http://aisoc:${CLICKHOUSE_PASSWORD}@localhost:8123/aisoc--# ─── Messaging ────────────────────────────────────────────────────────────────-KAFKA_BOOTSTRAP_SERVERS=localhost:9092--# ─── Search ───────────────────────────────────────────────────────────────────-OPENSEARCH_URL=http://localhost:9200-QDRANT_URL=http://localhost:6333--# ─── AI / LLM ─────────────────────────────────────────────────────────────────-# Empty, and empty is a working configuration: CORE ships Ollama with a small-# local model, so a default install does real AI triage with no key at all. Set-# a hosted provider key here to upgrade the quality — the gateway picks it up-# and routes the aliases to the hosted model instead.-#-# It is empty rather than `sk-your-…-here` because a bogus key is not the same-# as no key: the gateway would send it upstream, get a 401, and the deployment-# would look configured while producing nothing.-OPENAI_API_KEY=-# OPENAI_MODEL affects the "explain this alert" / BYOK path only. Task workloads-# (triage, recon, investigation, copilot, summary, report, NL) request a logical-# alias instead — see the LLM gateway block below.-#-# It ships as an alias, not a concrete model, because the default deployment-# routes through the bundled gateway and the gateway 400s anything-# infra/litellm/config.yaml does not define. It used to read-# `gpt-4-turbo-preview`, which no gateway config here has ever defined, and the-# auto-triage worker treated it as a per-tenant BYOK override — so it replaced-# `aisoc-triage` on the highest-volume path in the product.-# scripts/check_llm_model_routing.py fails the build if this names a model the-# gateway does not define. Set a concrete model only when calling a provider-# directly (see "ENABLE LIVE LLM" option (b) below).-OPENAI_MODEL=aisoc-summary--# ─── LLM gateway (LiteLLM) — issue #478 ───────────────────────────────────────-# Since #478 AiSOC asks for a logical task alias (aisoc-triage,-# aisoc-investigation, …) for every live LLM call — there is no hardcoded default-# model. The alias → real-model map lives in infra/litellm/config.yaml, so you-# swap local/hosted models there without changing AiSOC. The gateway service is-# defined in docker-compose.yml.-#-# Key that AiSOC uses to authenticate to the gateway (also LiteLLM's admin key):-LITELLM_MASTER_KEY=sk-aisoc-local-# In-network URL of the gateway. docker-compose.yml sets this on the api and-# agents services, and both resolvers READ it — `resolve_base_url()` returns it-# for any aisoc-<role> alias, and `resolve_api_key()` returns LITELLM_MASTER_KEY-# to pair with it. That wire was missing: compose set this variable, the-# resolvers deliberately ignored it in favour of OPENAI_BASE_URL (which nothing-# set), and so the product's headline capability could not reach a model in the-# default deployment.-LLM_GATEWAY_URL=http://litellm:4000/v1-#-# ENABLE LIVE LLM:-# (a) Through the gateway — the default, and already wired. Put your real-# provider key in OPENAI_API_KEY above (the gateway reads it) and bring up-# the `full` profile, which is where the litellm service lives:-# docker compose --profile full up -d-# Nothing else to set. To point at a gateway somewhere else, or to send a-# different bearer, set OPENAI_BASE_URL/OPENAI_API_KEY and they win.-# (b) Direct to a provider, no gateway — pin each task role to a concrete-# model. A concrete pin is NOT rerouted to the gateway, so this keeps-# working with LLM_GATEWAY_URL set:-# AISOC_MODEL_PIN_TRIAGE=gpt-4o-mini-# AISOC_MODEL_PIN_INVESTIGATION=gpt-4o-# OPENAI_MODEL=gpt-4o-mini # the BYOK/explain path, direct too-# With neither configured, AiSOC uses its deterministic offline path — which-# reports zero model calls and zero tokens rather than pretending to triage.--# ─── Threat Intelligence ──────────────────────────────────────────────────────-# Open-source / community feeds (work out of the box, no key required for some)-TAXII_FEEDS=https://cti-taxii.mitre.org/taxii/,enterprise-attack,,-MISP_URL=-MISP_API_KEY=-OTX_API_KEY=-CISA_KEV_ENABLED=true-ABUSE_CH_ENABLED=true--# Free / freemium enrichment (recommended for everyone)-VIRUSTOTAL_API_KEY=-ABUSEIPDB_API_KEY=-GREYNOISE_API_KEY=-SHODAN_API_KEY=-URLSCAN_API_KEY=-IPINFO_API_TOKEN=--# ─── Commercial threat intelligence (optional) ────────────────────────────────-# Cyble Vision — actor, malware, vulnerability, dark-web, brand-risk feeds-# Provision a key from https://cyble.com/cyble-vision/-CYBLE_VISION_API_KEY=-CYBLE_VISION_BASE_URL=https://api.cyble.com/v2-CYBLE_VISION_TENANT_ID=-CYBLE_VISION_FEEDS=indicators,actors,vulnerabilities,leaks,brand-CYBLE_HUNT_API_KEY=--# Recorded Future-RECORDED_FUTURE_API_KEY=--# Mandiant Threat Intelligence (formerly FireEye iSIGHT)-MANDIANT_API_KEY=-MANDIANT_API_SECRET=--# Crowdstrike Falcon Intelligence (separate from the EDR connector)-CROWDSTRIKE_INTEL_CLIENT_ID=-CROWDSTRIKE_INTEL_CLIENT_SECRET=--# Anomali ThreatStream-ANOMALI_USERNAME=-ANOMALI_API_KEY=-ANOMALI_BASE_URL=https://api.threatstream.com--# IBM X-Force Exchange-XFORCE_API_KEY=-XFORCE_API_PASSWORD=--# Flashpoint-FLASHPOINT_API_KEY=--# Intel 471-INTEL471_USERNAME=-INTEL471_API_KEY=--# DomainTools Iris-DOMAINTOOLS_USERNAME=-DOMAINTOOLS_API_KEY=--# RiskIQ / Microsoft Defender External ATP-RISKIQ_USERNAME=-RISKIQ_API_KEY=--# ─── Connectors ───────────────────────────────────────────────────────────────-# CrowdStrike Falcon-CROWDSTRIKE_CLIENT_ID=-CROWDSTRIKE_CLIENT_SECRET=-CROWDSTRIKE_BASE_URL=https://api.crowdstrike.com--# Splunk-SPLUNK_HOST=-SPLUNK_PORT=8089-SPLUNK_TOKEN=-SPLUNK_SCHEME=https-SPLUNK_VERIFY_SSL=true--# AWS Security Hub-AWS_REGION=us-east-1-AWS_ACCESS_KEY_ID=-AWS_SECRET_ACCESS_KEY=--# Okta-OKTA_DOMAIN=-OKTA_API_TOKEN=--# Microsoft Sentinel-AZURE_TENANT_ID=-AZURE_CLIENT_ID=-AZURE_CLIENT_SECRET=-AZURE_SUBSCRIPTION_ID=-AZURE_RESOURCE_GROUP=-AZURE_WORKSPACE_NAME=--# ─── Frontend ─────────────────────────────────────────────────────────────────-# Where the console proxies browser requests. The browser itself never uses-# these: the bundle calls same-origin paths like /api/v1/alerts and the Next.js-# server forwards them, which is why one image works on localhost, on a LAN-# address and behind a reverse proxy without rebuilding, and why there is no-# CORS configuration to get wrong.-#-# The defaults are the Compose service names and are right for the bundled-# stack. Change them when the API lives somewhere else — a different Compose-# project, another host, or an existing deployment.-AISOC_API_URL=http://api:8000-AISOC_AGENTS_URL=http://agents:8084-AISOC_REALTIME_URL=http://realtime:4000--# NEXT_PUBLIC_* is NOT listed here on purpose.-#-# Next.js inlines every NEXT_PUBLIC_* value into the JavaScript bundle when the-# image is *built*, so setting one in this file has no effect on the image-# `docker compose up` pulls. This file used to list NEXT_PUBLIC_API_URL,-# NEXT_PUBLIC_REALTIME_URL and NEXT_PUBLIC_WS_URL, which meant an operator-# pointing the console at their own host edited three variables, restarted,-# saw no change, and had nothing telling them the settings were inert. Use the-# three AISOC_*_URL values above; they are read at container start.-#-# To bypass the proxy and have the browser call another origin directly you-# have to rebuild the image with --build-arg NEXT_PUBLIC_API_URL=… and take on-# the CORS configuration that comes with it.--# Override the docs site base URL (default: https://beenuar.github.io/AiSOC/docs).-# Set this when serving docs from a custom domain like https://docs.tryaisoc.com/docs.-# NEXT_PUBLIC_DOCS_URL=https://docs.tryaisoc.com/docs--# ─── Service-to-service auth ──────────────────────────────────────────────────-# Shared bearer token for backend services that are NOT reachable from the-# browser (connectors, purple-team, honeytokens, ueba). Those surfaces execute-# privileged work — purple-team can start adversary-emulation operations against-# live hosts — and they fail closed: with no token configured and AISOC_DEV_MODE-# unset, they answer 503 rather than serving the route openly.-#-# Left empty here and generated by `scripts/ensure_env.py`. It is not optional-# in practice: the API presents it when it proxies the connector catalog and-# "Test connection" to the connectors service, so a blank value on one side and-# a configured secret on the other is a 401 the wizard used to report as the-# bare string "Connection test failed".-#-# `ingest` and `connectors` use it for the same purpose on the push path, and-# there AISOC_DEV_MODE does NOT cover it: POST /v1/ingest is a write into-# tenant-owned data and has no dev-mode shortcut. A direct curl/script push-# uses a per-tenant token from `make ingest-token` instead and needs nothing-# here. See apps/docs/docs/operations/ingest-authentication.md.-AISOC_SERVICE_TOKEN=t0K34vZQE5T0MpeicQyZp4w6Js_8eISs38Kd5WMYy2c-#-# The actions service gets its own, because it is the one that can isolate a-# host or disable an account and should not share a credential with the-# read-mostly services above. `make up` generates it. Without it the actions-# service answers 503 on every mutating route, which is deliberate: it used to-# answer 200 to anyone who could reach the port.-AISOC_ACTIONS_SERVICE_TOKEN=B0N61vx6KgsmcH3UoZuKtIgd5Ot2uI_djKu1M-qiB6A-#-# The agents service gets its own too, and it is the one that could not fall-# back: eight modules in `services/agents` present it as-# `X-AiSOC-Service-Token` on the API's internal context routes, and-# `service_token_valid` on the API verifies THIS variable and no other — it-# does not accept AISOC_SERVICE_TOKEN. Nothing generated it and no manifest-# passed it, so organisation memory, tenant skills, recent dispositions,-# identity context, knowledge-base runbooks, the MCP toolset, the playbook-# action bridge and SIEM writeback were all off on every default install, each-# logging a `no_service_token` warning and returning an empty result that-# reads exactly like "this tenant has none".-#-# The same value is verified in the other direction: the API posts a replay to-# the agents service as a bearer, and `app.security.tenant_scope` resolves-# `AISOC_AGENTS_SERVICE_TOKEN` ahead of the shared one. `make up` generates it-# and compose passes it to both.-AISOC_AGENTS_SERVICE_TOKEN=sCn5w7PhzIJVvY5_5gIZZDlswDbqCxSe63Wd54fWZyM--# Bearer for realtime's /internal/* fan-out routes (API/agents -> realtime),-# sent as the x-internal-token header. `make up` generates it. The guard used-# to treat an unset value as authorized, and nothing set it.-REALTIME_INTERNAL_TOKEN=j3gjCbBj8yHEdE9BS8mQvlD0cR_waYn6Oc4YKzafuak-# Optional per-service overrides, if you'd rather not share one secret. Each-# is read ahead of AISOC_SERVICE_TOKEN by the service it names, on both the-# presenting and the verifying side, so setting one narrows a single hop.-# `scripts/check_service_token_wiring.py` asserts every one of these that-# production code reads is listed here — four were not, which is how an-# operator would go looking for a variable the code uses and find nothing.-# AISOC_PURPLE_TEAM_SERVICE_TOKEN=-# AISOC_HONEYTOKENS_SERVICE_TOKEN=-# AISOC_UEBA_SERVICE_TOKEN=-# AISOC_INGEST_SERVICE_TOKEN=-# AISOC_CONNECTORS_SERVICE_TOKEN=-# AISOC_FUSION_SERVICE_TOKEN=-# AISOC_THREATINTEL_SERVICE_TOKEN=-#-# Bearer the agents service presents to the ChatOps fan-out (slack-bot).-# Unset means no ChatOps notification is sent; the approval is still durable-# and visible in the console.-AISOC_INTERNAL_TOKEN=wXugJXuZwACNxpoQfeyNoV048qLTZQUfPNDz4UOUTfU--# Canonical agents -> core API credential (DB-backed aisoc_ key, minted 2026-10-02)-AISOC_AGENTS_API_KEY=aisoc_517c15dcbf96ea8ba64b64a92682ce276b3700e5ecb56ab4diff --git a/.env.bak.1790944801 b/.env.bak.1790944801deleted file mode 100644index 2ff5308f9..000000000--- a/.env.bak.1790944801+++ /dev/null@@ -1,452 +0,0 @@-# AiSOC Environment Variables-# Copy this file to .env and fill in your values-#-# ╔════════════════════════════════════════════════════════════════════════════╗-# ║ YOU DO NOT HAVE TO EDIT THIS FILE TO GET A WORKING STACK. ║-# ║ ║-# ║ make up creates .env from this template and fills the four ║-# ║ generated secrets — AISOC_CREDENTIAL_KEY, SECRET_KEY, ║-# ║ AISOC_SERVICE_TOKEN, AISOC_ACTIONS_SERVICE_TOKEN — with ║-# ║ real random values. ║-# ║ make env does only that, if you want to look before you start. ║-# ║ ║-# ║ Those three ship EMPTY here rather than as `replace-me-…` prose. A ║-# ║ placeholder is not a neutral default: a non-empty AISOC_CREDENTIAL_KEY ║-# ║ that is not a valid Fernet key made every connector save answer HTTP 500, ║-# ║ so copying this template used to break the product while skipping the copy ║-# ║ left it working. `scripts/check_env_placeholders.py` now fails on any ║-# ║ placeholder shipped here, and `make doctor` reports them in your .env. ║-# ║ ║-# ║ Optional, and worth setting: ║-# ║ OPENAI_API_KEY a hosted provider key for better AI triage. CORE ║-# ║ ships a local model, so this is an upgrade, not a ║-# ║ requirement — see "AI / LLM" below. ║-# ║ POSTGRES_PASSWORD the dev default works out of the box; change it for ║-# ║ anything shared. ║-# ╚════════════════════════════════════════════════════════════════════════════╝--# ─── Image Pinning ────────────────────────────────────────────────────────────-# Tag pulled by every `image: ghcr.io/beenuar/aisoc-*:${AISOC_VERSION:-latest}`-# in docker-compose.yml. Docker pulls this tag on first boot and skips the-# local build (~90 s vs ~15 min). Pin to a release tag (e.g. v12.0.0) for-# reproducible deploys; leave as `latest` to track the newest release.-AISOC_VERSION=latest--# What `pull_policy:` resolves to for every first-party image. `missing` means-# "use what is local if there is one", which is right for an immutable release-# tag and wrong for `latest`, which is republished on every merge — so-# `make up` refreshes the images itself whenever AISOC_VERSION names a moving-# tag, and skips it when the tag is pinned.-#-# Set this to `never` on an air-gapped host: Compose then refuses to contact a-# registry, and `make up` skips its refresh for the same reason. `always` makes-# Compose itself re-check on every container start.-# AISOC_PULL_POLICY=never--# ─── Core Configuration ───────────────────────────────────────────────────────-# `development` enables the dev auth bypass in services/api/app/api/v1/dev_auth.py.-# Set `production` for anything network-reachable; docker-compose.yml reads this-# value, so changing it here is enough.-# Production is the default because the value of this variable used to-# decide whether an uncredentialed request was served as an administrator.-# For the developer conveniences, layer the overlay rather than editing-# this line: docker compose -f docker-compose.yml -f infra/compose/docker-compose.dev.yml up -d-ENVIRONMENT=production-LOG_LEVEL=info--# HS256 signing key for console sessions, shared by the API and the connectors-# service. Left empty here and generated by `scripts/ensure_env.py` — see the-# note on AISOC_CREDENTIAL_KEY below for why a placeholder is not used.-SECRET_KEY=1fd76079684f5b19cb24294429815bcbd570131b30f65d89edf6368b07e8b7d0--# Base URL `make bootstrap` prints as the sign-in address, and the origin the-# console is reached on. Leave it at the default for a laptop; set it to the-# address operators actually browse to for anything else, or the printed-# credential arrives with the wrong URL beside it.-AISOC_CONSOLE_URL=http://localhost:3000--# ─── Where the stack is reachable from ────────────────────────────────────────-# Every host port is published to 127.0.0.1 by default, which is right for a-# laptop and fatal for a server: nothing outside the machine can reach it,-# including the browser you meant to use.-#-# Set this to publish the console on all interfaces. It is the only port a-# browser needs — the console proxies to the API, the agents service and the-# realtime gateway over the internal network — so the datastores and the-# development passwords in this file stay on loopback.-#-# AISOC_CONSOLE_BIND_ADDR=0.0.0.0-#-# Put a TLS-terminating reverse proxy in front before exposing it to a network-# you do not control. See apps/docs/docs/deployment/single-host.md.-AISOC_CONSOLE_BIND_ADDR=0.0.0.0--# Moves EVERY published port, datastores included. Only for an isolated-# network: it hands Postgres, Redis, Kafka and Neo4j — with the development-# passwords in this file — to anything that can route to the host.-# AISOC_BIND_ADDR=0.0.0.0--# Whether this deployment is a demo. Read at run time by both the API and the-# console, and reported at GET /api/runtime-config so it can be checked from-# outside rather than inferred from the page. Leave false unless you are-# running a public demo; `make demo` is the opt-in that loads synthetic data.-AISOC_DEMO_MODE=false--# Shared HS256 secret used ONLY for short-lived realtime WS/SSE tickets-# (Issue #239). The API mints tickets at `POST /api/v1/realtime/ticket` and the-# Node realtime service (`services/realtime`) verifies them on every WebSocket-# upgrade and SSE request — both sides MUST read the same value. This is-# intentionally separate from SECRET_KEY so the realtime edge can be rotated-# independently and a leaked ticket can never forge a full API session.-#-# `make up` generates this. There is no development fallback any more: both-# sides used to fall back to a constant committed to this repository, and no-# manifest passed either the real secret or the variable the production check-# read, so that constant was the effective key in every deployment. Unset, the-# ticket endpoint fails closed (503) and realtime rejects every connection.-AISOC_REALTIME_JWT_SECRET=U_XjLpDZdXAj8zOYhL5Hv8eN3UOeTkUD6Ws_0dPsDLk--# HS256 key for the session tokens the SAML and OIDC callbacks issue, and the-# bearer that guards /metrics. Both are generated by `make env`. The API-# refuses to boot without them now that ENVIRONMENT defaults to production:-# in a development-class environment `enforce_secure_defaults` only warns,-# which is why neither was generated before.-JWT_SECRET=6zbpUL_SA81S17OmYi56IWMCyUBtNJKBigUqsCmyC9E-METRICS_TOKEN=7cUq7f5gLU-8rrVkd8b0bY8cZsMT2a4hu0qExcwe1HI-# Time-to-live (seconds) for minted realtime tickets. Kept short so a leaked-# ticket is only briefly useful; the frontend re-mints on every (re)connect.-# Clamped to a 60s ceiling by the ticket endpoint regardless of this value.-AISOC_REALTIME_TICKET_TTL_SECONDS=60--# Application-layer encryption key for connector credentials and other-# secret payloads stored in Postgres.-#-# LEAVE THIS EMPTY. `make up`, `make env` and `./install.sh` all run-# `scripts/ensure_env.py`, which writes a freshly generated key here the first-# time. Generating it by hand works too:-# python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"-#-# It ships empty rather than as a `replace-me-…` string on purpose. A non-empty-# value that is not a valid Fernet key is *worse* than no value: the vault takes-# its development path only when the key is empty, so a placeholder reached-# `Fernet()`, raised, and turned every connector save into an HTTP 500 —-# minutes after `cp .env.example .env`, with nothing linking the two. Empty is a-# value the vault understands.-#-# Treat a real value like a database password: never commit it, never reuse it-# across environments, never share it.-AISOC_CREDENTIAL_KEY=n-NAjH2x0NgryRcZrvSrQH-rp66MsI1HxDQr09j6L5w=-# Comma-separated list of previous credential keys to support zero-downtime-# rotation. Leave blank unless you are mid-rotation.-AISOC_CREDENTIAL_KEY_ROTATION_FROM=--# Internal URL the API uses to reach the connectors microservice for-# catalog lookups and "Test connection" actions in the click-and-connect-# wizard. ``http://connectors:8003`` is correct for docker-compose;-# use ``http://localhost:8088`` when running the API on the host while-# connectors is exposed via the docker-compose port forward.-CONNECTORS_SERVICE_URL=http://connectors:8003-CONNECTORS_SERVICE_TIMEOUT_SECONDS=15--# ─── Database ─────────────────────────────────────────────────────────────────-# Two roles, and the distinction is load-bearing rather than tidy.-#-# aisoc owner. Superuser (the postgres image makes POSTGRES_USER one),-# owns every table, applies the migration chain. No service-# connects as it.-# aisoc_app runtime. SELECT / INSERT / UPDATE / DELETE and nothing else, so-# that the schema's row-level-security policies apply to it. A-# superuser ignores those policies even under FORCE ROW LEVEL-# SECURITY, which is why running the services as `aisoc` left all-# 92 of them filtering nothing.-#-# POSTGRES_PASSWORD is the owner's; AISOC_APP_DB_PASSWORD is the runtime role's.-# Both are interpolated by docker-compose.yml, so changing them here rotates-# them across the stack. They are deliberately different values: they are-# different privilege levels.-#-# Empty here and generated by `scripts/ensure_env.py`, like the secrets-# above. This file used to ship `POSTGRES_PASSWORD=aisoc_dev_secret`, and-# `docker-compose.prod.yml` guarded it with-# `${POSTGRES_PASSWORD:?… the development default is a published literal}` —-# a guard that rejects an *empty* variable and cannot look at a value, so the-# one input its message named was the one input it accepted. A production-# stack booted on a password printed in this repository while the file said-# it could not. `preflight-secrets` in that file now refuses the value, and-# shipping these empty means a fresh install never has one to refuse.-POSTGRES_PASSWORD=440c43cf18f78e66a2885f12480c7551c934331b5a88eedc-AISOC_APP_DB_PASSWORD=03b5366c74c98fde62b1656d433f367e7cff75035d82cf4c--# What every service connects as.-DATABASE_URL=postgresql+asyncpg://aisoc_app:${AISOC_APP_DB_PASSWORD}@localhost:5432/aisoc--# DDL only: `python -m app.scripts.run_migrations` uses this and nothing else-# does. Leave it unset and migrations fall back to DATABASE_URL, which is-# correct for a deployment that has not split the roles and fails loudly-# ("permission denied for schema public") on one that has.-# aisoc-db-role: owner — the migration chain applies DDL; the runtime role holds none-DATABASE_MIGRATION_URL=postgresql+asyncpg://aisoc:${POSTGRES_PASSWORD}@localhost:5432/aisoc--# Generated alongside the Postgres pair, and referenced rather than repeated:-# these two used to spell the password out inside the DSN, so rotating-# REDIS_PASSWORD left a stale copy here that still worked.-REDIS_PASSWORD=12b5ff3f51c1cbb8a68b887b73ff3368cc3b30a4fa9b693d-CLICKHOUSE_PASSWORD=61dd17f85342f5f3a4850dab1da5a13a4c1bd682dd3dc70b-REDIS_URL=redis://:${REDIS_PASSWORD}@localhost:6379/0-CLICKHOUSE_URL=http://aisoc:${CLICKHOUSE_PASSWORD}@localhost:8123/aisoc--# ─── Messaging ────────────────────────────────────────────────────────────────-KAFKA_BOOTSTRAP_SERVERS=localhost:9092--# ─── Search ───────────────────────────────────────────────────────────────────-OPENSEARCH_URL=http://localhost:9200-QDRANT_URL=http://localhost:6333--# ─── AI / LLM ─────────────────────────────────────────────────────────────────-# Empty, and empty is a working configuration: CORE ships Ollama with a small-# local model, so a default install does real AI triage with no key at all. Set-# a hosted provider key here to upgrade the quality — the gateway picks it up-# and routes the aliases to the hosted model instead.-#-# It is empty rather than `sk-your-…-here` because a bogus key is not the same-# as no key: the gateway would send it upstream, get a 401, and the deployment-# would look configured while producing nothing.-OPENAI_API_KEY=-# OPENAI_MODEL affects the "explain this alert" / BYOK path only. Task workloads-# (triage, recon, investigation, copilot, summary, report, NL) request a logical-# alias instead — see the LLM gateway block below.-#-# It ships as an alias, not a concrete model, because the default deployment-# routes through the bundled gateway and the gateway 400s anything-# infra/litellm/config.yaml does not define. It used to read-# `gpt-4-turbo-preview`, which no gateway config here has ever defined, and the-# auto-triage worker treated it as a per-tenant BYOK override — so it replaced-# `aisoc-triage` on the highest-volume path in the product.-# scripts/check_llm_model_routing.py fails the build if this names a model the-# gateway does not define. Set a concrete model only when calling a provider-# directly (see "ENABLE LIVE LLM" option (b) below).-OPENAI_MODEL=aisoc-summary--# ─── LLM gateway (LiteLLM) — issue #478 ───────────────────────────────────────-# Since #478 AiSOC asks for a logical task alias (aisoc-triage,-# aisoc-investigation, …) for every live LLM call — there is no hardcoded default-# model. The alias → real-model map lives in infra/litellm/config.yaml, so you-# swap local/hosted models there without changing AiSOC. The gateway service is-# defined in docker-compose.yml.-#-# Key that AiSOC uses to authenticate to the gateway (also LiteLLM's admin key):-LITELLM_MASTER_KEY=sk-aisoc-local-# In-network URL of the gateway. docker-compose.yml sets this on the api and-# agents services, and both resolvers READ it — `resolve_base_url()` returns it-# for any aisoc-<role> alias, and `resolve_api_key()` returns LITELLM_MASTER_KEY-# to pair with it. That wire was missing: compose set this variable, the-# resolvers deliberately ignored it in favour of OPENAI_BASE_URL (which nothing-# set), and so the product's headline capability could not reach a model in the-# default deployment.-LLM_GATEWAY_URL=http://litellm:4000/v1-#-# ENABLE LIVE LLM:-# (a) Through the gateway — the default, and already wired. Put your real-# provider key in OPENAI_API_KEY above (the gateway reads it) and bring up-# the `full` profile, which is where the litellm service lives:-# docker compose --profile full up -d-# Nothing else to set. To point at a gateway somewhere else, or to send a-# different bearer, set OPENAI_BASE_URL/OPENAI_API_KEY and they win.-# (b) Direct to a provider, no gateway — pin each task role to a concrete-# model. A concrete pin is NOT rerouted to the gateway, so this keeps-# working with LLM_GATEWAY_URL set:-# AISOC_MODEL_PIN_TRIAGE=gpt-4o-mini-# AISOC_MODEL_PIN_INVESTIGATION=gpt-4o-# OPENAI_MODEL=gpt-4o-mini # the BYOK/explain path, direct too-# With neither configured, AiSOC uses its deterministic offline path — which-# reports zero model calls and zero tokens rather than pretending to triage.--# ─── Threat Intelligence ──────────────────────────────────────────────────────-# Open-source / community feeds (work out of the box, no key required for some)-TAXII_FEEDS=https://cti-taxii.mitre.org/taxii/,enterprise-attack,,-MISP_URL=-MISP_API_KEY=-OTX_API_KEY=-CISA_KEV_ENABLED=true-ABUSE_CH_ENABLED=true--# Free / freemium enrichment (recommended for everyone)-VIRUSTOTAL_API_KEY=-ABUSEIPDB_API_KEY=-GREYNOISE_API_KEY=-SHODAN_API_KEY=-URLSCAN_API_KEY=-IPINFO_API_TOKEN=--# ─── Commercial threat intelligence (optional) ────────────────────────────────-# Cyble Vision — actor, malware, vulnerability, dark-web, brand-risk feeds-# Provision a key from https://cyble.com/cyble-vision/-CYBLE_VISION_API_KEY=-CYBLE_VISION_BASE_URL=https://api.cyble.com/v2-CYBLE_VISION_TENANT_ID=-CYBLE_VISION_FEEDS=indicators,actors,vulnerabilities,leaks,brand-CYBLE_HUNT_API_KEY=--# Recorded Future-RECORDED_FUTURE_API_KEY=--# Mandiant Threat Intelligence (formerly FireEye iSIGHT)-MANDIANT_API_KEY=-MANDIANT_API_SECRET=--# Crowdstrike Falcon Intelligence (separate from the EDR connector)-CROWDSTRIKE_INTEL_CLIENT_ID=-CROWDSTRIKE_INTEL_CLIENT_SECRET=--# Anomali ThreatStream-ANOMALI_USERNAME=-ANOMALI_API_KEY=-ANOMALI_BASE_URL=https://api.threatstream.com--# IBM X-Force Exchange-XFORCE_API_KEY=-XFORCE_API_PASSWORD=--# Flashpoint-FLASHPOINT_API_KEY=--# Intel 471-INTEL471_USERNAME=-INTEL471_API_KEY=--# DomainTools Iris-DOMAINTOOLS_USERNAME=-DOMAINTOOLS_API_KEY=--# RiskIQ / Microsoft Defender External ATP-RISKIQ_USERNAME=-RISKIQ_API_KEY=--# ─── Connectors ───────────────────────────────────────────────────────────────-# CrowdStrike Falcon-CROWDSTRIKE_CLIENT_ID=-CROWDSTRIKE_CLIENT_SECRET=-CROWDSTRIKE_BASE_URL=https://api.crowdstrike.com--# Splunk-SPLUNK_HOST=-SPLUNK_PORT=8089-SPLUNK_TOKEN=-SPLUNK_SCHEME=https-SPLUNK_VERIFY_SSL=true--# AWS Security Hub-AWS_REGION=us-east-1-AWS_ACCESS_KEY_ID=-AWS_SECRET_ACCESS_KEY=--# Okta-OKTA_DOMAIN=-OKTA_API_TOKEN=--# Microsoft Sentinel-AZURE_TENANT_ID=-AZURE_CLIENT_ID=-AZURE_CLIENT_SECRET=-AZURE_SUBSCRIPTION_ID=-AZURE_RESOURCE_GROUP=-AZURE_WORKSPACE_NAME=--# ─── Frontend ─────────────────────────────────────────────────────────────────-# Where the console proxies browser requests. The browser itself never uses-# these: the bundle calls same-origin paths like /api/v1/alerts and the Next.js-# server forwards them, which is why one image works on localhost, on a LAN-# address and behind a reverse proxy without rebuilding, and why there is no-# CORS configuration to get wrong.-#-# The defaults are the Compose service names and are right for the bundled-# stack. Change them when the API lives somewhere else — a different Compose-# project, another host, or an existing deployment.-AISOC_API_URL=http://api:8000-AISOC_AGENTS_URL=http://agents:8084-AISOC_REALTIME_URL=http://realtime:4000--# NEXT_PUBLIC_* is NOT listed here on purpose.-#-# Next.js inlines every NEXT_PUBLIC_* value into the JavaScript bundle when the-# image is *built*, so setting one in this file has no effect on the image-# `docker compose up` pulls. This file used to list NEXT_PUBLIC_API_URL,-# NEXT_PUBLIC_REALTIME_URL and NEXT_PUBLIC_WS_URL, which meant an operator-# pointing the console at their own host edited three variables, restarted,-# saw no change, and had nothing telling them the settings were inert. Use the-# three AISOC_*_URL values above; they are read at container start.-#-# To bypass the proxy and have the browser call another origin directly you-# have to rebuild the image with --build-arg NEXT_PUBLIC_API_URL=… and take on-# the CORS configuration that comes with it.--# Override the docs site base URL (default: https://beenuar.github.io/AiSOC/docs).-# Set this when serving docs from a custom domain like https://docs.tryaisoc.com/docs.-# NEXT_PUBLIC_DOCS_URL=https://docs.tryaisoc.com/docs--# ─── Service-to-service auth ──────────────────────────────────────────────────-# Shared bearer token for backend services that are NOT reachable from the-# browser (connectors, purple-team, honeytokens, ueba). Those surfaces execute-# privileged work — purple-team can start adversary-emulation operations against-# live hosts — and they fail closed: with no token configured and AISOC_DEV_MODE-# unset, they answer 503 rather than serving the route openly.-#-# Left empty here and generated by `scripts/ensure_env.py`. It is not optional-# in practice: the API presents it when it proxies the connector catalog and-# "Test connection" to the connectors service, so a blank value on one side and-# a configured secret on the other is a 401 the wizard used to report as the-# bare string "Connection test failed".-#-# `ingest` and `connectors` use it for the same purpose on the push path, and-# there AISOC_DEV_MODE does NOT cover it: POST /v1/ingest is a write into-# tenant-owned data and has no dev-mode shortcut. A direct curl/script push-# uses a per-tenant token from `make ingest-token` instead and needs nothing-# here. See apps/docs/docs/operations/ingest-authentication.md.-AISOC_SERVICE_TOKEN=t0K34vZQE5T0MpeicQyZp4w6Js_8eISs38Kd5WMYy2c-#-# The actions service gets its own, because it is the one that can isolate a-# host or disable an account and should not share a credential with the-# read-mostly services above. `make up` generates it. Without it the actions-# service answers 503 on every mutating route, which is deliberate: it used to-# answer 200 to anyone who could reach the port.-AISOC_ACTIONS_SERVICE_TOKEN=B0N61vx6KgsmcH3UoZuKtIgd5Ot2uI_djKu1M-qiB6A-#-# The agents service gets its own too, and it is the one that could not fall-# back: eight modules in `services/agents` present it as-# `X-AiSOC-Service-Token` on the API's internal context routes, and-# `service_token_valid` on the API verifies THIS variable and no other — it-# does not accept AISOC_SERVICE_TOKEN. Nothing generated it and no manifest-# passed it, so organisation memory, tenant skills, recent dispositions,-# identity context, knowledge-base runbooks, the MCP toolset, the playbook-# action bridge and SIEM writeback were all off on every default install, each-# logging a `no_service_token` warning and returning an empty result that-# reads exactly like "this tenant has none".-#-# The same value is verified in the other direction: the API posts a replay to-# the agents service as a bearer, and `app.security.tenant_scope` resolves-# `AISOC_AGENTS_SERVICE_TOKEN` ahead of the shared one. `make up` generates it-# and compose passes it to both.-AISOC_AGENTS_SERVICE_TOKEN=sCn5w7PhzIJVvY5_5gIZZDlswDbqCxSe63Wd54fWZyM--# Bearer for realtime's /internal/* fan-out routes (API/agents -> realtime),-# sent as the x-internal-token header. `make up` generates it. The guard used-# to treat an unset value as authorized, and nothing set it.-REALTIME_INTERNAL_TOKEN=j3gjCbBj8yHEdE9BS8mQvlD0cR_waYn6Oc4YKzafuak-# Optional per-service overrides, if you'd rather not share one secret. Each-# is read ahead of AISOC_SERVICE_TOKEN by the service it names, on both the-# presenting and the verifying side, so setting one narrows a single hop.-# `scripts/check_service_token_wiring.py` asserts every one of these that-# production code reads is listed here — four were not, which is how an-# operator would go looking for a variable the code uses and find nothing.-# AISOC_PURPLE_TEAM_SERVICE_TOKEN=-# AISOC_HONEYTOKENS_SERVICE_TOKEN=-# AISOC_UEBA_SERVICE_TOKEN=-# AISOC_INGEST_SERVICE_TOKEN=-# AISOC_CONNECTORS_SERVICE_TOKEN=-# AISOC_FUSION_SERVICE_TOKEN=-# AISOC_THREATINTEL_SERVICE_TOKEN=-#-# Bearer the agents service presents to the ChatOps fan-out (slack-bot).-# Unset means no ChatOps notification is sent; the approval is still durable-# and visible in the console.-AISOC_INTERNAL_TOKEN=wXugJXuZwACNxpoQfeyNoV048qLTZQUfPNDz4UOUTfUdiff --git a/docker-compose.override.yml b/docker-compose.override.ymldeleted file mode 100644index f12e54cae..000000000--- a/docker-compose.override.yml+++ /dev/null@@ -1,9 +0,0 @@-services:- agents:- environment:- # Agent tools authenticate to the core API with get_current_user, which- # accepts only JWTs or DB-backed aisoc_-prefixed API keys - never the- # raw AISOC_AGENTS_SERVICE_TOKEN (that fails JWT decode with 401- # 'Could not validate credentials'). Use a minted aisoc_ API key from- # .env (scopes: connectors:read, actions:read, lake:query).- AISOC_AGENTS_API_KEY: ${AISOC_AGENTS_API_KEY:-}diff --git a/docker-compose.yml b/docker-compose.ymlindex 2ebd386e6..afd8fdfae 100644--- a/docker-compose.yml+++ b/docker-compose.yml@@ -826,13 +826,6 @@ services:- ollama_data:/root/.ollamaports:- "${AISOC_BIND_ADDR:-127.0.0.1}:11434:11434"- deploy:- resources:- reservations:- devices:- - driver: nvidia- count: 1- capabilities: [gpu]healthcheck:test: ["CMD-SHELL", "ollama list >/dev/null 2>&1 || exit 1"]interval: 10sdiff --git a/docker-compose.yml.bak.20261002-115859 b/docker-compose.yml.bak.20261002-115859deleted file mode 100644index 5c61519d8..000000000--- a/docker-compose.yml.bak.20261002-115859+++ /dev/null@@ -1,1564 +0,0 @@-###############################################################################-# AiSOC - AI Security Operations Center-# Docker Compose - Full Development Stack-# AiSOC — open-source under MIT License-#-# SECURITY NOTE-# --------------# All host port publishings below are bound to 127.0.0.1 so a default-# `docker compose up` does NOT expose Postgres / Kafka / OpenSearch /-# Neo4j / Redis (with their development passwords) to the network the-# host is attached to. This is intentional: the same passwords ship in-# every clone of this repo.-#-# DEPLOYING ON A HOST THAT IS NOT YOUR LAPTOP-# --------------------------------------------# Loopback is the right default and the wrong one for a server: it makes a-# single-host install unreachable from anywhere, including from the browser-# you intended to use it from. Set ONE variable in .env:-#-# AISOC_CONSOLE_BIND_ADDR=0.0.0.0-#-# That publishes the console on every interface and leaves everything else —-# Postgres, Redis, Kafka, Neo4j and their shipped development passwords — on-# loopback where they belong. One port is enough because the console proxies-# to every other service over the internal network, so the browser only ever-# talks to the console's own origin.-#-# `AISOC_BIND_ADDR` moves ALL of the bindings, including the datastores. It-# exists for an isolated network where that is what you want. On anything-# routable it hands out the passwords in this file, so reach for the console-# variable first and put a TLS-terminating reverse proxy in front.-#-# Per-service overrides still work through a docker-compose.override.yml, and-# there the list MUST be tagged `!override`:-#-# services:-# postgres:-# ports: !override-# - "0.0.0.0:5432:5432"-#-# The tag is not optional. Compose *appends* sequences when it merges, so an-# override without it publishes both the old binding and the new one — the-# 127.0.0.1 binding stays, and on a port that was already taken the conflict-# is still there. It looks like it worked and it did nothing. Use-# `ports: !reset []` to stop publishing a service on the host entirely.-#-# Rotate the password while you're there.-#-# TWO DATABASE ROLES-# -------------------# `aisoc` owner / migrations. Superuser, because POSTGRES_USER is what the-# postgres image makes it. Applies the migration chain. NO SERVICE-# CONNECTS AS THIS ROLE.-# `aisoc_app` runtime. SELECT / INSERT / UPDATE / DELETE and nothing else, and-# row-level security applies to it. Every service's DATABASE_URL.-#-# Until this split, every service ran as `aisoc`, and a superuser ignores RLS-# policies even under FORCE ROW LEVEL SECURITY — so all 92 policies in the-# schema filtered nothing. See services/api/migrations/061_runtime_app_role.sql.-#-# Every service below runs as the scoped role. Nothing needs a cross-tenant-# role: the policies are `tenant_id = current_tenant_id() OR current_tenant_id()-# IS NULL`, so ingest, fusion, the retention purge and the schedulers — none of-# which bind a tenant — keep seeing every tenant's rows. The per-service-# reasoning is tabulated in apps/docs/docs/operations/security.md.-###############################################################################--networks:- aisoc:- driver: bridge--volumes:- tempo_data:- postgres_data:- redis_data:- kafka_data:- zookeeper_data:- zookeeper_log:- clickhouse_data:- opensearch_data:- qdrant_data:- neo4j_data:- neo4j_logs:- # ~2 GB of model weights. Named so `make down` keeps them and only- # `make clean` re-downloads.- ollama_data:--services:-- # ─── Infrastructure ─────────────────────────────────────────────────────────-- postgres:- image: postgres:16-alpine- container_name: aisoc-postgres- environment:- # The OWNER / migration role. Superuser, because that is what the- # postgres image makes POSTGRES_USER, and it is what applies the chain.- # No service connects as this role — see the DATABASE_URL entries below.- POSTGRES_DB: aisoc- POSTGRES_USER: aisoc- POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-aisoc_dev_secret}- # The RUNTIME role's credential. `zz_runtime_role_password.sh` applies it- # on first boot, after 061_runtime_app_role.sql has created the role with- # DML-only grants. Rotate it the same way you rotate POSTGRES_PASSWORD;- # the two are deliberately different values because they are different- # privilege levels.- AISOC_APP_DB_PASSWORD: ${AISOC_APP_DB_PASSWORD:-aisoc_app_dev_secret}- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:5432:5432"- volumes:- - postgres_data:/var/lib/postgresql/data- # The init hooks: 10_ applies the migration chain, 20_ gives the runtime- # role its password. They live in one directory because Docker cannot- # create a mountpoint inside a bind mount — binding a single file into a- # bound `/docker-entrypoint-initdb.d` fails the container at startup.- # Only runs when the data directory is empty; an upgrade in place is- # covered by app.scripts.run_migrations instead.- - ./infra/postgres/initdb:/docker-entrypoint-initdb.d:ro- - ./services/api/migrations:/aisoc/migrations:ro- networks:- - aisoc- healthcheck:- test: ["CMD-SHELL", "pg_isready -U aisoc"]- interval: 10s- timeout: 5s- retries: 5-- redis:- image: redis:7-alpine- container_name: aisoc-redis- command: redis-server --appendonly yes --requirepass ${REDIS_PASSWORD:-redis_dev_secret}- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:6379:6379"- volumes:- - redis_data:/data- networks:- - aisoc- healthcheck:- test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD:-redis_dev_secret}", "ping"]- interval: 10s- timeout: 5s- retries: 5-- zookeeper:- image: confluentinc/cp-zookeeper:7.5.0- container_name: aisoc-zookeeper- environment:- ZOOKEEPER_CLIENT_PORT: 2181- ZOOKEEPER_TICK_TIME: 2000- # Kafka keeps its cluster id in `kafka_data`, which IS persisted, and- # takes the authoritative one from ZooKeeper, which was not. So the- # second `up` on a kept volume generated a fresh cluster id and the- # broker refused to join it:- #- # InconsistentClusterIdException: The Cluster ID <new> doesn't match- # stored clusterId Some(<old>) in meta.properties- #- # `make down && make up` — documented as "stop the stack, keep the data"- # — therefore could not bring Kafka back, and every service that depends- # on it stayed in `created`. Only `make clean`, which deletes the data,- # recovered. Persisting both halves of the pair is the fix; persisting- # neither would also be consistent, but it would discard the event log- # on every restart.- volumes:- - zookeeper_data:/var/lib/zookeeper/data- - zookeeper_log:/var/lib/zookeeper/log- networks:- - aisoc- healthcheck:- test: ["CMD", "nc", "-z", "localhost", "2181"]- interval: 10s- timeout: 5s- retries: 5-- kafka:- image: confluentinc/cp-kafka:7.5.0- container_name: aisoc-kafka- depends_on:- zookeeper:- condition: service_healthy- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:9092:9092"- environment:- KAFKA_BROKER_ID: 1- KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181- KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092- KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT- KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT- KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1- KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"- KAFKA_LOG_RETENTION_HOURS: 168- networks:- - aisoc- volumes:- - kafka_data:/var/lib/kafka/data- # Memory caps surface OOMKiller events as a clear container exit instead of- # silent broker rebalances. Tune up if you raise KAFKA_HEAP_OPTS.- mem_limit: 1536m- mem_reservation: 1g- healthcheck:- test: ["CMD", "kafka-broker-api-versions", "--bootstrap-server", "localhost:9092"]- interval: 15s- timeout: 10s- retries: 5-- kafka-ui:- # full profile — Kafka topic browser. A debugging convenience, not a pipeline component.- profiles: ["full"]- image: provectuslabs/kafka-ui:latest- container_name: aisoc-kafka-ui- depends_on:- kafka:- condition: service_healthy- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:8090:8080"- environment:- KAFKA_CLUSTERS_0_NAME: local- KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka:29092- networks:- - aisoc-- clickhouse:- # full profile — Event lake. Backs /lake/sql and the hunt surface; alerting does not read it.- profiles: ["full"]- image: clickhouse/clickhouse-server:23.8- container_name: aisoc-clickhouse- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:8123:8123"- - "${AISOC_BIND_ADDR:-127.0.0.1}:9000:9000"- volumes:- - clickhouse_data:/var/lib/clickhouse- - ./services/api/clickhouse:/docker-entrypoint-initdb.d:ro- environment:- CLICKHOUSE_DB: aisoc- CLICKHOUSE_USER: aisoc- CLICKHOUSE_PASSWORD: ${CLICKHOUSE_PASSWORD:-clickhouse_dev_secret}- CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT: 1- networks:- - aisoc- ulimits:- nofile:- soft: 262144- hard: 262144- mem_limit: 1g- mem_reservation: 768m-- opensearch:- # full profile — Full-text search over archived events. No CORE code path queries it.- profiles: ["full"]- image: opensearchproject/opensearch:2.11.0- container_name: aisoc-opensearch- environment:- - cluster.name=aisoc-cluster- - node.name=aisoc-node-1- - discovery.type=single-node- - bootstrap.memory_lock=true- - OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m- - DISABLE_INSTALL_DEMO_CONFIG=true- - DISABLE_SECURITY_PLUGIN=true- ulimits:- memlock:- soft: -1- hard: -1- volumes:- - opensearch_data:/usr/share/opensearch/data- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:9200:9200"- networks:- - aisoc- healthcheck:- # Without this the only condition a dependant can express is- # `service_started`, which is true the instant the container exists —- # including for the ~40s before the node listens, and for a node that- # is about to die. threatintel aborts its lifespan when this port does- # not answer, so it needs the stronger condition.- test: ["CMD-SHELL", "curl -fsS http://localhost:9200/_cluster/health || exit 1"]- interval: 10s- timeout: 5s- retries: 12- start_period: 60s- # A 512m heap is not a 512m process. OpenSearch 2.11 loads ~20 bundled- # plugins (knn, ml, security-analytics, performance-analyzer) and settles- # at ~1.05GB RSS with this heap — measured, steady-state, single node,- # no indices. At `mem_limit: 1g` it was OOM-killed during bootstrap every- # time (exit 137, OOMKilled=true, ~10s in), so the full profile shipped a- # store that could never start.- mem_limit: 2g- mem_reservation: 1g-- qdrant:- # CORE — the store `services/threatintel` writes collected indicators to,- # and the one the console's /threat-intel page is served from. It moved in- # with threatintel: see that service's comment for the reasoning.- #- # Measured: 245 MB image, 79 MiB RSS with the two threat-intel collections- # created. It is by a wide margin the cheapest of the four stores — the- # next one up, OpenSearch, is 2 GB of image and ~1.05 GB resident — which- # is why it is the one CORE takes.- image: qdrant/qdrant:v1.7.0- container_name: aisoc-qdrant- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:6333:6333"- volumes:- - qdrant_data:/qdrant/storage- networks:- - aisoc- healthcheck:- # threatintel creates its collections at startup, so it needs the store- # answering rather than merely existing. The image ships neither curl nor- # wget; `/proc/net/tcp` is what is left, and a listener on 6333 (hex- # 18BD) is the thing being asserted.- test: ["CMD-SHELL", "grep -q ':18BD' /proc/net/tcp || exit 1"]- interval: 10s- timeout: 5s- retries: 12- start_period: 20s-- neo4j:- # full profile — Entity graph. Backs graph traversals and blast radius, not alert generation.- profiles: ["full"]- image: neo4j:5.15-community- container_name: aisoc-neo4j- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:7474:7474" # HTTP browser- - "${AISOC_BIND_ADDR:-127.0.0.1}:7687:7687" # Bolt- environment:- NEO4J_AUTH: neo4j/${NEO4J_PASSWORD:-neo4j_dev_secret}- NEO4J_PLUGINS: '["apoc"]'- NEO4J_dbms_security_procedures_unrestricted: apoc.*- NEO4J_dbms_memory_heap_initial__size: 256m- NEO4J_dbms_memory_heap_max__size: 512m- volumes:- - neo4j_data:/data- - neo4j_logs:/logs- networks:- - aisoc- mem_limit: 1g- mem_reservation: 768m- healthcheck:- test: ["CMD-SHELL", "wget -qO- http://localhost:7474 || exit 1"]- interval: 15s- timeout: 10s- retries: 10-- # ─── Application Services ────────────────────────────────────────────────────-- api:- build:- context: ./services/api- dockerfile: Dockerfile- image: ghcr.io/beenuar/aisoc-core-api:${AISOC_VERSION:-latest}- pull_policy: ${AISOC_PULL_POLICY:-missing}- container_name: aisoc-api- depends_on:- postgres:- condition: service_healthy- redis:- condition: service_healthy- kafka:- condition: service_healthy- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:8000:8000"- environment:- # Two roles, on purpose. DATABASE_URL is the runtime role: DML only, and- # row-level security applies to it. DATABASE_MIGRATION_URL is the owner,- # used by `app.scripts.run_migrations` for DDL and nothing else. Running- # the service as the owner is what made all 92 RLS policies inert, since- # a superuser ignores them even under FORCE ROW LEVEL SECURITY.- DATABASE_URL: postgresql+asyncpg://aisoc_app:${AISOC_APP_DB_PASSWORD:-aisoc_app_dev_secret}@postgres:5432/aisoc- DATABASE_MIGRATION_URL: postgresql+asyncpg://aisoc:${POSTGRES_PASSWORD:-aisoc_dev_secret}@postgres:5432/aisoc- # Read by the migration runner to (re)apply the runtime role's password- # on an existing data volume, where the postgres init hook never fires.- AISOC_APP_DB_PASSWORD: ${AISOC_APP_DB_PASSWORD:-aisoc_app_dev_secret}- REDIS_URL: redis://:${REDIS_PASSWORD:-redis_dev_secret}@redis:6379/0- KAFKA_BOOTSTRAP_SERVERS: kafka:29092- # The lake client reads these five, not CLICKHOUSE_URL — nothing under- # services/api/app reads that name at all. Supplying only the URL left- # the settings at their defaults (localhost:9000, user `default`, no- # password), so /lake/sql answered every query with- # `Connection refused (localhost:9000)` while fusion archived to the- # same store perfectly well over the native port.- CLICKHOUSE_HOST: clickhouse- CLICKHOUSE_PORT: 9000- CLICKHOUSE_DATABASE: aisoc- CLICKHOUSE_USER: aisoc- CLICKHOUSE_PASSWORD: ${CLICKHOUSE_PASSWORD:-clickhouse_dev_secret}- OPENSEARCH_URL: http://opensearch:9200- QDRANT_URL: http://qdrant:6333- # The API owns the vault write path; the connectors service reads with- # the same key. Passing it to one and not the other means a credential- # saved in the console can never be decrypted at poll time.- AISOC_CREDENTIAL_KEY: ${AISOC_CREDENTIAL_KEY:-}- AISOC_CREDENTIAL_KEY_ROTATION_FROM: ${AISOC_CREDENTIAL_KEY_ROTATION_FROM:-}- NEO4J_URI: bolt://neo4j:7687- NEO4J_USER: neo4j- NEO4J_PASSWORD: ${NEO4J_PASSWORD:-neo4j_dev_secret}- # Was a bare literal, so a SECRET_KEY set in .env was ignored and the- # JWT signing key silently stayed on the dev default. It also has to- # match whatever the connectors service verifies with, or the console- # token the API forwards there fails its signature check.- SECRET_KEY: ${SECRET_KEY:-dev_secret_key_change_in_production}- # The API mints realtime tickets and calls realtime's /internal/* routes,- # so it needs the same two secrets the realtime container reads. Neither- # was passed to either side before, which is why both guards there were- # running on a published constant or not running at all.- AISOC_REALTIME_JWT_SECRET: ${AISOC_REALTIME_JWT_SECRET:-}- # Both are required for the API to boot now that ENVIRONMENT defaults- # to production, and neither was interpolated here — so a value in .env- # could not reach the container, which is the same shape that once left- # AISOC_SERVICE_TOKEN unreachable to this one service.- METRICS_TOKEN: ${METRICS_TOKEN:-}- JWT_SECRET: ${JWT_SECRET:-}- REALTIME_INTERNAL_TOKEN: ${REALTIME_INTERNAL_TOKEN:-}- # The credential this service presents when it proxies the connector- # catalog and "Test connection" to the connectors service.- # `_service_token()` in app/api/v1/endpoints/connectors.py reads- # AISOC_CONNECTORS_SERVICE_TOKEN then AISOC_SERVICE_TOKEN — and compose- # interpolated the shared token into connectors, ueba, honeytokens and- # purple-team but not into the one service that has to *send* it. There- # is no `env_file` here either, so a value in .env could not reach it by- # any route. The catalog call was answered 401 on every request and fell- # back to the bundled copy; the fix that made the proxy read the right- # variable shipped against a variable nothing ever set.- AISOC_SERVICE_TOKEN: ${AISOC_SERVICE_TOKEN:-}- AISOC_CONNECTORS_SERVICE_TOKEN: ${AISOC_CONNECTORS_SERVICE_TOKEN:-}- # The agents service's credential, on the verifying side. `alert_writeback- # .service_token_valid` reads this and only this — it does NOT fall back- # to AISOC_SERVICE_TOKEN — so the internal context routes the agents- # service calls (`/feedback/context-statements`,- # `/tenant-skills/resolved/active`, `/kb/runbooks/for-triage`,- # `/feedback/recent-dispositions`, `/graph/identity-context`, the MCP- # registry and `/alerts/{id}/source-writeback`) refuse every caller until- # it is set. Must match the value on `agents`.- AISOC_AGENTS_SERVICE_TOKEN: ${AISOC_AGENTS_SERVICE_TOKEN:-}- CONNECTORS_SERVICE_URL: ${CONNECTORS_SERVICE_URL:-http://connectors:8003}- # Where GET /api/v1/threat-intel/indicators is served from. The console- # has called that path since the IOC inbox shipped and nothing answered- # it; threatintel is in CORE now and this is the wire.- THREATINTEL_SERVICE_URL: ${THREATINTEL_SERVICE_URL:-http://threatintel:8005}- AISOC_THREATINTEL_SERVICE_TOKEN: ${AISOC_THREATINTEL_SERVICE_TOKEN:-}- # Printed by `make bootstrap` as the sign-in address and used as the- # console's public origin. Read from os.environ by- # app/scripts/bootstrap_admin.py, which no compose service set — so any- # deployment that is not a laptop printed the wrong address.- AISOC_CONSOLE_URL: ${AISOC_CONSOLE_URL:-http://localhost:3000}- # Was a bare literal, so `.env`'s documented ENVIRONMENT was ignored and- # `production` changed nothing — including the dev auth bypass in- # app/api/v1/dev_auth.py, which could not be switched off from .env at all.- ENVIRONMENT: ${ENVIRONMENT:-production}- # The bundled Postgres and Kafka serve no certificates, and- # ENVIRONMENT now defaults to production — where a cleartext transport- # is a boot refusal, not a warning. Without these two the documented- # `make up` fails on a machine where there is nothing to fix, which is- # exactly how a security default gets patched out downstream. Stated- # here, in the file that chooses it, rather than weakened in the code- # that enforces it: a real deployment that leaves either unset is- # refused at boot.- AISOC_ALLOW_CLEARTEXT_DB: "1"- AISOC_ALLOW_CLEARTEXT_KAFKA: "1"- # What this deployment published, so the service can refuse the- # anonymous auth shim on an address someone else can reach. It- # binds 0.0.0.0 inside the container and cannot tell otherwise.- AISOC_PUBLISHED_BIND_ADDRS: ${AISOC_CONSOLE_BIND_ADDR:-${AISOC_BIND_ADDR:-127.0.0.1}},${AISOC_BIND_ADDR:-127.0.0.1}- LOG_LEVEL: ${LOG_LEVEL:-info}- # Issue #478 — LiteLLM gateway. This service's live LLM endpoints- # (translation, hunts, KB, phishing, NL-detection) route here by default:- # `app.services.model_aliases.resolve_base_url` reads LLM_GATEWAY_URL for- # any aisoc-<role> alias. It was previously read by nothing, so the- # variable was set, documented, and dead — every alias went to the- # provider default and 404'd. OPENAI_BASE_URL still overrides it.- LLM_GATEWAY_URL: http://litellm:4000/v1- # The bearer that pairs with the URL above. Without it AiSOC would send a- # provider key to the gateway, which rejects it as an invalid proxy token.- LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY:-sk-aisoc-local}- OPENAI_API_KEY: ${OPENAI_API_KEY:-}- # services/actions — where an approved response action is executed.- # The API had no pointer to it at all: the one endpoint that called it- # read the URL from os.environ and fell back to `aisoc-actions`, which- # is the container_name, not the DNS name on this network.- AISOC_ACTIONS_BASE_URL: http://actions:8085- AISOC_ACTIONS_SERVICE_TOKEN: ${AISOC_ACTIONS_SERVICE_TOKEN:-}- # Phase 2.6 — use /readyz, not /health, so the orchestrator- # only routes traffic once the lifespan has finished wiring- # Postgres, Redis, Kafka, Qdrant, and Neo4j. (Not OpenSearch: this- # service constructs no OpenSearch client. `services/threatintel` is- # the only consumer of that store.)- # /livez is what the restart policy looks at via `pgrep`-style- # checks; we don't need a separate compose healthcheck for it- # because the process being up implies it's alive.- healthcheck:- test:- - CMD-SHELL- - "python -c \"import urllib.request,sys; r=urllib.request.urlopen('http://127.0.0.1:8000/readyz',timeout=2); sys.exit(0 if r.status==200 else 1)\" || exit 1"- interval: 10s- timeout: 5s- retries: 30- start_period: 60s- networks:- - aisoc- restart: unless-stopped-- ingest-worker:- build:- context: ./services/ingest- dockerfile: Dockerfile- image: ghcr.io/beenuar/aisoc-ingest:${AISOC_VERSION:-latest}- pull_policy: ${AISOC_PULL_POLICY:-missing}- container_name: aisoc-ingest- depends_on:- kafka:- condition: service_healthy- postgres:- condition: service_healthy- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:8081:8080"- - "${AISOC_BIND_ADDR:-127.0.0.1}:9090:9090"- environment:- # Interpolated, not a literal. `.env.example` tells an operator that- # setting ENVIRONMENT=production is enough, and for every other service- # it is. This one pinned `development`, and `envmode.Current()` reads- # ENV *before* ENVIRONMENT — so ingest stayed dev-mode however the- # operator set it, and `JWT_SECRET must be set in non-development- # environments` never fired on the one service taking events off the- # network.- ENV: ${ENVIRONMENT:-production}- # The bundled Postgres and Kafka serve no certificates, and- # ENVIRONMENT now defaults to production — where a cleartext transport- # is a boot refusal, not a warning. Without these two the documented- # `make up` fails on a machine where there is nothing to fix, which is- # exactly how a security default gets patched out downstream. Stated- # here, in the file that chooses it, rather than weakened in the code- # that enforces it: a real deployment that leaves either unset is- # refused at boot.- AISOC_ALLOW_CLEARTEXT_DB: "1"- AISOC_ALLOW_CLEARTEXT_KAFKA: "1"- # What this deployment published, so the service can refuse the- # anonymous auth shim on an address someone else can reach. It- # binds 0.0.0.0 inside the container and cannot tell otherwise.- AISOC_PUBLISHED_BIND_ADDRS: ${AISOC_CONSOLE_BIND_ADDR:-${AISOC_BIND_ADDR:-127.0.0.1}},${AISOC_BIND_ADDR:-127.0.0.1}- HTTP_PORT: 8080- METRICS_PORT: 9090- KAFKA_BOOTSTRAP_SERVERS: kafka:29092- REDIS_URL: redis://:${REDIS_PASSWORD:-redis_dev_secret}@redis:6379/1- # The inbox webhook routes (/v1/inbox/*) mount only when this is set,- # and no compose file set it, so the documented "push events straight- # from your SIEM" path was unreachable on every local and CI stack.- # This is a pgx DSN and is deliberately distinct from the SQLAlchemy- # DATABASE_URL the Python services use.- # sslmode=disable is correct *here* and nowhere else: the bundled- # Postgres serves no certificate, so requiring TLS would break the- # documented first run on a machine where there is nothing to fix.- # A production deployment that leaves this is reported at boot by- # warn_if_insecure_defaults, which also reports an *unset* sslmode —- # libpq reads that as `prefer`, which downgrades to cleartext without- # a log line, and is therefore worse than the honest `disable` below.- DATABASE_DSN: postgres://aisoc_app:${AISOC_APP_DB_PASSWORD:-aisoc_app_dev_secret}@postgres:5432/aisoc?sslmode=disable- LOG_LEVEL: info- # A bare literal, and a published one: `dev_secret_key_change_in_production`- # is in INSECURE_SECRET_KEY_DEFAULTS, so anyone who read this repository- # held the key this service verified with. Interpolated now, and `make- # env` generates it.- JWT_SECRET: ${JWT_SECRET:-}- # /v1/ingest requires a credential. Two shapes are accepted: a- # per-tenant push token minted with `make ingest-token` (resolved- # through DATABASE_DSN above), and this shared service token, which- # `connectors` presents when it polls on behalf of many tenants.- # Empty by default — with it unset the connector-poll path is refused- # and only minted tokens work. The same variable must be set on the- # `connectors` service. See apps/docs/docs/operations/ingest-authentication.md.- AISOC_SERVICE_TOKEN: ${AISOC_SERVICE_TOKEN:-}- ATTCK_DATA_PATH: /app/data/enterprise-attack.json- # Graph-at-ingest needs Neo4j, which is a `full`-profile service. It is- # off in CORE and switched on by the `full` overlay. It used to be- # hardcoded "true" with a hard `depends_on: neo4j`, which is why the- # minimal stack could not start without the graph database.- AISOC_GRAPH_ENABLED: "${AISOC_GRAPH_ENABLED:-false}"- AISOC_NEO4J_URI: bolt://neo4j:7687- AISOC_NEO4J_USER: neo4j- AISOC_NEO4J_PASSWORD: ${NEO4J_PASSWORD:-neo4j_dev_secret}- networks:- - aisoc- restart: unless-stopped-- enrichment:- # full profile — Fuse-time threat-intel and vulnerability enrichment. Alerts fire without it.- profiles: ["full"]- build:- context: ./services/enrichment- dockerfile: Dockerfile- image: ghcr.io/beenuar/aisoc-enrichment:${AISOC_VERSION:-latest}- pull_policy: ${AISOC_PULL_POLICY:-missing}- container_name: aisoc-enrichment- depends_on:- redis:- condition: service_healthy- kafka:- condition: service_healthy- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:8080:8082"- environment:- KAFKA_BOOTSTRAP_SERVERS: kafka:29092- REDIS_URL: redis://:${REDIS_PASSWORD:-redis_dev_secret}@redis:6379/2- VIRUSTOTAL_API_KEY: ${VIRUSTOTAL_API_KEY:-}- ABUSEIPDB_API_KEY: ${ABUSEIPDB_API_KEY:-}- GREYNOISE_API_KEY: ${GREYNOISE_API_KEY:-}- SHODAN_API_KEY: ${SHODAN_API_KEY:-}- networks:- - aisoc- restart: unless-stopped-- fusion:- build:- context: ./services/fusion- dockerfile: Dockerfile- image: ghcr.io/beenuar/aisoc-fusion:${AISOC_VERSION:-latest}- pull_policy: ${AISOC_PULL_POLICY:-missing}- container_name: aisoc-fusion- depends_on:- # Postgres was missing here, so during a cold boot fusion tried to write- # alerts before the database accepted connections and logged the failure- # exactly once.- postgres:- condition: service_healthy- redis:- condition: service_healthy- kafka:- condition: service_healthy- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:8003:8003"- environment:- KAFKA_BOOTSTRAP_SERVERS: kafka:29092- REDIS_URL: redis://:${REDIS_PASSWORD:-redis_dev_secret}@redis:6379/3- # `tenant_scope` refuses outright when neither a console secret nor a- # service token is configured and dev mode is off, and this service- # received none of the three — the same trio `connectors` and- # `threatintel` already carry.- AISOC_DEV_MODE: ${AISOC_DEV_MODE:-0}- # What this deployment published, so the service can refuse the- # anonymous auth shim on an address someone else can reach. It- # binds 0.0.0.0 inside the container and cannot tell otherwise.- AISOC_PUBLISHED_BIND_ADDRS: ${AISOC_CONSOLE_BIND_ADDR:-${AISOC_BIND_ADDR:-127.0.0.1}},${AISOC_BIND_ADDR:-127.0.0.1}- SECRET_KEY: ${SECRET_KEY:-}- AISOC_SERVICE_TOKEN: ${AISOC_SERVICE_TOKEN:-}- DATABASE_URL: postgresql+asyncpg://aisoc_app:${AISOC_APP_DB_PASSWORD:-aisoc_app_dev_secret}@postgres:5432/aisoc- # The ClickHouse event lake is a `full`-profile capability: it backs- # /lake/sql and the hunt surface, not alerting. CORE runs without it and- # the writer disables itself.- AISOC_LAKE_WRITER_ENABLED: "${AISOC_LAKE_WRITER_ENABLED:-false}"- CLICKHOUSE_HOST: clickhouse- CLICKHOUSE_PORT: "9000"- CLICKHOUSE_DATABASE: aisoc- CLICKHOUSE_USER: aisoc- CLICKHOUSE_PASSWORD: ${CLICKHOUSE_PASSWORD:-clickhouse_dev_secret}- # Fusion defaulted to http://localhost:8082, which inside its own- # container is fusion itself — so fuse-time enrichment always failed and- # the failure was swallowed at DEBUG. `enrichment` is a `full`-profile- # service; in CORE this resolves to nothing and enrichment is skipped,- # which is the same outcome as before but now deliberate.- ENRICHMENT_SERVICE_URL: ${ENRICHMENT_SERVICE_URL:-http://enrichment:8082}- networks:- - aisoc- restart: unless-stopped-- agents:- build:- context: ./services/agents- dockerfile: Dockerfile- image: ghcr.io/beenuar/aisoc-agents:${AISOC_VERSION:-latest}- pull_policy: ${AISOC_PULL_POLICY:-missing}- container_name: aisoc-agents- depends_on:- postgres:- condition: service_healthy- redis:- condition: service_healthy- kafka:- condition: service_healthy- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:8001:8084"- environment:- DATABASE_URL: postgresql+asyncpg://aisoc_app:${AISOC_APP_DB_PASSWORD:-aisoc_app_dev_secret}@postgres:5432/aisoc- REDIS_URL: redis://:${REDIS_PASSWORD:-redis_dev_secret}@redis:6379/4- CORE_API_URL: http://api:8000- # Phase B1 — auto-triage every fused alert off the Kafka stream (copilot:- # read-only triage, no response dispatch). Degrades to deterministic- # triage when no LLM key is set.- KAFKA_BOOTSTRAP_SERVERS: kafka:29092- KAFKA_TOPIC_ALERTS_FUSED: aisoc.alerts.fused- QDRANT_URL: http://qdrant:6333- NEO4J_URI: bolt://neo4j:7687- NEO4J_USER: neo4j- NEO4J_PASSWORD: ${NEO4J_PASSWORD:-neo4j_dev_secret}- OPENAI_API_KEY: ${OPENAI_API_KEY:-}- ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:-}- # BYOK / "explain this alert" only — never a task role. A task role asks- # `app.llm.model_pins` for its aisoc-<role> alias, and this leaking into- # that path is what replaced `aisoc-triage` with a model the gateway does- # not define. Defaults to the summary alias so the explain path resolves- # at the gateway too; set a concrete model only for direct-to-provider.- OPENAI_MODEL: ${OPENAI_MODEL:-aisoc-summary}- # Issue #478 — the LiteLLM gateway (below) is the single entry point for- # live LLM calls. `app.llm.factory.resolve_base_url` reads this for any- # aisoc-<role> alias, so the default deployment reaches the gateway with- # no further configuration; OPENAI_BASE_URL still overrides it, and a- # concrete AISOC_MODEL_PIN_<ROLE> is left pointing at its provider.- LLM_GATEWAY_URL: http://litellm:4000/v1- # The bearer that pairs with the URL above — see the api service.- LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY:-sk-aisoc-local}- ATTCK_DATA_PATH: /app/data/enterprise-attack.json- # Where to post an approval card when triage raises one. Unset means- # no ChatOps notification; the approval is still durable and visible in- # the console and the responder app. slack-bot is a `chatops`-profile- # service, so this only resolves when that profile is up.- AISOC_SLACK_BOT_URL: ${AISOC_SLACK_BOT_URL:-}- AISOC_INTERNAL_TOKEN: ${AISOC_INTERNAL_TOKEN:-}- # Bearer for realtime's /internal/* fan-out. `app/playbook/engine.py`- # reads it to publish step progress; unset, realtime refuses the call- # and a running playbook is invisible in the console.- REALTIME_INTERNAL_TOKEN: ${REALTIME_INTERNAL_TOKEN:-}- # The three below are the same trio `connectors` and `threatintel`- # already carry, and this service had none of them. `tenant_scope`- # refuses outright when neither a console secret nor a service token is- # configured *and* dev mode is off — which is what every tenant-scoped- # route here answered: 503, with no credential material to verify- # against and nothing in the image supplying any.- AISOC_DEV_MODE: ${AISOC_DEV_MODE:-0}- # What this deployment published, so the service can refuse the- # anonymous auth shim on an address someone else can reach. It- # binds 0.0.0.0 inside the container and cannot tell otherwise.- AISOC_PUBLISHED_BIND_ADDRS: ${AISOC_CONSOLE_BIND_ADDR:-${AISOC_BIND_ADDR:-127.0.0.1}},${AISOC_BIND_ADDR:-127.0.0.1}- SECRET_KEY: ${SECRET_KEY:-}- AISOC_SERVICE_TOKEN: ${AISOC_SERVICE_TOKEN:-}- # The agents service's own service credential, used in both directions:- # presented as `X-AiSOC-Service-Token` on the API's internal context- # routes, and verified by `app.security.tenant_scope` when the API calls- # back (it resolves `AISOC_AGENTS_SERVICE_TOKEN` before the shared one).- #- # Eight modules in this service read it and nothing produced it, so on a- # default install organisation memory, tenant skills, recent- # dispositions, identity context, knowledge-base runbooks, the MCP- # toolset, the playbook action bridge and SIEM writeback were all off —- # each logging its own `no_service_token` warning and returning empty.- # `make env` generates it; it must match the value on `api` below.- AISOC_AGENTS_SERVICE_TOKEN: ${AISOC_AGENTS_SERVICE_TOKEN:-}- networks:- - aisoc- restart: unless-stopped-- # ─── LLM gateway (LiteLLM) — issue #478 ─────────────────────────────────────- # Single entry point for every live LLM call. AiSOC requests a logical task- # alias (aisoc-triage, aisoc-investigation, …); the mapping alias → real model- # lives entirely in infra/litellm/config.yaml, so operators swap local/hosted- # models without touching AiSOC. Per-task latency/tokens/cost/errors are- # exported on /metrics (scraped as job `aisoc-litellm`). Lightweight and inert- # unless a service is pointed at it via OPENAI_BASE_URL — see the note on the- # `agents` service above.- ollama:- # CORE — the model behind the gateway.- #- # The gateway shipped in CORE and the models it pointed at did not: every- # alias resolved to a hosted provider, so the default install could route a- # key it did not have and otherwise did no AI at all. This closes that: a- # clone with no account produces a real triage verdict, from a real model,- # with real token counts.- #- # Promoted from infra/compose/docker-compose.airgap.yml, where this exact- # pairing has been proven end to end. The model is pinned rather than- # floating on `latest` for the same reason every other image here is.- #- # Measured: 6.92 GB image — by some distance the largest thing in CORE —- # 2.02 GB of weights in the volume, and 9.6 MiB resident when idle.- #- # Quoting the idle figure alone would be the misleading half of the truth:- # the model is mapped in on first inference and the container measured- # 2.96 GiB while serving. That is what moved the README's memory figure to- # 8 GB, not the idle number.- image: ollama/ollama:0.6.7- container_name: aisoc-ollama- volumes:- - ollama_data:/root/.ollama- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:11434:11434"- deploy:- resources:- reservations:- devices:- - driver: nvidia- count: 1- capabilities: [gpu]- healthcheck:- test: ["CMD-SHELL", "ollama list >/dev/null 2>&1 || exit 1"]- interval: 10s- timeout: 5s- retries: 18- start_period: 30s- networks:- - aisoc- restart: unless-stopped-- ollama-pull:- # One-shot: pull the pinned model, then exit. A cache hit is a no-op, so- # this costs one `ollama list` on every `up` after the first.- #- # litellm waits on `service_completed_successfully` here rather than on- # ollama's health: a gateway that is up before the weights exist answers the- # first triage request with "model not found", and the first request is the- # one a new user makes.- image: ollama/ollama:0.6.7- container_name: aisoc-ollama-pull- depends_on:- ollama:- condition: service_healthy- entrypoint: ["ollama", "pull", "${AISOC_LLM_LOCAL_MODEL:-llama3.2:3b-instruct-q4_K_M}"]- environment:- OLLAMA_HOST: http://ollama:11434- networks:- - aisoc- restart: "no"-- litellm:- # CORE, deliberately — see docs/decisions/0006-llm-gateway-in-core.md.- #- # This was `profiles: ["full"]` on the reasoning that the gateway is "only- # needed when a provider key is configured". The reasoning skipped a step:- # a CORE deployment *with* a key could not use it either, because every- # `aisoc-<role>` alias resolves nowhere except here. So the documented- # default install — what `make up` starts and what the quickstart walks a- # new evaluator through — could not do AI triage at all, and the profile- # table said it did "triage".- #- # With no key it boots, serves its seven aliases and answers /health;- # AiSOC's deterministic path is unchanged and still honest. Measured cost:- # a 1.67 GB image idling at ~451 MiB, which is why the RAM figure in the- # README moved rather than staying put.- #- # It is also the only party that can say what a call cost — the real- # figure comes off its response headers — so leaving it out of CORE left- # the default install's cost dashboard permanently unmeasurable.- image: ghcr.io/berriai/litellm:main-stable- container_name: aisoc-litellm- command: ["--config", "/etc/litellm/config.yaml", "--port", "4000"]- volumes:- - ./infra/litellm/config.yaml:/etc/litellm/config.yaml:ro- depends_on:- ollama-pull:- condition: service_completed_successfully- environment:- # Clients authenticate to the gateway with this key (AiSOC sends it as its- # OPENAI_API_KEY when OPENAI_BASE_URL points here). Override in .env.- LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY:-sk-aisoc-local}- # Which backend every alias resolves to. infra/litellm/config.yaml reads- # these through `os.environ/...` instead of naming a model inline, so the- # local→hosted upgrade is three variables in .env rather than an edit to a- # mounted config file.- #- # The defaults are the bundled local model, which is what makes AI triage- # work in a default `make up` with no account and no key. Set- # OPENAI_API_KEY plus these two to openai/gpt-4o-mini and openai/gpt-4o —- # and clear AISOC_LLM_API_BASE — to route to a hosted provider instead.- AISOC_LLM_MODEL_FAST: ${AISOC_LLM_MODEL_FAST:-ollama_chat/llama3.2:3b-instruct-q4_K_M}- AISOC_LLM_MODEL_DEEP: ${AISOC_LLM_MODEL_DEEP:-ollama_chat/llama3.2:3b-instruct-q4_K_M}- # `-` and not `:-`, deliberately: an operator moving to a hosted provider- # sets AISOC_LLM_API_BASE= (empty) in .env, and `:-` would helpfully undo- # that by substituting the Ollama address back in.- AISOC_LLM_API_BASE: ${AISOC_LLM_API_BASE-http://ollama:11434}- # Upstream provider keys the aliases resolve against (os.environ/... in the- # config). Empty is fine — the local model answers instead.- OPENAI_API_KEY: ${OPENAI_API_KEY:-}- ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:-}- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:4000:4000"- healthcheck:- test:- - CMD-SHELL- - "python -c \"import urllib.request,sys; r=urllib.request.urlopen('http://127.0.0.1:4000/health/liveliness',timeout=2); sys.exit(0 if r.status==200 else 1)\" || exit 1"- interval: 15s- timeout: 5s- retries: 10- start_period: 30s- networks:- - aisoc- restart: unless-stopped-- osquery-tls:- build:- context: ./services/osquery-tls- dockerfile: Dockerfile- image: ghcr.io/beenuar/aisoc-osquery-tls:${AISOC_VERSION:-latest}- pull_policy: ${AISOC_PULL_POLICY:-missing}- container_name: aisoc-osquery-tls- depends_on:- postgres:- condition: service_healthy- # Host port 8007 was previously colliding with the `ueba` service's- # 127.0.0.1:8007 binding; remapped to 8091 here. UEBA still owns 8007.- #- # The container side was 8007 too, and nothing has ever listened there:- # this service's Dockerfile runs uvicorn on 9001 and EXPOSEs 9001. The- # published port therefore resolved to a closed socket, and because the- # service had no healthcheck either, the container still read as- # `running`. Found by adding the healthcheck below and watching it sit- # at `starting` against a port with no server on it.- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:8091:9001"- profiles:- - osquery- environment:- # Two roles, the same split the api service uses. This service manages- # its own schema through alembic, so it needs the owner to apply DDL and- # must not serve requests as it. The marker on the second line below is- # what tells scripts/check_runtime_db_role.py the owner is deliberate.- DATABASE_URL: postgresql+asyncpg://aisoc_app:${AISOC_APP_DB_PASSWORD:-aisoc_app_dev_secret}@postgres:5432/aisoc- # aisoc-db-role: owner — alembic applies this service's chain; the runtime role holds no DDL- DATABASE_MIGRATION_URL: postgresql+asyncpg://aisoc:${POSTGRES_PASSWORD:-aisoc_dev_secret}@postgres:5432/aisoc- # Read by the postgres init hook on a fresh volume and by the api- # migration runner otherwise. Passed here too so this service's own- # DATABASE_URL and the role's actual credential come from one value.- AISOC_APP_DB_PASSWORD: ${AISOC_APP_DB_PASSWORD:-aisoc_app_dev_secret}- AISOC_OSQUERY_TLS_ENROLL_SECRET: ${AISOC_OSQUERY_TLS_ENROLL_SECRET:-change-me-in-production}- # `tenant_scope` refuses outright when neither a console secret nor a- # service token is configured and dev mode is off, and this service- # received none of the three — the same trio `connectors` and- # `threatintel` already carry.- AISOC_DEV_MODE: ${AISOC_DEV_MODE:-0}- # What this deployment published, so the service can refuse the- # anonymous auth shim on an address someone else can reach. It- # binds 0.0.0.0 inside the container and cannot tell otherwise.- AISOC_PUBLISHED_BIND_ADDRS: ${AISOC_CONSOLE_BIND_ADDR:-${AISOC_BIND_ADDR:-127.0.0.1}},${AISOC_BIND_ADDR:-127.0.0.1}- SECRET_KEY: ${SECRET_KEY:-}- AISOC_SERVICE_TOKEN: ${AISOC_SERVICE_TOKEN:-}- # The ingest service is registered as `ingest-worker` in this compose- # file; the previous `http://ingest:8080` would not resolve via DNS.- AISOC_INGEST_BASE_URL: http://ingest-worker:8080- # A detached consumer used to read as `running`. /readyz answers 503 when- # the subscription is gone or the schema is not at head, so `docker compose- # ps` and `make _wait` now see the difference.- healthcheck:- test:- - CMD-SHELL- - "python -c \"import urllib.request,sys; r=urllib.request.urlopen('http://127.0.0.1:9001/readyz',timeout=2); sys.exit(0 if r.status==200 else 1)\" || exit 1"- interval: 10s- timeout: 5s- retries: 20- start_period: 90s- networks:- - aisoc- restart: unless-stopped-- actions:- # Response execution, and the only service that holds the read-only vendor- # verbs an investigation pivots through.- #- # This was `profiles: ["full", "chatops"]` on the reasoning that nothing in- # CORE dispatches to it. Two things were wrong with that. The `api` service- # in CORE already sets AISOC_ACTIONS_BASE_URL to this hostname, so the- # live-actions surface was enabled and pointed at a name that does not- # resolve; and the investigation agent's vendor read tools dispatch here,- # so on the default profile the agent had no way to reach a customer's EDR- # or IdP at all. Measured cost of moving it: 48.3 MiB resident, 539 MB of- # image. See docs/decisions/0007-connectors-and-actions-in-core.md.- #- # No `profiles:` key rather than a longer list: a service in no profile is- # included in every profile run, so `chatops` still gets it and- # slack-bot's dependency still renders.- build:- context: ./services/actions- dockerfile: Dockerfile- image: ghcr.io/beenuar/aisoc-actions:${AISOC_VERSION:-latest}- pull_policy: ${AISOC_PULL_POLICY:-missing}- container_name: aisoc-actions- depends_on:- redis:- condition: service_healthy- kafka:- condition: service_healthy- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:8002:8085"- environment:- KAFKA_BOOTSTRAP_SERVERS: kafka:29092- REDIS_URL: redis://:${REDIS_PASSWORD:-redis_dev_secret}@redis:6379/5- DATABASE_URL: postgresql+asyncpg://aisoc_app:${AISOC_APP_DB_PASSWORD:-aisoc_app_dev_secret}@postgres:5432/aisoc- CORE_API_URL: http://api:8000- # Must match AISOC_ACTIONS_SERVICE_TOKEN on the api and slack-bot- # services. Unset in dev, where AISOC_DEV_MODE relaxes the check;- # require_service_auth returns 503 on every mutating route without it- # in any other environment, which is the intended failure.- AISOC_ACTIONS_SERVICE_TOKEN: ${AISOC_ACTIONS_SERVICE_TOKEN:-}- AISOC_DEV_MODE: ${AISOC_DEV_MODE:-0}- # What this deployment published, so the service can refuse the- # anonymous auth shim on an address someone else can reach. It- # binds 0.0.0.0 inside the container and cannot tell otherwise.- AISOC_PUBLISHED_BIND_ADDRS: ${AISOC_CONSOLE_BIND_ADDR:-${AISOC_BIND_ADDR:-127.0.0.1}},${AISOC_BIND_ADDR:-127.0.0.1}- networks:- - aisoc- restart: unless-stopped-- connectors:- # Scheduled vendor polling, the 84-connector registry, and the fan-out- # target for federated SIEM search.- #- # This was `profiles: ["full"]` on the reasoning that CORE accepts events- # over the ingest HTTP API instead. That is true of ingest and says nothing- # about the other two jobs: `api` in CORE sets CONNECTORS_SERVICE_URL to- # this hostname and AISOC_FEATURE_FED_SEARCH defaults on, so federated- # search fanned out to a name that does not resolve. Measured cost of- # moving it: 76.5 MiB resident, 586 MB of image. See- # docs/decisions/0007-connectors-and-actions-in-core.md.- #- # The poll scheduler is per-instance, so a fresh install schedules nothing- # until an operator saves a connector.- build:- context: ./services/connectors- dockerfile: Dockerfile- image: ghcr.io/beenuar/aisoc-connectors:${AISOC_VERSION:-latest}- pull_policy: ${AISOC_PULL_POLICY:-missing}- container_name: aisoc-connectors- depends_on:- kafka:- condition: service_healthy- redis:- condition: service_healthy- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:8088:8003"- environment:- # app/scripts/serve.py binds PORT, defaulting to 8087. The mapping above- # publishes container port 8003, so leaving PORT unset put the listener- # on a port nothing forwards to: `connectors:8003` was refused from- # inside the network and the published host port answered nothing.- PORT: 8003- # Without one of these the tenant-scope dependency fails closed and- # every route answers 503, including the catalog the API proxies — so- # the console silently fell back to the 26-entry catalog bundled in the- # API image instead of the 84 connectors the registry actually has.- SECRET_KEY: ${SECRET_KEY:-dev_secret_key_change_in_production}- AISOC_SERVICE_TOKEN: ${AISOC_SERVICE_TOKEN:-}- # The third way this service answered 503 to everything. `SECRET_KEY`'s- # compose default is one of INSECURE_SECRET_DEFAULTS, so- # `resolve_console_secret()` returns "" for it; with AISOC_SERVICE_TOKEN- # also blank, `require_console_or_service_auth` has no credential- # material at all and fails closed — correctly. The dev-mode escape that- # exists for exactly this case was set on actions, ueba, honeytokens,- # purple-team and slack-bot, and not on connectors. Dev mode only applies- # when nothing is configured, so once `make env` generates a real- # AISOC_SERVICE_TOKEN this flag stops having any effect.- AISOC_DEV_MODE: ${AISOC_DEV_MODE:-0}- # What this deployment published, so the service can refuse the- # anonymous auth shim on an address someone else can reach. It- # binds 0.0.0.0 inside the container and cannot tell otherwise.- AISOC_PUBLISHED_BIND_ADDRS: ${AISOC_CONSOLE_BIND_ADDR:-${AISOC_BIND_ADDR:-127.0.0.1}},${AISOC_BIND_ADDR:-127.0.0.1}- ENVIRONMENT: ${ENVIRONMENT:-production}- # The bundled Postgres and Kafka serve no certificates, and- # ENVIRONMENT now defaults to production — where a cleartext transport- # is a boot refusal, not a warning. Without these two the documented- # `make up` fails on a machine where there is nothing to fix, which is- # exactly how a security default gets patched out downstream. Stated- # here, in the file that chooses it, rather than weakened in the code- # that enforces it: a real deployment that leaves either unset is- # refused at boot.- AISOC_ALLOW_CLEARTEXT_DB: "1"- AISOC_ALLOW_CLEARTEXT_KAFKA: "1"- KAFKA_BOOTSTRAP_SERVERS: kafka:29092- REDIS_URL: redis://:${REDIS_PASSWORD:-redis_dev_secret}@redis:6379/6- DATABASE_URL: postgresql+asyncpg://aisoc_app:${AISOC_APP_DB_PASSWORD:-aisoc_app_dev_secret}@postgres:5432/aisoc- CORE_API_URL: http://api:8000- # The scheduler decrypts stored connector credentials through the vault- # and refuses to start without this key. .env.example documents it as- # one of the three required values, but no compose service forwarded it,- # so setting it had no effect and every deployment lost the scheduler to- # `connector.scheduler.start_failed` — while the container still- # reported itself started.- AISOC_CREDENTIAL_KEY: ${AISOC_CREDENTIAL_KEY:-}- AISOC_CREDENTIAL_KEY_ROTATION_FROM: ${AISOC_CREDENTIAL_KEY_ROTATION_FROM:-}- CROWDSTRIKE_CLIENT_ID: ${CROWDSTRIKE_CLIENT_ID:-}- CROWDSTRIKE_CLIENT_SECRET: ${CROWDSTRIKE_CLIENT_SECRET:-}- AWS_REGION: ${AWS_REGION:-us-east-1}- AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID:-}- AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY:-}- networks:- - aisoc- restart: unless-stopped-- threatintel:- # CORE — the only thing in the default install that puts *real*, externally- # sourced security data in front of a new user on day one.- #- # `app/clients/cisa_kev.py` fetches the CISA Known Exploited Vulnerabilities- # catalog: authoritative, public, and needing no API key. It has been wired- # as a scheduled handler the whole time and sat in a profile nobody starting- # out runs. Meanwhile the console shipped a /threat-intel page calling- # `/api/v1/threat-intel/indicators`, which had nothing behind it in CORE and- # was for a while filled with five invented IOCs. The missing feed and the- # fabrication were the same hole: there was no real data to show, so- # something invented some.- #- # Measured: 1.4 GB image, 219 MiB RSS steady state.- build:- context: ./services/threatintel- dockerfile: Dockerfile- image: ghcr.io/beenuar/aisoc-threatintel:${AISOC_VERSION:-latest}- pull_policy: ${AISOC_PULL_POLICY:-missing}- container_name: aisoc-threatintel- depends_on:- redis:- condition: service_healthy- kafka:- condition: service_healthy- qdrant:- condition: service_healthy- # OpenSearch is deliberately NOT a dependency any more. It was one because- # the lifespan called os_store.initialize() without a try/except, so an- # unreachable node killed the container — which made a `full`-profile store- # a hard requirement of a CORE service. Both that call and the pipeline's- # bulk index are best-effort now: with OpenSearch present you additionally- # get full-text IOC search, and without it the feeds still write to Qdrant,- # which is what the console reads. `make up-full` starts it and this- # service picks it up on its next restart.- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:8005:8005"- environment:- KAFKA_BOOTSTRAP_SERVERS: kafka:29092- REDIS_URL: redis://:${REDIS_PASSWORD:-redis_dev_secret}@redis:6379/7- # This service reads OPENSEARCH_HOST/PORT. It was given OPENSEARCH_URL,- # which its Settings has no field for and `extra="ignore"` discards, so- # the connection worked only because the defaults happened to match the- # compose service name. Set what is actually read.- OPENSEARCH_HOST: opensearch- OPENSEARCH_PORT: 9200- QDRANT_URL: http://qdrant:6333- NEO4J_URI: bolt://neo4j:7687- NEO4J_USER: neo4j- NEO4J_PASSWORD: ${NEO4J_PASSWORD:-neo4j_dev_secret}- # Same credential the API presents when it proxies /threat-intel/indicators.- SECRET_KEY: ${SECRET_KEY:-dev_secret_key_change_in_production}- AISOC_SERVICE_TOKEN: ${AISOC_SERVICE_TOKEN:-}- AISOC_DEV_MODE: ${AISOC_DEV_MODE:-0}- # What this deployment published, so the service can refuse the- # anonymous auth shim on an address someone else can reach. It- # binds 0.0.0.0 inside the container and cannot tell otherwise.- AISOC_PUBLISHED_BIND_ADDRS: ${AISOC_CONSOLE_BIND_ADDR:-${AISOC_BIND_ADDR:-127.0.0.1}},${AISOC_BIND_ADDR:-127.0.0.1}- MISP_URL: ${MISP_URL:-}- MISP_API_KEY: ${MISP_API_KEY:-}- OTX_API_KEY: ${OTX_API_KEY:-}- TAXII_URL: ${TAXII_URL:-}- TAXII_USERNAME: ${TAXII_USERNAME:-}- TAXII_PASSWORD: ${TAXII_PASSWORD:-}- networks:- - aisoc- restart: unless-stopped-- ueba:- # full profile — Behavioural baselining. Consumes its own topic; see docs/audit/REPOSITORY_REALITY.md.- profiles: ["full"]- build:- context: ./services/ueba- dockerfile: Dockerfile- image: ghcr.io/beenuar/aisoc-ueba:${AISOC_VERSION:-latest}- pull_policy: ${AISOC_PULL_POLICY:-missing}- container_name: aisoc-ueba- depends_on:- postgres:- condition: service_healthy- redis:- condition: service_healthy- kafka:- condition: service_healthy- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:8007:8004"- environment:- # Two roles, the same split the api service uses. This service manages- # its own schema through alembic, so it needs the owner to apply DDL and- # must not serve requests as it. The marker on the second line below is- # what tells scripts/check_runtime_db_role.py the owner is deliberate.- DATABASE_URL: postgresql+asyncpg://aisoc_app:${AISOC_APP_DB_PASSWORD:-aisoc_app_dev_secret}@postgres:5432/aisoc- # aisoc-db-role: owner — alembic applies this service's chain; the runtime role holds no DDL- DATABASE_MIGRATION_URL: postgresql+asyncpg://aisoc:${POSTGRES_PASSWORD:-aisoc_dev_secret}@postgres:5432/aisoc- # Read by the postgres init hook on a fresh volume and by the api- # migration runner otherwise. Passed here too so this service's own- # DATABASE_URL and the role's actual credential come from one value.- AISOC_APP_DB_PASSWORD: ${AISOC_APP_DB_PASSWORD:-aisoc_app_dev_secret}- REDIS_URL: redis://:${REDIS_PASSWORD:-redis_dev_secret}@redis:6379/8- KAFKA_BOOTSTRAP_SERVERS: kafka:29092- CLICKHOUSE_URL: http://aisoc:${CLICKHOUSE_PASSWORD:-clickhouse_dev_secret}@clickhouse:8123/aisoc- CORE_API_URL: http://api:8000- AISOC_SERVICE_TOKEN: ${AISOC_SERVICE_TOKEN:-}- AISOC_DEV_MODE: ${AISOC_DEV_MODE:-0}- # What this deployment published, so the service can refuse the- # anonymous auth shim on an address someone else can reach. It- # binds 0.0.0.0 inside the container and cannot tell otherwise.- AISOC_PUBLISHED_BIND_ADDRS: ${AISOC_CONSOLE_BIND_ADDR:-${AISOC_BIND_ADDR:-127.0.0.1}},${AISOC_BIND_ADDR:-127.0.0.1}- # A detached consumer used to read as `running`. /readyz answers 503 when- # the subscription is gone or the schema is not at head, so `docker compose- # ps` and `make _wait` now see the difference.- healthcheck:- test:- - CMD-SHELL- - "python -c \"import urllib.request,sys; r=urllib.request.urlopen('http://127.0.0.1:8004/readyz',timeout=2); sys.exit(0 if r.status==200 else 1)\" || exit 1"- interval: 10s- timeout: 5s- retries: 20- start_period: 90s- networks:- - aisoc- restart: unless-stopped-- honeytokens:- # Gated behind the `extras` profile: image isn't published to GHCR yet, so- # `aisoc serve` skips it by default. Opt in with `--profile extras`.- # `pull_policy: build` used to be the only thing that worked here,- # because the image this names had never been published; it is in the- # publish matrix now, so `--profile extras` pulls like every other- # service instead of spending ten minutes building one.- profiles: ["extras"]- build:- context: ./services/honeytokens- dockerfile: Dockerfile- image: ghcr.io/beenuar/aisoc-honeytokens:${AISOC_VERSION:-latest}- pull_policy: ${AISOC_PULL_POLICY:-missing}- container_name: aisoc-honeytokens- depends_on:- postgres:- condition: service_healthy- redis:- condition: service_healthy- kafka:- condition: service_healthy- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:8008:8005"- environment:- # Two roles, the same split the api service uses. This service manages- # its own schema through alembic, so it needs the owner to apply DDL and- # must not serve requests as it. The marker on the second line below is- # what tells scripts/check_runtime_db_role.py the owner is deliberate.- DATABASE_URL: postgresql+asyncpg://aisoc_app:${AISOC_APP_DB_PASSWORD:-aisoc_app_dev_secret}@postgres:5432/aisoc- # aisoc-db-role: owner — alembic applies this service's chain; the runtime role holds no DDL- DATABASE_MIGRATION_URL: postgresql+asyncpg://aisoc:${POSTGRES_PASSWORD:-aisoc_dev_secret}@postgres:5432/aisoc- # Read by the postgres init hook on a fresh volume and by the api- # migration runner otherwise. Passed here too so this service's own- # DATABASE_URL and the role's actual credential come from one value.- AISOC_APP_DB_PASSWORD: ${AISOC_APP_DB_PASSWORD:-aisoc_app_dev_secret}- REDIS_URL: redis://:${REDIS_PASSWORD:-redis_dev_secret}@redis:6379/9- KAFKA_BOOTSTRAP_SERVERS: kafka:29092- CORE_API_URL: http://api:8000- AISOC_SERVICE_TOKEN: ${AISOC_SERVICE_TOKEN:-}- AISOC_DEV_MODE: ${AISOC_DEV_MODE:-0}- # What this deployment published, so the service can refuse the- # anonymous auth shim on an address someone else can reach. It- # binds 0.0.0.0 inside the container and cannot tell otherwise.- AISOC_PUBLISHED_BIND_ADDRS: ${AISOC_CONSOLE_BIND_ADDR:-${AISOC_BIND_ADDR:-127.0.0.1}},${AISOC_BIND_ADDR:-127.0.0.1}- # A detached consumer used to read as `running`. /readyz answers 503 when- # the subscription is gone or the schema is not at head, so `docker compose- # ps` and `make _wait` now see the difference.- healthcheck:- test:- - CMD-SHELL- - "python -c \"import urllib.request,sys; r=urllib.request.urlopen('http://127.0.0.1:8005/readyz',timeout=2); sys.exit(0 if r.status==200 else 1)\" || exit 1"- interval: 10s- timeout: 5s- retries: 20- start_period: 90s- networks:- - aisoc- restart: unless-stopped-- purple-team:- # Gated behind the `extras` profile: image isn't published to GHCR yet, so- # `aisoc serve` skips it by default. Opt in with `--profile extras`.- # `pull_policy: build` used to be the only thing that worked here,- # because the image this names had never been published; it is in the- # publish matrix now, so `--profile extras` pulls like every other- # service instead of spending ten minutes building one.- profiles: ["extras"]- build:- context: ./services/purple-team- dockerfile: Dockerfile- image: ghcr.io/beenuar/aisoc-purple-team:${AISOC_VERSION:-latest}- pull_policy: ${AISOC_PULL_POLICY:-missing}- container_name: aisoc-purple-team- depends_on:- postgres:- condition: service_healthy- redis:- condition: service_healthy- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:8006:8006"- environment:- # Two roles, the same split the api service uses. This service manages- # its own schema through alembic, so it needs the owner to apply DDL and- # must not serve requests as it. The marker on the second line below is- # what tells scripts/check_runtime_db_role.py the owner is deliberate.- DATABASE_URL: postgresql+asyncpg://aisoc_app:${AISOC_APP_DB_PASSWORD:-aisoc_app_dev_secret}@postgres:5432/aisoc- # aisoc-db-role: owner — alembic applies this service's chain; the runtime role holds no DDL- DATABASE_MIGRATION_URL: postgresql+asyncpg://aisoc:${POSTGRES_PASSWORD:-aisoc_dev_secret}@postgres:5432/aisoc- # Read by the postgres init hook on a fresh volume and by the api- # migration runner otherwise. Passed here too so this service's own- # DATABASE_URL and the role's actual credential come from one value.- AISOC_APP_DB_PASSWORD: ${AISOC_APP_DB_PASSWORD:-aisoc_app_dev_secret}- REDIS_URL: redis://:${REDIS_PASSWORD:-redis_dev_secret}@redis:6379/10- CORE_API_URL: http://api:8000- ATTCK_DATA_PATH: /app/data/enterprise-attack.json- AISOC_SERVICE_TOKEN: ${AISOC_SERVICE_TOKEN:-}- AISOC_DEV_MODE: ${AISOC_DEV_MODE:-0}- # What this deployment published, so the service can refuse the- # anonymous auth shim on an address someone else can reach. It- # binds 0.0.0.0 inside the container and cannot tell otherwise.- AISOC_PUBLISHED_BIND_ADDRS: ${AISOC_CONSOLE_BIND_ADDR:-${AISOC_BIND_ADDR:-127.0.0.1}},${AISOC_BIND_ADDR:-127.0.0.1}- # A detached consumer used to read as `running`. /readyz answers 503 when- # the subscription is gone or the schema is not at head, so `docker compose- # ps` and `make _wait` now see the difference.- healthcheck:- test:- - CMD-SHELL- - "python -c \"import urllib.request,sys; r=urllib.request.urlopen('http://127.0.0.1:8006/readyz',timeout=2); sys.exit(0 if r.status==200 else 1)\" || exit 1"- interval: 10s- timeout: 5s- retries: 20- start_period: 90s- networks:- - aisoc- restart: unless-stopped-- realtime:- build:- context: ./services/realtime- dockerfile: Dockerfile- image: ghcr.io/beenuar/aisoc-realtime:${AISOC_VERSION:-latest}- pull_policy: ${AISOC_PULL_POLICY:-missing}- container_name: aisoc-realtime- depends_on:- redis:- condition: service_healthy- kafka:- condition: service_healthy- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:8086:4000"- environment:- KAFKA_BOOTSTRAP_SERVERS: kafka:29092- REDIS_URL: redis://:${REDIS_PASSWORD:-redis_dev_secret}@redis:6379- REDIS_PASSWORD: ${REDIS_PASSWORD:-redis_dev_secret}- PORT: 4000- # Both are generated by `make up`. Neither was passed to this container- # before, which is what made the ticket verifier fall back to a constant- # published in this repository and the /internal/* guard treat every- # caller as authorized.- AISOC_REALTIME_JWT_SECRET: ${AISOC_REALTIME_JWT_SECRET:-}- REALTIME_INTERNAL_TOKEN: ${REALTIME_INTERNAL_TOKEN:-}- networks:- - aisoc- restart: unless-stopped-- # ChatOps adapter. Holds no state — forwards `/aisoc …` slash commands and- # approval-card button clicks to services/api and services/actions. Runs- # under the `chatops` profile so a default `docker compose up` doesn't fail- # when the operator has not yet provisioned a Slack app or service tokens.- slack-bot:- build:- context: ./services/slack-bot- dockerfile: Dockerfile- image: ghcr.io/beenuar/aisoc-slack-bot:${AISOC_VERSION:-latest}- pull_policy: ${AISOC_PULL_POLICY:-missing}- container_name: aisoc-slack-bot- depends_on:- - api- - actions- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:8009:8089"- profiles:- - chatops- environment:- SLACK_BOT_TOKEN: ${SLACK_BOT_TOKEN:-}- SLACK_SIGNING_SECRET: ${SLACK_SIGNING_SECRET:-}- AISOC_API_BASE_URL: http://api:8000- AISOC_ACTIONS_BASE_URL: http://actions:8085- AISOC_API_SERVICE_TOKEN: ${AISOC_SLACK_API_TOKEN:-}- AISOC_ACTIONS_SERVICE_TOKEN: ${AISOC_SLACK_ACTIONS_TOKEN:-}- AISOC_DEFAULT_TENANT_ID: ${AISOC_DEFAULT_TENANT_ID:-00000000-0000-0000-0000-000000000000}- AISOC_WEB_BASE_URL: ${AISOC_WEB_BASE_URL:-http://localhost:3000}- AISOC_SLACK_BOT_PORT: 8089- AISOC_HTTP_TIMEOUT_SECONDS: 10- # Proactive approval cards. Empty channel disables posting, which is- # the right default: a bot that posts into a channel nobody chose is- # worse than one that posts nowhere.- SLACK_APPROVALS_CHANNEL: ${SLACK_APPROVALS_CHANNEL:-}- # Shared with the agents service, which calls /internal/approval-card.- AISOC_INTERNAL_TOKEN: ${AISOC_INTERNAL_TOKEN:-}- AISOC_DEV_MODE: ${AISOC_DEV_MODE:-0}- # What this deployment published, so the service can refuse the- # anonymous auth shim on an address someone else can reach. It- # binds 0.0.0.0 inside the container and cannot tell otherwise.- AISOC_PUBLISHED_BIND_ADDRS: ${AISOC_CONSOLE_BIND_ADDR:-${AISOC_BIND_ADDR:-127.0.0.1}},${AISOC_BIND_ADDR:-127.0.0.1}- networks:- - aisoc- restart: unless-stopped-- web:- build:- context: .- dockerfile: apps/web/Dockerfile- image: ghcr.io/beenuar/aisoc-web:${AISOC_VERSION:-latest}- pull_policy: ${AISOC_PULL_POLICY:-missing}- container_name: aisoc-web- depends_on:- - api- - realtime- ports:- # The console has its own bind address because it is the only port a- # browser needs. Everything it talks to — the API, the agents service,- # the realtime gateway — is reached through this server's same-origin- # proxy on the internal network, so exposing AiSOC to a single host's- # LAN means publishing one port, not fourteen.- #- # AISOC_CONSOLE_BIND_ADDR=0.0.0.0 # reachable at http://<host>:3000- #- # Falls back to AISOC_BIND_ADDR and then to loopback, so the default- # install is still reachable only from the machine it runs on. Put it- # behind a reverse proxy with TLS before anything hostile can route to- # it; see apps/docs/docs/deployment/single-host.md.- - "${AISOC_CONSOLE_BIND_ADDR:-${AISOC_BIND_ADDR:-127.0.0.1}}:3000:3000"- environment:- # Where this server proxies to. These are read at container start by- # apps/web/docker-entrypoint.sh, which re-points the compiled rewrite- # table at them — so unlike the NEXT_PUBLIC_* pair that used to sit- # here, they take effect on a pulled image.- API_URL: ${AISOC_API_URL:-http://api:8000}- AGENTS_URL: ${AISOC_AGENTS_URL:-http://agents:8084}- REALTIME_URL: ${AISOC_REALTIME_URL:-http://realtime:4000}- # NOTE: do not add NEXT_PUBLIC_* here. Next inlines those into the- # JavaScript bundle at `next build` time, so on the pulled image this- # service runs they are read by nothing. NEXT_PUBLIC_API_URL and- # NEXT_PUBLIC_WS_URL were set here and did exactly nothing; an operator- # pointing the console at their own host edited them, saw no change,- # and had no way to tell the setting was inert. The browser needs no- # absolute URL at all — the bundle calls same-origin paths and this- # server proxies them onward.- AISOC_DEMO_MODE: ${AISOC_DEMO_MODE:-false}- NODE_ENV: production- networks:- - aisoc- restart: unless-stopped-- # ─── Observability ────────────────────────────────────────────────────────────-- prometheus:- image: prom/prometheus:v2.48.0- container_name: aisoc-prometheus- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:9091:9090"- volumes:- - ./infra/docker/prometheus.yml:/etc/prometheus/prometheus.yml:ro- # Phase 2.4 — alert rules mounted as a directory so adding a- # new rule file under `infra/docker/alerts/` is picked up on- # next `docker compose restart prometheus` (or a SIGHUP).- - ./infra/docker/alerts:/etc/prometheus/alerts:ro- # Phase 2.4 — Prometheus needs to reach the Alertmanager service- # before it can forward alerts. The `depends_on` keeps `docker- # compose up monitoring` healthy on cold boot — without it,- # Prometheus logs "level=warn ... alertmanager_url=... err=lookup"- # for the first ~30s of every restart.- depends_on:- - alertmanager- networks:- - aisoc- profiles:- - monitoring-- # Phase 2.4 — in-cluster Alertmanager. Receives firing alerts- # from Prometheus, groups + deduplicates them per- # `infra/docker/alertmanager.yml`, and forwards them to the- # configured receivers. The dev stack ships with a no-op webhook- # receiver that points at the API's `/internal/alerts/sink`- # endpoint (which logs the payload); production operators replace- # the receiver block with PagerDuty / Slack / Opsgenie. The- # `monitoring` profile keeps it out of `docker compose up`'s- # default surface so a stock `aisoc serve` doesn't fail when the- # operator hasn't provisioned downstream alerting.- alertmanager:- image: prom/alertmanager:v0.27.0- container_name: aisoc-alertmanager- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:9094:9093"- volumes:- - ./infra/docker/alertmanager.yml:/etc/alertmanager/alertmanager.yml:ro- command:- - '--config.file=/etc/alertmanager/alertmanager.yml'- - '--storage.path=/alertmanager'- # External URL is what Alertmanager uses to render deep-links- # in notification templates (e.g. "Silence in Alertmanager").- # Port 9094 here matches the host-side mapping so the link- # works from a browser on the operator's laptop.- - '--web.external-url=http://localhost:9094'- # NOTE: The Alertmanager binary does NOT substitute shell-style- # env vars in `alertmanager.yml`. To change receivers per- # environment, mount a different config file (production- # deployments do this through Helm values templating).- networks:- - aisoc- profiles:- - monitoring-- # ── Tracing ────────────────────────────────────────────────────────────────- # The services default OTEL_EXPORTER_OTLP_ENDPOINT to- # http://otel-collector:4317 and no collector shipped in any compose file,- # so out of the box every span went into a connection error. That is worse- # than having no tracing at all: the code looks instrumented, the logs fill- # with export failures, and nobody can tell whether a missing trace means a- # missing span or a missing collector.- #- # Scope, stated because the docs previously overstated it: api, agents,- # ueba and honeytokens are instrumented. ingest (Go) and realtime- # (TypeScript) are not, and they are the two ends of the Kafka spine.- otel-collector:- image: otel/opentelemetry-collector-contrib:0.109.0- container_name: aisoc-otel-collector- command: ["--config=/etc/otel/collector.yaml"]- depends_on:- - tempo- volumes:- - ./infra/docker/otel/collector.yaml:/etc/otel/collector.yaml:ro- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:4317:4317"- - "${AISOC_BIND_ADDR:-127.0.0.1}:4318:4318"- healthcheck:- test: ["CMD", "/otelcol-contrib", "--version"]- interval: 30s- timeout: 5s- retries: 3- networks:- - aisoc- profiles:- - monitoring-- tempo:- image: grafana/tempo:2.6.0- container_name: aisoc-tempo- command: ["-config.file=/etc/tempo/tempo.yaml"]- user: "0:0"- volumes:- - ./infra/docker/otel/tempo.yaml:/etc/tempo/tempo.yaml:ro- - tempo_data:/var/tempo- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:3200:3200"- networks:- - aisoc- profiles:- - monitoring-- grafana:- image: grafana/grafana:10.2.0- container_name: aisoc-grafana- depends_on:- - prometheus- - tempo- ports:- - "${AISOC_BIND_ADDR:-127.0.0.1}:3001:3000"- # Default admin password is intentionally weak for the dev compose stack;- # the 127.0.0.1 binding above keeps the dashboard off the LAN. Override- # GF_SECURITY_ADMIN_PASSWORD in production deployments (helm/terraform).- environment:- GF_SECURITY_ADMIN_PASSWORD: ${GRAFANA_ADMIN_PASSWORD:-admin}- volumes:- - ./infra/docker/grafana:/etc/grafana/provisioning:ro- networks:- - aisoc- profiles:- - monitoringdiff --git a/services/agents/app/llm/prompt_registry.py b/services/agents/app/llm/prompt_registry.pyindex 1e09e111e..ce8434b85 100644--- a/services/agents/app/llm/prompt_registry.py+++ b/services/agents/app/llm/prompt_registry.py@@ -561,7 +561,7 @@ def default_registry() -> PromptRegistry:reg.register("insider_threat.system", "1", _INSIDER_THREAT_SYSTEM)reg.register("phishing.system", "1", _PHISHING_SYSTEM)reg.register("recon.system", "1", _RECON_SYSTEM)- reg.register("forensic.system", "1", _FORENSIC_SYSTEM)+ reg.register("forensic.system", "2", _FORENSIC_SYSTEM)reg.register("responder.system", "1", _RESPONDER_SYSTEM)reg.register("report_writer.system", "1", _REPORT_WRITER_SYSTEM)reg.register("playbook_drafter.system", "1", _PLAYBOOK_DRAFTER_SYSTEM)diff --git a/services/agents/app/llm/prompts.lock.json b/services/agents/app/llm/prompts.lock.jsonindex 94be40b64..71c9ed426 100644--- a/services/agents/app/llm/prompts.lock.json+++ b/services/agents/app/llm/prompts.lock.json@@ -48,8 +48,8 @@"version": "1"},"forensic.system": {- "sha256": "f2eb47f8a0c9bb7d5d1f30c5569bdd698a0f961d22b10f0fd6b6e9470464e928",- "version": "1"+ "sha256": "3084c2e15386500b1c61e741b01ef36456e6b6a1ddfde4f6c51e103e9ac2472a",+ "version": "2"},"hunt.system": {"sha256": "cb7225c08a170bf2511ec0a04936cf91fd83dce38659f72de872481c217e5dd5",diff --git a/services/api/app/api/v1/endpoints/approvals.py b/services/api/app/api/v1/endpoints/approvals.pyindex 08bc5be14..4c5022849 100644--- a/services/api/app/api/v1/endpoints/approvals.py+++ b/services/api/app/api/v1/endpoints/approvals.py@@ -40,7 +40,6 @@ from sqlalchemy import and_, func, selectfrom app.api.v1.deps import AuthUser, require_permissionfrom app.core.config import settings-from app.core.security import ROLE_PERMISSIONS, has_permissionfrom app.db.rls import TenantDBSessionfrom app.models.responder import AgentApprovalfrom app.services.actions_client import ActionsServiceError, decide_action, submit_action@@ -329,7 +328,9 @@ async def _dispatch_decision("user_id": str(user.user_id),"tenant_id": str(user.tenant_id),"email": getattr(user, "email", None),- "roles": [role] if role else (list(getattr(user, "roles", []) or []) if not role else []),+ # Singular on this class; sent as a list because that is the shape+ # `services/actions` reads.+ "roles": [user.role] if user.role else [],"permissions": permissions,}diff --git a/services/api/app/api/v1/endpoints/cases.py b/services/api/app/api/v1/endpoints/cases.pyindex dc9def0ff..6a286dbfa 100644--- a/services/api/app/api/v1/endpoints/cases.py+++ b/services/api/app/api/v1/endpoints/cases.py@@ -46,8 +46,7 @@ import httpxfrom fastapi import APIRouter, Depends, HTTPException, Query, Response, statusfrom fastapi.responses import HTMLResponse, PlainTextResponsefrom pydantic import BaseModel, Field-from sqlalchemy import text-from sqlalchemy import select+from sqlalchemy import select, textfrom app.api.v1.deps import AuthUser, DBSession, require_permissionfrom app.core.logging import safe_log_value@@ -141,11 +140,6 @@ def _status_transition_ok(current: str, target: str) -> bool:return target in _TRANSITIONS.get(current, set())-class ReopenCaseRequest(BaseModel):- reason: str = Field(..., min_length=3, max_length=500)- target_status: Literal["new", "triaged", "investigating"] = "investigating"--class CreateCaseRequest(BaseModel):title: str = Field(..., min_length=3)description: str | None = None@@ -724,7 +718,11 @@ async def update_case(if not _status_transition_ok(existing.status, body.status):raise HTTPException(status_code=422,- detail=f"Invalid status transition: {existing.status} → {body.status}. Allowed: {sorted(_TRANSITIONS.get(existing.status, set())) or 'none'}; reopening requires the reopen action.",+ detail=(+ f"Invalid status transition: {existing.status} → {body.status}. "+ f"Allowed: {sorted(_TRANSITIONS.get(existing.status, set())) or 'none'}; "+ "backwards moves require POST /cases/{id}/reopen."+ ),)now = datetime.now(UTC)@@ -907,68 +905,6 @@ def case_queue_for(case_row: Any, queues: list[Any]) -> Any:queues,)[email protected]("/{case_id}/reopen", response_model=CaseResponse, summary="Reopen a case")-async def reopen_case(- case_id: str,- body: ReopenCaseRequest,- db: DBSession,- user: Annotated[AuthUser, Depends(require_permission("cases:write"))],-) -> CaseResponse:- """Explicit reopen — the sanctioned backwards status move.-- PATCH /cases/{id} is forward-only by design (see _status_transition_ok);- this endpoint is the 'reopen action' its 422 refers to. It resets the- future-stage timestamps so the reopened case reads cleanly on the board,- and records a timeline event.- """- import json as _json-- cid = await _resolve_case_id(case_id, db, user.tenant_id)- existing = (- await db.execute(- text("SELECT status FROM aisoc_cases WHERE id = :id AND tenant_id = :tenant_id").bindparams(id=cid, tenant_id=user.tenant_id)- )- ).fetchone()- if not existing:- raise HTTPException(status_code=404, detail="Case not found.")- if _status_transition_ok(existing.status, body.target_status):- raise HTTPException(- status_code=422,- detail=f"{existing.status} → {body.target_status} is a forward transition; use PATCH /cases/{case_id}.",- )-- now = datetime.now(UTC)- row = (- await db.execute(- text(- "UPDATE aisoc_cases SET status = :status, reopened_at = :now, "- "triaged_at = NULL, resolved_at = NULL, closed_at = NULL, updated_at = :now "- "WHERE id = :id AND tenant_id = :tenant_id RETURNING *"- ).bindparams(id=cid, tenant_id=user.tenant_id, status=body.target_status, now=now)- )- ).fetchone()- if row is None:- raise HTTPException(status_code=404, detail="Case not found.")- try:- await db.execute(- text(- "INSERT INTO case_timeline_events (case_id, tenant_id, event_type, actor_type, summary, detail) "- "VALUES (:case_id, :tenant_id, 'status_change', 'user', :summary, CAST(:detail AS JSONB))"- ).bindparams(- case_id=cid,- tenant_id=user.tenant_id,- summary=f"Case reopened: {existing.status} → {body.target_status}",- detail=_json.dumps({"reason": body.reason, "from": existing.status, "to": body.target_status}),- )- )- await db.commit()- except Exception as exc:- await db.rollback()- logger.exception("cases.reopen.timeline_event_failed case=%s", cid)- raise HTTPException(status_code=503, detail="Database error") from exc- return _row_to_case(row)--@router.post("/{case_id}/alerts", response_model=CaseResponse, summary="Link alerts to a case")async def add_alerts(case_id: str, body: AddAlertsRequest, db: DBSession, user: Annotated[AuthUser, Depends(require_permission("cases:write"))]@@ -1648,14 +1584,17 @@ async def case_investigate(try:st_row = (await db.execute(- text("SELECT status FROM aisoc_cases WHERE id = :id AND tenant_id = :tenant_id").bindparams(id=cid, tenant_id=user.tenant_id)+ text(+ "SELECT status FROM aisoc_cases WHERE id = :id AND tenant_id = :tenant_id"+ ).bindparams(id=cid, tenant_id=user.tenant_id))).fetchone()if st_row and st_row.status != "investigating" and _status_transition_ok(st_row.status, "investigating"):await db.execute(- text("UPDATE aisoc_cases SET status = 'investigating', updated_at = :now WHERE id = :id AND tenant_id = :tenant_id").bindparams(- id=cid, tenant_id=user.tenant_id, now=datetime.now(UTC)- )+ text(+ "UPDATE aisoc_cases SET status = 'investigating', updated_at = :now "+ "WHERE id = :id AND tenant_id = :tenant_id"+ ).bindparams(id=cid, tenant_id=user.tenant_id, now=datetime.now(UTC)))await db.commit()except Exception: # noqa: BLE001 — never block the launch on the status bump@@ -1666,7 +1605,7 @@ async def case_investigate("POST",f"/api/v1/cases/{cid}/investigate",json={- "alert_summary": body.alert_summary or (raw_alert_payload.get("alerts", [{}])[0].get("title") if raw_alert_payload.get("alerts") else ""),+ "alert_summary": summary,"raw_alert": raw_alert_payload,"tenant_id": str(user.tenant_id),},@@ -1952,7 +1891,11 @@ async def case_investigation_pdf(user: AuthUser,) -> Response:safe_run_id = quote(run_id, safe="")- resp = await _agents_proxy("GET", f"/api/v1/investigations/{safe_run_id}/report.pdf", headers={"X-AiSOC-Tenant-ID": str(user.tenant_id)})+ resp = await _agents_proxy(+ "GET",+ f"/api/v1/investigations/{safe_run_id}/report.pdf",+ headers={"X-AiSOC-Tenant-ID": str(user.tenant_id)},+ )if resp.status_code >= 400:raise HTTPException(status_code=resp.status_code, detail=resp.text)safe_case_id = _safe_filename_segment(case_id)diff --git a/services/api/app/api/v1/endpoints/cases.py.bak.1790945982 b/services/api/app/api/v1/endpoints/cases.py.bak.1790945982deleted file mode 100644index 8e9d21727..000000000--- a/services/api/app/api/v1/endpoints/cases.py.bak.1790945982+++ /dev/null@@ -1,1478 +0,0 @@-"""First-class case management (tier2-cases).--Implements the full case lifecycle: new → triaged → investigating → contained →-resolved → closed. Each case aggregates multiple alerts, maintains an observable-graph, evidence chain, and MITRE ATT&CK coverage map.--Tenant isolation------------------Every query in this module is scoped by ``tenant_id = user.tenant_id``. The-``aisoc_cases`` table has carried a ``tenant_id`` column since migration 012-but the API historically did not filter by it, allowing any authenticated-caller to read or mutate another tenant's cases just by guessing a UUID (or-by guessing the short ``INC-001`` form). Companion tables-(``aisoc_case_comments``, ``aisoc_case_tasks``) gained a ``tenant_id``-column in migration 044 and are backfilled from their parent case so the-same RLS predicate applies uniformly.--The ``_resolve_case_id`` helper is also tenant-scoped — looking up-``INC-001`` for tenant A must never return tenant B's INC-001.--Endpoints-----------* ``GET /cases`` List cases (filterable by status/severity/assignee).-* ``POST /cases`` Create a new case.-* ``GET /cases/{id}`` Retrieve a case.-* ``PATCH /cases/{id}`` Update case fields (status, severity, assignee, …).-* ``POST /cases/{id}/alerts`` Link alerts to a case.-* ``POST /cases/{id}/observables`` Add / update observable graph nodes/edges.-* ``POST /cases/{id}/comments`` Add a comment / timeline entry.-* ``GET /cases/{id}/comments`` List comments for a case.-* ``GET /cases/{id}/evidence`` Export the evidence chain as a structured report.-"""--from __future__ import annotations--import logging-import os-import re-import uuid-from datetime import UTC, datetime-from typing import Annotated, Any, Literal-from urllib.parse import quote--import httpx-from fastapi import APIRouter, Depends, HTTPException, Query, Response, status-from fastapi.responses import HTMLResponse, PlainTextResponse-from pydantic import BaseModel, Field-from sqlalchemy import text--from app.api.v1.deps import AuthUser, DBSession, require_permission-from app.core.logging import safe_log_value-from app.services.case_fanout import (- FanoutResult,- fanout_create_case,- fanout_status_change,-)-from app.services.case_postmortem import build_case_postmortem-from app.services.case_postmortem_html import render_case_postmortem_html-from app.services.case_summary import build_case_summary-from app.services.case_summary_html import render_case_summary_html--logger = logging.getLogger(__name__)--router = APIRouter(prefix="/cases", tags=["cases"])--# The agents service owns the actual investigation orchestrator (Pillar-1).-# We keep the route surface on `/cases/{id}/investigate` and-# `/cases/{id}/investigations/{run_id}` for the web console and proxy through-# to the agents service so the front end has a single, stable API origin.-_AGENTS_URL = (os.getenv("AGENTS_SERVICE_URL") or os.getenv("AGENTS_API_URL") or "http://agents:8084").rstrip("/")--# Tight allowlist for proxied request paths. We only ever proxy to a fixed-# upstream (`_AGENTS_URL`) on a known set of investigation routes, so the-# path must be a pure relative path — no scheme, no host, no control bytes,-# no traversal sequences. This neutralises partial-SSRF (an attacker cannot-# redirect the request to a different host) and log-injection (the path-# can never contain CR/LF or other control characters).-_SAFE_PROXY_PATH_RE = re.compile(r"^/[A-Za-z0-9_\-./%]*$")---def _validate_agents_path(path: str) -> str:- """Reject any proxied path that isn't a tightly constrained relative path."""- if (- not isinstance(path, str) or not _SAFE_PROXY_PATH_RE.match(path) or ".." in path or path.startswith("//") # protocol-relative URL- ):- raise HTTPException(status_code=400, detail="invalid_request_path")- return path---# ────────────────────────────────────────────────────────────────────────────-# Pydantic schemas-# ────────────────────────────────────────────────────────────────────────────--CaseStatus = Literal["new", "triaged", "investigating", "contained", "resolved", "closed"]-CaseSeverity = Literal["info", "low", "medium", "high", "critical"]--# Valid forward-only state transitions-_TRANSITIONS: dict[str, set[str]] = {- "new": {"triaged"},- "triaged": {"investigating"},- "investigating": {"contained", "resolved"},- "contained": {"resolved"},- "resolved": {"closed"},- "closed": set(),-}---class CreateCaseRequest(BaseModel):- title: str = Field(..., min_length=3)- description: str | None = None- severity: CaseSeverity = "medium"- assignee: str | None = None- alert_ids: list[uuid.UUID] = Field(default_factory=list)- mitre_techniques: list[str] = Field(default_factory=list)- compliance_frameworks: list[str] = Field(default_factory=list)- tags: dict[str, str] = Field(default_factory=dict)- sla_due_at: datetime | None = None- # Workstream 8 — bidirectional ITSM. When the operator selects one or- # more connector instances at create time, the API fans the new case- # out to those external systems and persists the linkage in- # ``case_external_refs``. Connector instances must be tenant-owned,- # enabled, and declare ``Capability.PUSH_CASE`` (e.g. Jira,- # ServiceNow). Empty list = case stays AiSOC-only.- push_to_connector_ids: list[uuid.UUID] = Field(default_factory=list)---class UpdateCaseRequest(BaseModel):- title: str | None = None- description: str | None = None- severity: CaseSeverity | None = None- status: CaseStatus | None = None- assignee: str | None = None- mitre_techniques: list[str] | None = None- compliance_frameworks: list[str] | None = None- tags: dict[str, str] | None = None- sla_due_at: datetime | None = None---class AddAlertsRequest(BaseModel):- alert_ids: list[uuid.UUID]---class ObservableNode(BaseModel):- id: str- kind: Literal["ip", "user", "host", "domain", "hash", "file", "process", "alert"]- value: str- tags: list[str] = Field(default_factory=list)---class ObservableEdge(BaseModel):- source: str- target: str- relation: str---class UpdateObservablesRequest(BaseModel):- nodes: list[ObservableNode] = Field(default_factory=list)- edges: list[ObservableEdge] = Field(default_factory=list)---class AddCommentRequest(BaseModel):- body: str = Field(..., min_length=1)- is_system: bool = False---class CaseResponse(BaseModel):- id: uuid.UUID- case_number: str | None = None- title: str- description: str | None- severity: str- status: str- assignee: str | None- mitre_techniques: list[str]- alert_ids: list[uuid.UUID]- observable_graph: dict[str, Any]- evidence_chain: list[dict[str, Any]]- compliance_frameworks: list[str]- opened_at: datetime- triaged_at: datetime | None- resolved_at: datetime | None- closed_at: datetime | None- created_at: datetime- updated_at: datetime- created_by: str | None- tags: dict[str, Any]- sla_due_at: datetime | None- # Workstream 8 — populated only by ``POST /cases`` (create-time- # fan-out) and ``PATCH /cases/{id}`` when ``status`` is changed.- # The list is per-connector-instance and includes failures so the- # UI can show "Jira ✓ · ServiceNow ✗ (timeout)" beside the case- # without a follow-up GET. ``None`` (default) means "no fan-out- # was attempted on this call" and is distinct from ``[]`` ("fan-out- # was attempted but no connectors were selected").- fanout_results: list[FanoutResult] | None = None---class CommentResponse(BaseModel):- id: uuid.UUID- case_id: uuid.UUID- author: str | None- body: str- is_system: bool- created_at: datetime---class EvidenceReport(BaseModel):- case_id: uuid.UUID- title: str- severity: str- status: str- alert_count: int- mitre_techniques: list[str]- compliance_frameworks: list[str]- evidence_chain: list[dict[str, Any]]- generated_at: datetime---# Timeline / tasks-TaskStatus = Literal["todo", "in_progress", "done"]---class TimelineEvent(BaseModel):- id: str- type: str # 'created' | 'status_change' | 'comment' | 'alert_linked' | 'task'- timestamp: datetime- title: str- description: str | None = None- actor: str | None = None---class TimelineResponse(BaseModel):- events: list[TimelineEvent]---class CreateTaskRequest(BaseModel):- title: str = Field(..., min_length=1)- status: TaskStatus = "todo"- assignee: str | None = None- due_at: datetime | None = Field(None, alias="dueAt")-- model_config = {"populate_by_name": True}---class UpdateTaskRequest(BaseModel):- title: str | None = None- status: TaskStatus | None = None- assignee: str | None = None- due_at: datetime | None = Field(None, alias="dueAt")-- model_config = {"populate_by_name": True}---class TaskResponse(BaseModel):- id: str- title: str- status: TaskStatus- assignee: str | None = None- due_at: datetime | None = Field(None, alias="dueAt")- created_at: datetime = Field(..., alias="createdAt")-- model_config = {"populate_by_name": True}---def _row_to_task(row: Any) -> TaskResponse:- return TaskResponse(- id=str(row.id),- title=row.title,- status=row.status,- assignee=row.assignee,- dueAt=row.due_at,- createdAt=row.created_at,- )---# Investigation proxy schemas (forwarded to the agents service).-class InvestigateRequest(BaseModel):- alert_summary: str | None = ""---# ────────────────────────────────────────────────────────────────────────────-# Helpers-# ────────────────────────────────────────────────────────────────────────────---def _coerce_mitre(values: Any) -> list[str]:- """Normalize mitre_techniques to a list of technique IDs.-- Accepts list[str] or list[dict] like ``{"id": "T1041", "name": "..."}``.- Older seed fixtures stored objects; newer ones store flat IDs.- """- if not values:- return []- out: list[str] = []- for item in values:- if isinstance(item, str):- out.append(item)- elif isinstance(item, dict):- tid = item.get("id") or item.get("technique_id") or item.get("name")- if tid:- out.append(str(tid))- return out---def _coerce_tags(value: Any) -> dict[str, Any]:- """aisoc_cases.tags is JSONB which may be an object or array."""- if isinstance(value, dict):- return value- if isinstance(value, list):- return {"labels": [str(v) for v in value]}- return {}---def _row_to_case(row: Any) -> CaseResponse:- return CaseResponse(- id=row.id,- case_number=getattr(row, "case_number", None),- title=row.title,- description=row.description,- severity=row.severity,- status=row.status,- assignee=row.assignee,- mitre_techniques=_coerce_mitre(row.mitre_techniques),- alert_ids=list(row.alert_ids or []),- observable_graph=dict(row.observable_graph or {}),- evidence_chain=list(row.evidence_chain or []),- compliance_frameworks=list(row.compliance_frameworks or []),- opened_at=row.opened_at,- triaged_at=row.triaged_at,- resolved_at=row.resolved_at,- closed_at=row.closed_at,- created_at=row.created_at,- updated_at=row.updated_at,- created_by=row.created_by,- tags=_coerce_tags(row.tags),- sla_due_at=row.sla_due_at,- )---async def _resolve_case_id(case_id: str, db: Any, tenant_id: uuid.UUID) -> uuid.UUID:- """Resolve a case identifier (UUID **or** human-readable case_number) to a UUID- scoped to the caller's tenant.-- The web console and demo deeplinks use short identifiers like ``INC-001``,- while the database primary key is a UUID. Accepting both forms here keeps- the API ergonomic without forcing the front end to do a separate lookup.-- Tenant scoping is enforced in both branches:-- * For the UUID branch we return the parsed UUID without touching the DB;- callers must still apply ``WHERE tenant_id = :tenant_id`` in their own- query, which they all do. (We don't probe existence here because every- caller does its own SELECT and would issue a redundant round-trip.)- * For the ``case_number`` branch we look up the row with- ``WHERE tenant_id = :tenant_id AND case_number = :case_number`` so an- attacker can't enumerate another tenant's INC-NNN identifiers.-- Raises ``404`` if the identifier doesn't match either form within the- caller's tenant.- """- case_id = (case_id or "").strip()- if not case_id:- raise HTTPException(status_code=404, detail="Case not found.")-- # Try UUID first — that's the canonical form. Tenant scoping is enforced- # by the per-endpoint queries that consume the returned UUID, so we- # don't need a DB round-trip just to validate the format.- try:- return uuid.UUID(case_id)- except (ValueError, AttributeError):- pass-- # Fall back to case_number lookup. Use a parameterized query — never- # interpolate the raw string into SQL — and scope by tenant so the- # short-id form can't be used as a cross-tenant oracle.- row = (- await db.execute(- text(- "SELECT id FROM aisoc_cases WHERE tenant_id = :tenant_id AND case_number = :case_number ORDER BY created_at DESC LIMIT 1"- ).bindparams(tenant_id=tenant_id, case_number=case_id)- )- ).fetchone()- if not row:- raise HTTPException(status_code=404, detail="Case not found.")- return row.id---# ────────────────────────────────────────────────────────────────────────────-# Endpoints-# ────────────────────────────────────────────────────────────────────────────--[email protected]("", response_model=list[CaseResponse], summary="List cases")-async def list_cases(- db: DBSession,- user: AuthUser,- status_filter: str | None = Query(None, alias="status"),- severity: str | None = Query(None),- assignee: str | None = Query(None),- limit: int = Query(50, ge=1, le=500),- offset: int = Query(0, ge=0),-) -> list[CaseResponse]:- # Tenant scoping is non-optional. The aisoc_cases table has carried a- # tenant_id column since migration 012; until P2-W1 the API simply- # ignored it, exposing every tenant's caseload to every authenticated- # caller. Anchor the WHERE clause on tenant_id so the filter is always- # present even when the operator passes no other filters.- where_clauses = ["tenant_id = :tenant_id"]- params: dict[str, Any] = {- "limit": limit,- "offset": offset,- "tenant_id": user.tenant_id,- }- if status_filter:- where_clauses.append("status = :status")- params["status"] = status_filter- if severity:- where_clauses.append("severity = :severity")- params["severity"] = severity- if assignee:- where_clauses.append("assignee = :assignee")- params["assignee"] = assignee-- q = f"""- SELECT * FROM aisoc_cases- WHERE {" AND ".join(where_clauses)}- ORDER BY created_at DESC- LIMIT :limit OFFSET :offset- """- try:- rows = await db.execute(text(q).bindparams(**params))- return [_row_to_case(r) for r in rows.fetchall()]- except Exception as exc:- logger.exception("Database error in cases endpoint")- raise HTTPException(status_code=503, detail="Database error") from exc--[email protected]("", response_model=CaseResponse, status_code=status.HTTP_201_CREATED, summary="Create case")-async def create_case(- body: CreateCaseRequest, db: DBSession, user: Annotated[AuthUser, Depends(require_permission("cases:write"))]-) -> CaseResponse:- import json as _json-- case_id = uuid.uuid4()- now = datetime.now(UTC)- # Note: PostgreSQL ``::type`` casts inside text() confuse SQLAlchemy's- # bind-parameter parser; use ``CAST(:param AS TYPE)`` so the bind survives- # the round-trip (see also Batch 2 / hunts.py for the same gotcha).- q = text("""- INSERT INTO aisoc_cases (- id, tenant_id, title, description, severity, status, assignee,- mitre_techniques, alert_ids, compliance_frameworks, tags,- sla_due_at, opened_at, created_at, updated_at, created_by- ) VALUES (- :id, :tenant_id, :title, :description, :severity, 'new', :assignee,- CAST(:mitre AS JSONB), CAST(:alert_ids AS UUID[]),- CAST(:frameworks AS TEXT[]), CAST(:tags AS JSONB),- :sla, :now, :now, :now, :user- ) RETURNING *- """).bindparams(- id=case_id,- tenant_id=user.tenant_id,- title=body.title,- description=body.description,- severity=body.severity,- assignee=body.assignee,- mitre=_json.dumps(body.mitre_techniques),- alert_ids=list(map(str, body.alert_ids)) or [],- frameworks=body.compliance_frameworks or [],- tags=_json.dumps(body.tags),- sla=body.sla_due_at,- now=now,- user=getattr(user, "email", None) or "system",- )- try:- row = (await db.execute(q)).fetchone()- if row is None:- raise HTTPException(status_code=503, detail="Database error: INSERT returned no row")- await db.commit()- except Exception as exc:- await db.rollback()- logger.exception("Database error in cases endpoint")- raise HTTPException(status_code=503, detail="Database error") from exc-- response = _row_to_case(row)-- # Workstream 8 — fan out to operator-selected ITSM connectors. The- # case is already persisted (commit above) so a flaky Jira can't- # block the create. ``fanout_create_case`` swallows per-backend- # errors into the FanoutResult.status field; we only need to- # commit the ``case_external_refs`` writes it staged.- if body.push_to_connector_ids:- try:- results = await fanout_create_case(- db,- case_row=row,- tenant_id=user.tenant_id,- connector_ids=body.push_to_connector_ids,- pushed_by=getattr(user, "email", None),- )- await db.commit()- response.fanout_results = results- except Exception:- # Persistence of external refs failed but the AiSOC case- # itself is fine; surface the partial result and move on.- logger.exception("cases.create.fanout_persistence_failed case=%s", row.id)- await db.rollback()- response.fanout_results = []-- return response--[email protected]("/{case_id}", response_model=CaseResponse, summary="Get case")-async def get_case(case_id: str, db: DBSession, user: AuthUser) -> CaseResponse:- cid = await _resolve_case_id(case_id, db, user.tenant_id)- row = (- await db.execute(- text("SELECT * FROM aisoc_cases WHERE id = :id AND tenant_id = :tenant_id").bindparams(id=cid, tenant_id=user.tenant_id)- )- ).fetchone()- if not row:- raise HTTPException(status_code=404, detail="Case not found.")- return _row_to_case(row)--[email protected]("/{case_id}", response_model=CaseResponse, summary="Update case")-async def update_case(- case_id: str, body: UpdateCaseRequest, db: DBSession, user: Annotated[AuthUser, Depends(require_permission("cases:write"))]-) -> CaseResponse:- import json as _json-- cid = await _resolve_case_id(case_id, db, user.tenant_id)- existing = (- await db.execute(- text("SELECT status FROM aisoc_cases WHERE id = :id AND tenant_id = :tenant_id").bindparams(id=cid, tenant_id=user.tenant_id)- )- ).fetchone()- if not existing:- raise HTTPException(status_code=404, detail="Case not found.")-- if body.status and body.status != existing.status:- allowed = _TRANSITIONS.get(existing.status, set())- if body.status not in allowed:- raise HTTPException(- status_code=422,- detail=f"Invalid status transition: {existing.status} → {body.status}. Allowed: {sorted(allowed) or 'none'}",- )-- now = datetime.now(UTC)- sets = ["updated_at = :now"]- params: dict[str, Any] = {- "id": cid,- "now": now,- "tenant_id": user.tenant_id,- }-- if body.title is not None:- sets.append("title = :title")- params["title"] = body.title- if body.description is not None:- sets.append("description = :description")- params["description"] = body.description- if body.severity is not None:- sets.append("severity = :severity")- params["severity"] = body.severity- if body.assignee is not None:- sets.append("assignee = :assignee")- params["assignee"] = body.assignee- if body.sla_due_at is not None:- sets.append("sla_due_at = :sla")- params["sla"] = body.sla_due_at- if body.mitre_techniques is not None:- sets.append("mitre_techniques = CAST(:mitre AS JSONB)")- params["mitre"] = _json.dumps(body.mitre_techniques)- if body.compliance_frameworks is not None:- sets.append("compliance_frameworks = CAST(:frameworks AS TEXT[])")- params["frameworks"] = body.compliance_frameworks- if body.tags is not None:- sets.append("tags = CAST(:tags AS JSONB)")- params["tags"] = _json.dumps(body.tags)-- if body.status:- sets.append("status = :status")- params["status"] = body.status- ts_col = {"triaged": "triaged_at", "resolved": "resolved_at", "closed": "closed_at"}.get(body.status)- if ts_col:- sets.append(f"{ts_col} = :now")-- q = text(f"UPDATE aisoc_cases SET {', '.join(sets)} WHERE id = :id AND tenant_id = :tenant_id RETURNING *").bindparams(**params)- try:- row = (await db.execute(q)).fetchone()- if row is None:- raise HTTPException(status_code=404, detail="Case not found.")- await db.commit()- except Exception as exc:- await db.rollback()- logger.exception("Database error in cases endpoint")- raise HTTPException(status_code=503, detail="Database error") from exc-- response = _row_to_case(row)-- # Workstream 8 — if the operator changed status, project the- # transition onto every connector this case is already linked to.- # ``fanout_status_change`` no-ops when the case has no- # ``case_external_refs`` rows yet, so the call is cheap for- # AiSOC-only cases.- if body.status and body.status != existing.status:- try:- results = await fanout_status_change(- db,- case_row=row,- tenant_id=user.tenant_id,- old_status=existing.status,- new_status=body.status,- pushed_by=getattr(user, "email", None),- )- if results:- await db.commit()- response.fanout_results = results- except Exception:- logger.exception("cases.update.fanout_persistence_failed case=%s", row.id)- await db.rollback()- response.fanout_results = []-- # WS-D2 — when a case enters its terminal lifecycle stages, drop a- # system note pointing to the auto-summary artifact. The summary- # itself is generated on-demand by GET /cases/{id}/summary, so this- # is a cheap, idempotent breadcrumb rather than a heavy precompute.- if body.status in {"resolved", "closed"}:- try:- await _emit_summary_breadcrumb(- db,- case_id=row.id,- tenant_id=user.tenant_id,- status=body.status,- )- except Exception: # pragma: no cover — defensive: never block status change.- logger.exception("cases.update.summary_breadcrumb_failed case=%s", row.id)- await db.rollback()-- return response--[email protected]("/{case_id}/alerts", response_model=CaseResponse, summary="Link alerts to a case")-async def add_alerts(- case_id: str, body: AddAlertsRequest, db: DBSession, user: Annotated[AuthUser, Depends(require_permission("cases:write"))]-) -> CaseResponse:- cid = await _resolve_case_id(case_id, db, user.tenant_id)- ids_str = [str(a) for a in body.alert_ids]- q = text("""- UPDATE aisoc_cases- SET alert_ids = array(SELECT DISTINCT unnest(alert_ids || CAST(:new_ids AS UUID[]))),- updated_at = now()- WHERE id = :id AND tenant_id = :tenant_id- RETURNING *- """).bindparams(id=cid, new_ids=ids_str, tenant_id=user.tenant_id)- try:- row = (await db.execute(q)).fetchone()- if not row:- raise HTTPException(status_code=404, detail="Case not found.")- await db.commit()- return _row_to_case(row)- except HTTPException:- raise- except Exception as exc:- await db.rollback()- logger.exception("Database error in cases endpoint")- raise HTTPException(status_code=503, detail="Database error") from exc--[email protected]("/{case_id}/observables", response_model=CaseResponse, summary="Update observable graph")-async def update_observables(- case_id: str, body: UpdateObservablesRequest, db: DBSession, user: Annotated[AuthUser, Depends(require_permission("cases:write"))]-) -> CaseResponse:- import json as _json-- cid = await _resolve_case_id(case_id, db, user.tenant_id)- existing = (- await db.execute(- text("SELECT observable_graph FROM aisoc_cases WHERE id = :id AND tenant_id = :tenant_id").bindparams(- id=cid, tenant_id=user.tenant_id- )- )- ).fetchone()- if not existing:- raise HTTPException(status_code=404, detail="Case not found.")-- graph: dict[str, Any] = dict(existing.observable_graph or {})- nodes = graph.get("nodes", [])- edges = graph.get("edges", [])-- existing_ids = {n["id"] for n in nodes}- for n in body.nodes:- if n.id not in existing_ids:- nodes.append(n.model_dump())- else:- nodes = [n.model_dump() if x["id"] == n.id else x for x in nodes]-- edge_keys = {(e["source"], e["target"]) for e in edges}- for e in body.edges:- if (e.source, e.target) not in edge_keys:- edges.append(e.model_dump())-- graph = {"nodes": nodes, "edges": edges}- q = text(- "UPDATE aisoc_cases SET observable_graph = CAST(:g AS JSONB), updated_at = now() "- "WHERE id = :id AND tenant_id = :tenant_id RETURNING *"- ).bindparams(id=cid, g=_json.dumps(graph), tenant_id=user.tenant_id)- try:- row = (await db.execute(q)).fetchone()- await db.commit()- return _row_to_case(row)- except Exception as exc:- await db.rollback()- logger.exception("Database error in cases endpoint")- raise HTTPException(status_code=503, detail="Database error") from exc--[email protected]("/{case_id}/comments", response_model=CommentResponse, status_code=201, summary="Add comment")-async def add_comment(- case_id: str, body: AddCommentRequest, db: DBSession, user: Annotated[AuthUser, Depends(require_permission("cases:write"))]-) -> CommentResponse:- cid = await _resolve_case_id(case_id, db, user.tenant_id)- exists = (- await db.execute(- text("SELECT 1 FROM aisoc_cases WHERE id = :id AND tenant_id = :tenant_id").bindparams(id=cid, tenant_id=user.tenant_id)- )- ).fetchone()- if not exists:- raise HTTPException(status_code=404, detail="Case not found.")-- comment_id = uuid.uuid4()- now = datetime.now(UTC)- # Stamp the row with tenant_id so the comment table is independently- # tenant-scoped — see migration 044.- q = text("""- INSERT INTO aisoc_case_comments- (id, case_id, tenant_id, author, body, is_system, created_at)- VALUES- (:id, :case_id, :tenant_id, :author, :body, :sys, :now)- RETURNING *- """).bindparams(- id=comment_id,- case_id=cid,- tenant_id=user.tenant_id,- author=getattr(user, "email", None) or "system",- body=body.body,- sys=body.is_system,- now=now,- )- try:- row = (await db.execute(q)).fetchone()- if row is None:- raise HTTPException(status_code=503, detail="Database error: INSERT returned no row")- await db.commit()- return CommentResponse(- id=row.id,- case_id=row.case_id,- author=row.author,- body=row.body,- is_system=row.is_system,- created_at=row.created_at,- )- except Exception as exc:- await db.rollback()- logger.exception("Database error in cases endpoint")- raise HTTPException(status_code=503, detail="Database error") from exc--[email protected]("/{case_id}/comments", response_model=list[CommentResponse], summary="List case comments")-async def list_comments(case_id: str, db: DBSession, user: AuthUser) -> list[CommentResponse]:- cid = await _resolve_case_id(case_id, db, user.tenant_id)- rows = (- await db.execute(- text("SELECT * FROM aisoc_case_comments WHERE case_id = :id AND tenant_id = :tenant_id ORDER BY created_at").bindparams(- id=cid, tenant_id=user.tenant_id- )- )- ).fetchall()- return [- CommentResponse(id=r.id, case_id=r.case_id, author=r.author, body=r.body, is_system=r.is_system, created_at=r.created_at)- for r in rows- ]---# Alias `/notes` → `/comments` so the web console (which calls `/notes` for the-# "Add Note" affordance) hits the same backing store. Keeping both routes-# avoids a behavior change for any existing integration that already speaks-# `/comments`.[email protected]("/{case_id}/notes", response_model=list[CommentResponse], summary="List case notes (alias of /comments)")-async def list_notes(case_id: str, db: DBSession, user: AuthUser) -> list[CommentResponse]:- return await list_comments(case_id, db, user)--[email protected]("/{case_id}/notes", response_model=CommentResponse, status_code=201, summary="Add case note (alias of /comments)")-async def add_note(- case_id: str, body: AddCommentRequest, db: DBSession, user: Annotated[AuthUser, Depends(require_permission("cases:write"))]-) -> CommentResponse:- return await add_comment(case_id, body, db, user)--[email protected]("/{case_id}/evidence", response_model=EvidenceReport, summary="Export evidence chain report")-async def evidence_report(case_id: str, db: DBSession, user: AuthUser) -> EvidenceReport:- cid = await _resolve_case_id(case_id, db, user.tenant_id)- row = (- await db.execute(- text("SELECT * FROM aisoc_cases WHERE id = :id AND tenant_id = :tenant_id").bindparams(id=cid, tenant_id=user.tenant_id)- )- ).fetchone()- if not row:- raise HTTPException(status_code=404, detail="Case not found.")- return EvidenceReport(- case_id=row.id,- title=row.title,- severity=row.severity,- status=row.status,- alert_count=len(row.alert_ids or []),- mitre_techniques=_coerce_mitre(row.mitre_techniques),- compliance_frameworks=list(row.compliance_frameworks or []),- evidence_chain=list(row.evidence_chain or []),- generated_at=datetime.now(UTC),- )---# ─────────────────────────────────────────────────────────────────────────────-# Timeline-#-# A flattened activity stream that the case workspace renders on the right.-# We synthesize the timeline from the case's own state transitions, linked-# alerts, comments, and tasks so the front end gets one unified view without-# orchestrating four separate calls.-# ─────────────────────────────────────────────────────────────────────────────--[email protected]("/{case_id}/timeline", response_model=TimelineResponse, summary="Case activity timeline")-async def case_timeline(case_id: str, db: DBSession, user: AuthUser) -> TimelineResponse:- cid = await _resolve_case_id(case_id, db, user.tenant_id)- case_row = (- await db.execute(- text("SELECT * FROM aisoc_cases WHERE id = :id AND tenant_id = :tenant_id").bindparams(id=cid, tenant_id=user.tenant_id)- )- ).fetchone()- if not case_row:- raise HTTPException(status_code=404, detail="Case not found.")-- events: list[TimelineEvent] = []-- # Case creation event- events.append(- TimelineEvent(- id=f"case-{case_row.id}-created",- type="created",- timestamp=case_row.created_at,- title="Case opened",- description=case_row.title,- actor=case_row.assignee,- )- )-- # Status / severity reflection (we only surface the latest snapshot here;- # full audit history is available via the audit log endpoints).- if case_row.updated_at and case_row.updated_at != case_row.created_at:- events.append(- TimelineEvent(- id=f"case-{case_row.id}-status",- type="status_change",- timestamp=case_row.updated_at,- title=f"Status: {case_row.status}",- description=f"Severity {case_row.severity}",- actor=case_row.assignee,- )- )-- # Comments- comment_rows = (- await db.execute(- text(- "SELECT id, author, body, is_system, created_at "- "FROM aisoc_case_comments "- "WHERE case_id = :id AND tenant_id = :tenant_id "- "ORDER BY created_at"- ).bindparams(id=cid, tenant_id=user.tenant_id)- )- ).fetchall()- for c in comment_rows:- events.append(- TimelineEvent(- id=f"comment-{c.id}",- type="comment",- timestamp=c.created_at,- title="System note" if c.is_system else "Comment",- description=(c.body or "")[:280],- actor=c.author,- )- )-- # Linked alerts (best-effort — we tolerate a missing alerts table)- for alert_id in list(case_row.alert_ids or [])[:25]:- try:- a = (- # Tenant-scoped as defence in depth. Reaching this loop already- # required a tenant-scoped case, but the predicate costs- # nothing and a poisoned ``alert_ids`` array would otherwise- # hydrate another tenant's alert title into this timeline.- await db.execute(- text("SELECT id, title, severity, created_at FROM aisoc_alerts WHERE id = :id AND tenant_id = :tenant_id").bindparams(- id=alert_id,- tenant_id=str(user.tenant_id),- )- )- ).fetchone()- if a:- events.append(- TimelineEvent(- id=f"alert-{a.id}",- type="alert_linked",- timestamp=a.created_at or case_row.created_at,- title=f"Alert linked: {a.title}",- description=f"Severity {a.severity}",- actor=None,- )- )- except Exception: # noqa: BLE001 — alerts table may not exist in some demos- continue-- # Tasks- try:- task_rows = (- await db.execute(- text(- "SELECT id, title, status, assignee, created_at "- "FROM aisoc_case_tasks "- "WHERE case_id = :id AND tenant_id = :tenant_id "- "ORDER BY created_at"- ).bindparams(id=cid, tenant_id=user.tenant_id)- )- ).fetchall()- for t in task_rows:- events.append(- TimelineEvent(- id=f"task-{t.id}",- type="task",- timestamp=t.created_at,- title=f"Task ({t.status}): {t.title}",- description=None,- actor=t.assignee,- )- )- except Exception: # noqa: BLE001 — table created by migration 027; tolerate absence- logger.exception("aisoc_case_tasks not available; skipping task events")-- events.sort(key=lambda e: e.timestamp)- return TimelineResponse(events=events)---# ─────────────────────────────────────────────────────────────────────────────-# Tasks-# ─────────────────────────────────────────────────────────────────────────────--[email protected]("/{case_id}/tasks", response_model=list[TaskResponse], summary="List case tasks")-async def list_tasks(case_id: str, db: DBSession, user: AuthUser) -> list[TaskResponse]:- cid = await _resolve_case_id(case_id, db, user.tenant_id)- exists = (- await db.execute(- text("SELECT 1 FROM aisoc_cases WHERE id = :id AND tenant_id = :tenant_id").bindparams(id=cid, tenant_id=user.tenant_id)- )- ).fetchone()- if not exists:- raise HTTPException(status_code=404, detail="Case not found.")- rows = (- await db.execute(- text(- "SELECT id, title, status, assignee, due_at, created_at "- "FROM aisoc_case_tasks "- "WHERE case_id = :id AND tenant_id = :tenant_id "- "ORDER BY created_at"- ).bindparams(id=cid, tenant_id=user.tenant_id)- )- ).fetchall()- return [_row_to_task(r) for r in rows]--[email protected]("/{case_id}/tasks", response_model=TaskResponse, status_code=201, summary="Create task")-async def create_task(- case_id: str,- body: CreateTaskRequest,- db: DBSession,- user: Annotated[AuthUser, Depends(require_permission("cases:write"))],-) -> TaskResponse:- cid = await _resolve_case_id(case_id, db, user.tenant_id)- exists = (- await db.execute(- text("SELECT 1 FROM aisoc_cases WHERE id = :id AND tenant_id = :tenant_id").bindparams(id=cid, tenant_id=user.tenant_id)- )- ).fetchone()- if not exists:- raise HTTPException(status_code=404, detail="Case not found.")-- task_id = uuid.uuid4()- now = datetime.now(UTC)- try:- row = (- await db.execute(- text(- """- INSERT INTO aisoc_case_tasks- (id, case_id, tenant_id, title, status, assignee, due_at, created_at, updated_at, created_by)- VALUES- (:id, :case_id, :tenant_id, :title, :status, :assignee, :due_at, :now, :now, :created_by)- RETURNING id, title, status, assignee, due_at, created_at- """- ).bindparams(- id=task_id,- case_id=cid,- tenant_id=user.tenant_id,- title=body.title,- status=body.status,- assignee=body.assignee,- due_at=body.due_at,- now=now,- created_by=getattr(user, "email", None) or "system",- )- )- ).fetchone()- await db.commit()- return _row_to_task(row)- except Exception as exc:- await db.rollback()- logger.exception("Database error in cases endpoint")- raise HTTPException(status_code=503, detail="Database error") from exc--[email protected]("/{case_id}/tasks/{task_id}", response_model=TaskResponse, summary="Update task")-async def update_task(- case_id: str,- task_id: uuid.UUID,- body: UpdateTaskRequest,- db: DBSession,- user: Annotated[AuthUser, Depends(require_permission("cases:write"))],-) -> TaskResponse:- cid = await _resolve_case_id(case_id, db, user.tenant_id)- sets: list[str] = []- params: dict[str, Any] = {- "id": task_id,- "case_id": cid,- "tenant_id": user.tenant_id,- "now": datetime.now(UTC),- }- if body.title is not None:- sets.append("title = :title")- params["title"] = body.title- if body.status is not None:- sets.append("status = :status")- params["status"] = body.status- if body.assignee is not None:- sets.append("assignee = :assignee")- params["assignee"] = body.assignee- if body.due_at is not None:- sets.append("due_at = :due_at")- params["due_at"] = body.due_at- if not sets:- raise HTTPException(status_code=400, detail="No fields to update.")- sets.append("updated_at = :now")-- try:- row = (- await db.execute(- text(- f"""- UPDATE aisoc_case_tasks- SET {", ".join(sets)}- WHERE id = :id AND case_id = :case_id AND tenant_id = :tenant_id- RETURNING id, title, status, assignee, due_at, created_at- """- ).bindparams(**params)- )- ).fetchone()- if not row:- raise HTTPException(status_code=404, detail="Task not found.")- await db.commit()- return _row_to_task(row)- except HTTPException:- raise- except Exception as exc:- await db.rollback()- logger.exception("Database error in cases endpoint")- raise HTTPException(status_code=503, detail="Database error") from exc---# ─────────────────────────────────────────────────────────────────────────────-# Investigation proxy-#-# The investigation orchestrator lives in the agents service. The web console-# only knows about /api/v1/cases/{id}/..., so we proxy through and keep the-# external API surface stable.-# ─────────────────────────────────────────────────────────────────────────────---async def _agents_proxy(method: str, path: str, **kwargs: Any) -> httpx.Response:- safe_path = _validate_agents_path(path)- url = f"{_AGENTS_URL}{safe_path}"- timeout = kwargs.pop("timeout", 30.0)- try:- async with httpx.AsyncClient(timeout=timeout) as client:- return await client.request(method, url, **kwargs)- except httpx.HTTPError as exc:- logger.exception(- "agents_proxy.request_failed",- extra={- "method": safe_log_value(method),- "path": safe_log_value(safe_path),- },- )- raise HTTPException(- status_code=503,- detail=f"Agents service unavailable: {exc}",- ) from exc--[email protected]("/{case_id}/investigate", summary="Launch investigation for case")-async def case_investigate(- case_id: str,- body: InvestigateRequest,- db: DBSession,- user: Annotated[AuthUser, Depends(require_permission("cases:write"))],-) -> dict[str, Any]:- cid = await _resolve_case_id(case_id, db, user.tenant_id)- exists = (- await db.execute(- text("SELECT 1 FROM aisoc_cases WHERE id = :id AND tenant_id = :tenant_id").bindparams(id=cid, tenant_id=user.tenant_id)- )- ).fetchone()- if not exists:- raise HTTPException(status_code=404, detail="Case not found.")-- # Forward the authenticated tenant so the agents service attributes the run- # to the real tenant instead of falling back to the "default" placeholder,- # which the demo seed no longer maps to any tenant (issue #601). The ledger- # is scoped by tenant_id, so without this the whole run is never persisted.- resp = await _agents_proxy(- "POST",- f"/api/v1/cases/{cid}/investigate",- json={"alert_summary": body.alert_summary or "", "tenant_id": str(user.tenant_id)},- )- if resp.status_code >= 400:- raise HTTPException(status_code=resp.status_code, detail=resp.text)- return resp.json()--[email protected]("/{case_id}/investigations", summary="List investigation runs for a case")-async def list_case_investigations(- case_id: str,- db: DBSession,- user: AuthUser,-) -> dict[str, Any]:- """List all investigation runs for a case.-- The agents service is the source of truth. When it's unreachable we return- an empty list rather than 503 so the case detail page still renders.- """- cid = await _resolve_case_id(case_id, db, user.tenant_id)- try:- resp = await _agents_proxy("GET", f"/api/v1/cases/{cid}/investigations")- if resp.status_code == 404:- return {"runs": []}- if resp.status_code >= 400:- return {"runs": []}- return resp.json()- except HTTPException:- # Agents service unavailable — render a soft-empty list instead of 503.- return {"runs": []}--[email protected]("/{case_id}/investigations/{run_id}", summary="Get investigation run")-async def case_investigation_run(- case_id: str,- run_id: str,- db: DBSession,- user: AuthUser,-) -> dict[str, Any]:- """One investigation run belonging to this case, in the caller's tenant."""- # Scoping kept out of the docstring because FastAPI publishes that verbatim- # in docs/openapi.yaml, and the rationale is for the next maintainer rather- # than for API consumers.- #- # This handler took no database session and never read `user.tenant_id`:- # `case_id` was declared and never used, and `run_id` alone was proxied to- # the agents service, so any authenticated user could read any run by id- # across tenants (GHSA-x2gf-3p79-wvgm). The sibling list route two- # functions up already resolved the case against the caller's tenant.- #- # Both halves below are needed. Resolving the case proves the caller may- # see *this case*; checking the run belongs to it proves the id in the path- # is not somebody else's run smuggled under a case the caller does own.- cid = await _resolve_case_id(case_id, db, user.tenant_id)- # URL-encode the user-supplied run_id so it cannot inject `/`, `?`, `#`,- # CR/LF, or other URL syntax into the proxied path.- safe_run_id = quote(run_id, safe="")- resp = await _agents_proxy("GET", f"/api/v1/investigations/{safe_run_id}")- if resp.status_code >= 400:- raise HTTPException(status_code=resp.status_code, detail=resp.text)- run = resp.json()-- # A run that names a different case is not this case's run. Compared as- # strings because the agents service echoes whatever case id it was given.- run_case_id = run.get("case_id")- if run_case_id is not None and str(run_case_id) != str(cid):- raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Investigation run not found for this case")- return run--- "/{case_id}/investigations/{run_id}/report.md",- response_class=PlainTextResponse,- summary="Markdown incident report for one investigation run",-)-async def case_investigation_report_md(- case_id: str,- run_id: str,- db: DBSession,- user: AuthUser,-) -> PlainTextResponse:- """The Markdown report the case workspace renders."""- # `CaseWorkspace.tsx` fetched this from two places and the route did not- # exist, so the report pane was permanently empty. The agents service has- # served `/api/v1/investigations/{run_id}/report.md` all along; nothing- # proxied it.- #- # Scoped in the same two steps as the sibling route above, and for the- # same reason: that one shipped without them and any authenticated user- # could read any run by id across tenants (GHSA-x2gf-3p79-wvgm).- # Resolving the case proves the caller may see this case; checking the- # run belongs to it proves the id in the path is not somebody else's run- # smuggled under a case the caller does own.- cid = await _resolve_case_id(case_id, db, user.tenant_id)- safe_run_id = quote(run_id, safe="")-- meta = await _agents_proxy("GET", f"/api/v1/investigations/{safe_run_id}")- if meta.status_code >= 400:- raise HTTPException(status_code=meta.status_code, detail=meta.text)- run_case_id = meta.json().get("case_id")- if run_case_id is not None and str(run_case_id) != str(cid):- raise HTTPException(- status_code=status.HTTP_404_NOT_FOUND,- detail="Investigation run not found for this case",- )-- resp = await _agents_proxy("GET", f"/api/v1/investigations/{safe_run_id}/report.md")- if resp.status_code >= 400:- raise HTTPException(status_code=resp.status_code, detail=resp.text)- return PlainTextResponse(content=resp.text, media_type="text/markdown; charset=utf-8")---# Filename sanitiser for Content-Disposition: keep only safe ASCII so the-# header cannot be split with CR/LF and the value renders consistently.-_SAFE_FILENAME_RE = re.compile(r"[^A-Za-z0-9._-]")---def _safe_filename_segment(value: str) -> str:- """Reduce an identifier to a Content-Disposition-safe filename segment."""- return _SAFE_FILENAME_RE.sub("_", value)[:64] or "unknown"---# ────────────────────────────────────────────────────────────────────────────-# WS-D2 — auto-summary artifact at investigation close.-# ────────────────────────────────────────────────────────────────────────────--# We keep the breadcrumb body in one place so the system note stays uniform-# across deployments — easier to grep, easier to redact in CI exports.-_SUMMARY_BREADCRUMB_BODY = (- "Case auto-summary generated. Download from "- "/api/v1/cases/{case_id}/summary?format=html "- "(or ?format=json for the structured payload). "- "For the blameless retrospective, fetch "- "/api/v1/cases/{case_id}/postmortem?format=html. "- "Print either page (Ctrl/Cmd-P → Save as PDF) for the runbook archive."-)---async def _emit_summary_breadcrumb(- db: Any,- *,- case_id: uuid.UUID,- tenant_id: uuid.UUID | str,- status: str,-) -> None:- """Drop a system comment pointing to the on-demand summary endpoint.-- Idempotent on repeated status changes — we re-emit only when transitioning- *into* a terminal state, never on noop updates. The body intentionally- avoids embedding tenant-specific URLs so the artifact link still works- behind reverse proxies that rewrite the host.- """- body = _SUMMARY_BREADCRUMB_BODY.format(case_id=case_id) + f" (status={status})"- await db.execute(- text(- """- INSERT INTO aisoc_case_comments- (id, case_id, tenant_id, author, body, is_system, created_at)- VALUES (:id, :case_id, :tenant_id, :author, :body, true, :now)- """- ).bindparams(- id=uuid.uuid4(),- case_id=case_id,- tenant_id=tenant_id,- author="aisoc",- body=body,- now=datetime.now(UTC),- )- )- await db.commit()--- "/{case_id}/summary",- summary="Download the per-case auto-summary (JSON or HTML)",- response_model=None,-)-async def case_auto_summary(- case_id: str,- db: DBSession,- user: AuthUser,- format: Literal["json", "html"] = Query(- "json",- description=(- "Response format. ``json`` returns the structured CaseAutoSummary; "- "``html`` returns a print-ready report (use the browser's "- "Save-as-PDF affordance for archival)."- ),- ),-) -> Any:- cid = await _resolve_case_id(case_id, db, user.tenant_id)- summary = await build_case_summary(db, cid, user.tenant_id)- if summary is None:- raise HTTPException(status_code=404, detail="Case not found.")-- if format == "html":- rendered = render_case_summary_html(summary)- case_label = summary.case.case_number or str(summary.case.case_id)[:8]- filename = _safe_filename_segment(f"case-{case_label}-summary") + ".html"- return HTMLResponse(- content=rendered,- headers={"Content-Disposition": f'inline; filename="{filename}"'},- )-- return summary--- "/{case_id}/postmortem",- summary="Download the per-case blameless post-mortem (JSON or HTML)",- response_model=None,-)-async def case_auto_postmortem(- case_id: str,- db: DBSession,- user: AuthUser,- format: Literal["json", "html"] = Query(- "json",- description=(- "Response format. ``json`` returns the structured CasePostmortem; "- "``html`` returns a print-ready blameless retrospective (use the "- "browser's Save-as-PDF affordance for the runbook archive)."- ),- ),-) -> Any:- """Generate a deterministic blameless post-mortem for one case.-- Distinct from ``/summary``: the summary is a snapshot of *what the case- looks like right now*; the post-mortem is a retrospective oriented around- *what happened, when did we find out, what did we do, and what should we- change*. Both are deterministic — same case state in, same artefact out.- """- cid = await _resolve_case_id(case_id, db, user.tenant_id)- postmortem = await build_case_postmortem(db, cid, user.tenant_id)- if postmortem is None:- raise HTTPException(status_code=404, detail="Case not found.")-- if format == "html":- rendered = render_case_postmortem_html(postmortem)- case_label = postmortem.case.case_number or str(postmortem.case.case_id)[:8]- filename = _safe_filename_segment(f"case-{case_label}-postmortem") + ".html"- return HTMLResponse(- content=rendered,- headers={"Content-Disposition": f'inline; filename="{filename}"'},- )-- return postmortem--- "/{case_id}/investigations/{run_id}/report.pdf",- summary="Download investigation PDF report",-)-async def case_investigation_pdf(- case_id: str,- run_id: str,- user: AuthUser,-) -> Response:- safe_run_id = quote(run_id, safe="")- resp = await _agents_proxy("GET", f"/api/v1/investigations/{safe_run_id}/report.pdf")- if resp.status_code >= 400:- raise HTTPException(status_code=resp.status_code, detail=resp.text)- safe_case_id = _safe_filename_segment(case_id)- safe_run_id_filename = _safe_filename_segment(run_id)- return Response(- content=resp.content,- media_type=resp.headers.get("content-type", "application/pdf"),- headers={- "Content-Disposition": resp.headers.get(- "content-disposition",- f'attachment; filename="case-{safe_case_id}-{safe_run_id_filename}.pdf"',- ),- },- )--[email protected]("/{case_id}/related", summary="List related cases (by alert/observable overlap)")-async def list_related_cases(- case_id: str,- db: DBSession,- user: AuthUser,-) -> dict[str, Any]:- """Return other cases that share alerts or observables with this case.-- This is intentionally a lightweight heuristic — we look for cases that- overlap on `alert_ids` or share an observable IP/host. It's the smallest- thing the case detail page needs to stop 404'ing.- """- cid = await _resolve_case_id(case_id, db, user.tenant_id)- row = (- await db.execute(- text("SELECT alert_ids, observable_graph FROM aisoc_cases WHERE id = :id AND tenant_id = :tenant_id").bindparams(- id=cid, tenant_id=user.tenant_id- )- )- ).fetchone()- if not row:- raise HTTPException(status_code=404, detail="Case not found.")-- alert_ids = list(row.alert_ids or [])- related: list[dict[str, Any]] = []-- if alert_ids:- rows = (- await db.execute(- text(- """- SELECT id, case_number, title, severity, status, created_at- FROM aisoc_cases- WHERE id <> :id- AND tenant_id = :tenant_id- AND alert_ids && CAST(:alerts AS UUID[])- ORDER BY created_at DESC- LIMIT 20- """- ).bindparams(- id=cid,- tenant_id=user.tenant_id,- alerts=[str(a) for a in alert_ids],- )- )- ).fetchall()- for r in rows:- related.append(- {- "id": str(r.id),- "case_number": r.case_number,- "title": r.title,- "severity": r.severity or "medium",- "status": r.status or "new",- "created_at": r.created_at.isoformat() if r.created_at else None,- "reason": "shared_alerts",- }- )-- return {"related": related}diff --git a/services/api/app/api/v1/endpoints/insights.py b/services/api/app/api/v1/endpoints/insights.pyindex 410dea158..573f28b4d 100644--- a/services/api/app/api/v1/endpoints/insights.py+++ b/services/api/app/api/v1/endpoints/insights.py@@ -48,11 +48,6 @@ from __future__ import annotationsimport loggingimport uuid--def _tid(t):- """Bind a native UUID for raw SQL on uuid-typed columns (asyncpg)."""- return t if isinstance(t, uuid.UUID) else uuid.UUID(str(t))-from datetime import UTC, datetime, timedeltafrom typing import Literal@@ -68,6 +63,11 @@ from app.services import case_statuslogger = logging.getLogger(__name__)++def _tid(t):+ """Bind a native UUID for raw SQL on uuid-typed columns (asyncpg)."""+ return t if isinstance(t, uuid.UUID) else uuid.UUID(str(t))+router = APIRouter(prefix="/insights", tags=["insights"])diff --git a/services/api/app/api/v1/endpoints/metrics.py b/services/api/app/api/v1/endpoints/metrics.pyindex c4f93bb58..44e48de36 100644--- a/services/api/app/api/v1/endpoints/metrics.py+++ b/services/api/app/api/v1/endpoints/metrics.py@@ -146,14 +146,18 @@ class MitreCoverage(BaseModel):class FunnelDeltas(BaseModel):- """Period-over-period percentage deltas for the funnel KPI bar."""+ """Period-over-period percentage deltas for the funnel KPI bar.- events_of_interest: float- correlation_instances: float- alerts_generated: float- signal_to_noise: float- mttd_seconds: float- analyst_queue_depth: float+ ``None`` means the previous window was empty — no baseline exists, and+ the UI renders that as "no baseline" instead of a misleading 0%.+ """++ events_of_interest: float | None = None+ correlation_instances: float | None = None+ alerts_generated: float | None = None+ signal_to_noise: float | None = None+ mttd_seconds: float | None = None+ analyst_queue_depth: float | None = Noneclass FunnelMetrics(BaseModel):@@ -333,6 +337,9 @@ async def get_dashboard_metrics(}window_delta, trend_trunc = _period_map[period]window_start = now - window_delta+ # One window predicate for every windowed count below, so no tile can+ # silently answer for a different period than the selector says.+ _in_window = Alert.created_at >= window_start# ── Alert counts ──────────────────────────────────────────────────────────# Unresolved only. See `_count_alerts_by_status`: these tiles are labelled@@ -417,7 +424,9 @@ async def get_dashboard_metrics(# Count alerts per connector_typesource_counts_rows = (await db.execute(- select(Alert.connector_type, func.count().label("cnt")).where(and_(Alert.tenant_id == tenant_id, _in_window)).group_by(Alert.connector_type)+ select(Alert.connector_type, func.count().label("cnt"))+ .where(and_(Alert.tenant_id == tenant_id, _in_window))+ .group_by(Alert.connector_type))).all()source_count_map: dict[str, int] = {row.connector_type: row.cnt for row in source_counts_rows if row.connector_type}@@ -444,7 +453,9 @@ async def get_dashboard_metrics(# jsonb_array_elements_text + GROUP BY in the same SELECT 500s on some# Postgres builds.tactic_rows = (- await db.execute(select(Alert.mitre_tactics).where(and_(Alert.tenant_id == tenant_id, Alert.mitre_tactics.isnot(None), _in_window)))+ await db.execute(+ select(Alert.mitre_tactics).where(and_(Alert.tenant_id == tenant_id, Alert.mitre_tactics.isnot(None), _in_window))+ )).all()tactic_counts: dict[str, int] = {}@@ -461,12 +472,11 @@ async def get_dashboard_metrics(# ── trend over the selected period ────────────────────────────────────────# Bucket by hour inside a day-scale window; `1h` still buckets by hour.- trend_start = now - window- bucket_fn = "minute" if period == "1h" else "hour"+ trend_start = window_starttrend_rows = (await db.execute(select(- func.date_trunc(bucket_fn, Alert.created_at).label("bucket"),+ func.date_trunc(trend_trunc, Alert.created_at).label("bucket"),Alert.severity,func.count().label("cnt"),)@@ -501,7 +511,6 @@ async def get_dashboard_metrics(topMitre=top_mitre,alertsTrend=alerts_trend,threatsBySource=threats_by_source,- period=period,)@@ -863,15 +872,17 @@ async def get_alert_trend(# ──────────────────────────── v1.5 funnel endpoint ────────────────────────────-def _pct_delta(current: float, previous: float) -> float:+def _pct_delta(current: float, previous: float) -> float | None:"""Compute period-over-period percentage change.- Returns 0.0 when the previous value is zero (avoids `inf`). Otherwise:+ Returns ``None`` when the previous window was empty: there is no+ baseline, and rendering `0.0` would claim "flat" where the honest+ answer is "unknown". The UI renders None as "no baseline". Otherwise:((current - previous) / previous) * 100, rounded to 2 decimals."""if previous == 0:- return 0.0+ return Nonereturn round(((current - previous) / previous) * 100.0, 2)@@ -1220,7 +1231,7 @@ async def get_funnel_metrics(``deltas`` compares the current window to the immediately preceding windowof the same length (e.g. for ``period=24h`` we compare to the 24h ending24h ago). Values are percentage changes rounded to two decimals; a delta- of ``0.0`` means "no meaningful baseline" (the previous window was empty).+ of ``None`` means "no baseline" (the previous window was empty)."""delta = _FUNNEL_PERIOD_MAP[period]now = datetime.now(UTC)