Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
# AiSOC — NVIDIA GPU overlay for the bundled Ollama
#
# Usage:
# make up-gpu
#
# or by hand:
# docker compose -f docker-compose.yml -f infra/compose/docker-compose.gpu.yml up -d
#
# What this changes
# -----------------
# One thing: it reserves an NVIDIA device for the `ollama` service. Nothing
# else moves. The model, the gateway, the aliases and every other service are
# whatever the base file says.
#
# Why it is an overlay and not a line in the base file
# ----------------------------------------------------
# A `deploy.resources.reservations.devices` block naming the nvidia driver is
# not advisory. On a host without an NVIDIA GPU *and* the container toolkit,
# the daemon refuses to start the service at all:
#
# Error response from daemon: could not select device driver "nvidia" with
# capabilities: [[gpu]]
#
# `make up` is the documented first-run path and CORE ships
# `llama3.2:3b-instruct-q4_K_M` to run CPU-only in 8 GB, so putting this in the
# base file takes out first run for every Mac, every CPU-only Linux box and
# every CI runner. It has to be opt-in, and `make up-gpu` runs
# `scripts/check_gpu_runtime.py` first so the failure names a cause and a next
# command rather than handing over the daemon's error.
#
# Apple Silicon
# -------------
# This overlay does nothing for you. Docker Desktop cannot pass the Metal GPU
# into a Linux container, so a bundled Ollama is CPU-only on a Mac no matter
# what is reserved here. Run Ollama natively instead -- it uses Metal -- and
# point the stack at it with `make up-host-llm`.
#
# Sizing
# ------
# The pinned model needs roughly 3 GB of VRAM. `count: 1` takes the first
# visible device; set `AISOC_GPU_COUNT=all` to expose every device, or pin
# specific ones with `device_ids` instead of `count` if you share the host.
# Model
# -----
# This overlay moves the default off the 3B the CPU profile ships. That is a
# measurement, not a preference: graded on the 72-item balanced verdict
# corpus, `llama3.1:8b-instruct-q4_K_M` scores 0.357 balanced accuracy
# against the 3B's 0.285 and a 0.250 constant-answer floor, and is the only
# model measured whose calibration error falls below 0.40. The table and the
# four rows it beat are published on the benchmark page under "Verdict
# quality"; `scripts/run_verdict_eval.py` reproduces it.
#
# It needs roughly 6 GB of VRAM rather than the 3B's 3 GB. Override with
# AISOC_LLM_LOCAL_MODEL plus the two routing variables if your card is
# smaller -- all three have to move together, or the gateway routes to a
# model nothing pulled.
services:
ollama:
environment:
# Ollama decides layer placement itself and logs what it did. Left
# unset rather than forced, because a hardcoded layer count is wrong
# on every card except the one it was measured on.
OLLAMA_NUM_PARALLEL: "${OLLAMA_NUM_PARALLEL:-1}"
# The reservation belongs to the service that runs the model. Ollama is
# the only one here that does any inference: litellm proxies and
# ollama-pull downloads, so a device reserved on either of those is a
# GPU held by something that cannot use it while generation runs on the
# CPU -- the overlay reporting success having done nothing.
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: ${AISOC_GPU_COUNT:-1}
capabilities: [gpu]
ollama-pull:
entrypoint: ["ollama", "pull", "${AISOC_LLM_LOCAL_MODEL:-llama3.1:8b-instruct-q4_K_M}"]
litellm:
environment:
AISOC_LLM_MODEL_FAST: ${AISOC_LLM_MODEL_FAST:-ollama_chat/llama3.1:8b-instruct-q4_K_M}
AISOC_LLM_MODEL_DEEP: ${AISOC_LLM_MODEL_DEEP:-ollama_chat/llama3.1:8b-instruct-q4_K_M}