Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
agents: enforce LLM input contract on raw-HTTP call sites (T2.3) (#143)
Beenu Arora4ecce70a2aa0
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 7b187d9ad..9d17fa926 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]+### LLM input contract — raw-HTTP wrapper + remaining call sites (T2.3, v8.0)++Closes the last LLM-input-contract gap in `services/agents`. Every LangChain+chat-model path already routed through `safe_ainvoke` / `safe_astream` (or a+`make_safe_chat_model` wrap), but two call sites talked to OpenAI directly+over `httpx` and so could ship un-validated prompts on the wire: the Copilot+endpoint (`app/api/copilot.py::_get_openai_reply`) and the NL→query+translator's optional LLM enhancement (`app/nl_query/translator.py::enhance_with_llm`).+Both now go through a new `safe_chat_completions_request()` helper in+`app/llm/contract.py` that calls `LLMInputContract.validate()` **before** the+network request — a contract violation never leaves the process. The helper+exposes the same surface as the existing OpenAI-compatible `POST`+(`api_key`, `model`, `messages`, plus pass-through `response_format`,+`temperature`, `max_tokens`, custom `url`, extra headers), surfaces+`httpx.HTTPStatusError` so callers can fall back to deterministic behaviour,+and refuses to run with an empty API key.++Embedding calls in `app/tools/mitre_full.py` (`openai.AsyncOpenAI.embeddings`)+are intentionally unchanged: they embed curated MITRE technique descriptions,+not user input, so the contract does not apply.### `/investigate` endpoint swap behind feature flag (T2.2, v8.0)`services/agents/app/api/investigate.py` now picks an orchestrator per call@@ -201,12 +221,16 @@ rather than silently ignored.### LLM input contract — CI tests (T2.3, v8.0)`services/agents/tests/test_llm_contract.py` exercises `classify_message` /-`LLMInputContract.validate` / `validate_messages`: raw OCSF-shaped JSON in a-user message fails closed when `AISOC_AGENTS_LLM_CONTRACT_ENFORCED=1`-(default), and prose plus `summarize_structure_for_llm` output passes. Tests-use `{"role", "content"}` dict messages so they run without importing-`langchain_core` (the contract already coerces LangChain `BaseMessage` and-dicts the same way).+`LLMInputContract.validate` / `validate_messages` (raw OCSF-shaped JSON in a+user message fails closed when `AISOC_AGENTS_LLM_CONTRACT_ENFORCED=1`,+default; prose plus `summarize_structure_for_llm` output passes; tests use+`{"role", "content"}` dict messages so they run without importing+`langchain_core` — the contract coerces LangChain `BaseMessage` and dicts the+same way). New `services/agents/tests/test_llm_contract_http.py` covers the+HTTP wrapper end-to-end: happy path with body/header forwarding, contract+rejection of OCSF-shaped payloads with **no** network call, empty-API-key+guard, `HTTPStatusError` propagation on non-2xx, extra-body (`response_format`,+`temperature`) forwarding, extra-header merging, and custom-URL override.### Real-time graph-update WebSocket (T1.4, v8.0)diff --git a/apps/docs/docs/operations/security.md b/apps/docs/docs/operations/security.mdindex 64b8773fd..f4224ea83 100644--- a/apps/docs/docs/operations/security.md+++ b/apps/docs/docs/operations/security.md@@ -23,6 +23,7 @@ The companion page [Credentials & secrets](./credentials) covers connector-crede| **Audit log** | Immutable append-only log with actor, IP, user-agent, request-ID | All write actions || **Plugin verification** | Ed25519 signature verification on plugin manifests | Marketplace + private plugins || **LLM prompt safety** | Enrichment / alert text sanitised before reaching the model; outputs revalidated against schema | All investigator-agent LLM calls |+| **LLM input contract** | `safe_ainvoke` / `safe_astream` / `safe_chat_completions_request` enforce minimum-leak policy (no raw OCSF, no raw log lines, no obvious PII shapes) before any model call — including raw-HTTP call sites | Every LLM call in `services/agents` || **Transport** | TLS terminated at the ingress; service-mesh mTLS optional | All traffic || **Browser origin policy** | `AISOC_CORS_ORIGINS` allow-list with production wildcard-plus-credentials guard | Every Python, Go, and TypeScript service |@@ -213,6 +214,33 @@ This is **defence in depth**, not a guarantee — there is no way to make prompt- Run the agents with the strictest BYOK [air-gap policy](./credentials) that matches your data-residency requirements.- Restrict which playbook actions an LLM-summarised case can trigger automatically — destructive actions should still require a human approval step.+### LLM input contract (minimum-leak policy)++Layered on top of the prompt sanitiser, the **LLM input contract** in [`services/agents/app/llm/contract.py`](https://github.com/beenuar/AiSOC/blob/main/services/agents/app/llm/contract.py) enforces a different invariant: **what the prompt is allowed to contain in the first place**. Sanitisation cleans data; the contract refuses to let certain kinds of data reach the model at all.++The contract classifies every outgoing message as one of:++| Classification | Allowed? | Why |+|---|---|---|+| Prose / analyst text | yes | The intended payload — natural-language questions, structured summaries, schema-validated context. |+| Raw OCSF JSON | **no** | OCSF events embed deep nested PII (user names, hostnames, raw URLs, credentials in command lines). The agents are expected to call `summarize_structure_for_llm()` instead. |+| Raw vendor log lines | **no** | Syslog / EDR / CDN log lines smuggle attacker-controlled strings *and* internal telemetry that has no business reaching a third-party LLM. |+| Likely-PII payloads | **no** | Heuristic match on email/phone/IP-heavy bodies and obvious secret shapes (AWS keys, JWTs, `password=`, …). |++Enforcement is global, default-on (`AISOC_AGENTS_LLM_CONTRACT_ENFORCED=1`), and applied to **every** LLM call site in `services/agents` via three entry points — there is no other way to talk to a model from this service:++1. **`safe_ainvoke(llm, messages, **kwargs)`** and **`safe_astream(llm, messages, **kwargs)`** — wrap any LangChain-compatible chat model. The contract validates the materialised message list before `ainvoke` / `astream` is called.+2. **`make_safe_chat_model(llm)`** — adapts an existing model reference so its `.ainvoke` / `.astream` enforce the contract without rewriting call sites (used for LangGraph nodes that close over `llm`).+3. **`safe_chat_completions_request(api_key=..., model=..., messages=...)`** — the same guarantee for the two raw-HTTP call sites (`app/api/copilot.py::_get_openai_reply` and `app/nl_query/translator.py::enhance_with_llm`) that talk to an OpenAI-compatible `chat/completions` endpoint directly. The contract runs **before** the `httpx.post`, so a violation never leaves the process. The helper surfaces `httpx.HTTPStatusError` so callers can fall back to deterministic behaviour, and refuses to run with an empty API key.++A violation raises `LLMContractViolation` and is logged as `event="llm.contract.violation"`. In `enforced=False` mode (set explicitly per-test or via env var, never by default in production) the violation is logged but the call proceeds — useful for grandfathering a legacy call site while you migrate it, never appropriate for a real deployment.++Two intentional non-targets:+- The MITRE embedding tool (`app/tools/mitre_full.py`) calls `openai.AsyncOpenAI.embeddings` on curated MITRE technique descriptions, not user input, so the contract does not apply.+- The contract is about **structural classes of leak** (raw events, raw logs, obvious PII shapes). Semantic privacy review of free-form prose is out of scope — that's what `summarize_structure_for_llm` and BYOK air-gap policies are for.++Test coverage: [`services/agents/tests/test_llm_contract.py`](https://github.com/beenuar/AiSOC/blob/main/services/agents/tests/test_llm_contract.py) for the classifier and `safe_ainvoke` / `safe_astream` paths, [`services/agents/tests/test_llm_contract_http.py`](https://github.com/beenuar/AiSOC/blob/main/services/agents/tests/test_llm_contract_http.py) for the raw-HTTP wrapper (happy path, OCSF rejection with **no** network call, empty-API-key guard, `HTTPStatusError` propagation, extra-body / extra-header forwarding, custom URL).+### OCI install hardening (H-3)Plugins can be installed from an OCI image via `oras pull` (`POST /api/v1/plugins/install/oci`). Because the install path writes arbitrary files into `AISOC_PLUGINS_DIR` and then imports them, it is wrapped with several non-negotiable checks. They live in `services/api/app/services/plugin_manager.py` and are exercised by the test suite at `services/api/tests/test_plugin_manager.py`.diff --git a/services/agents/app/api/copilot.py b/services/agents/app/api/copilot.pyindex 75d0f7efb..ec39ea4df 100644--- a/services/agents/app/api/copilot.py+++ b/services/agents/app/api/copilot.py@@ -121,9 +121,9 @@ async def _get_openai_reply(return _synthetic_reply(user_message)try:- import httpx+ from app.llm.contract import safe_chat_completions_request- messages = [+ messages: list[dict[str, str]] = [{"role": "system","content": (@@ -138,14 +138,13 @@ async def _get_openai_reply(messages.append({"role": m["role"], "content": m["content"]})messages.append({"role": "user", "content": user_message})- async with httpx.AsyncClient(timeout=30) as client:- resp = await client.post(- "https://api.openai.com/v1/chat/completions",- headers={"Authorization": f"Bearer {api_key}"},- json={"model": "gpt-4o-mini", "messages": messages, "max_tokens": 512},- )- resp.raise_for_status()- return resp.json()["choices"][0]["message"]["content"]+ body = await safe_chat_completions_request(+ api_key=api_key,+ model="gpt-4o-mini",+ messages=messages,+ max_tokens=512,+ )+ return body["choices"][0]["message"]["content"]except Exception as exc:logger.warning("copilot.openai_error", error=str(exc))return _synthetic_reply(user_message)diff --git a/services/agents/app/llm/contract.py b/services/agents/app/llm/contract.pyindex 84c45ef96..d2994cf0d 100644--- a/services/agents/app/llm/contract.py+++ b/services/agents/app/llm/contract.py@@ -340,3 +340,62 @@ def make_safe_chat_model(llm: Any) -> Any:return safe_astream(self._inner, messages, **kwargs)return _ContractGuardedChatModel(llm)+++# ---------------------------------------------------------------------------+# Raw OpenAI-compatible chat-completions HTTP wrapper+# ---------------------------------------------------------------------------++DEFAULT_OPENAI_CHAT_COMPLETIONS_URL = "https://api.openai.com/v1/chat/completions"+++async def safe_chat_completions_request(+ *,+ api_key: str,+ model: str,+ messages: Iterable[Any],+ url: str = DEFAULT_OPENAI_CHAT_COMPLETIONS_URL,+ timeout: float = 30.0,+ extra_headers: dict[str, str] | None = None,+ **extra_body: Any,+) -> dict[str, Any]:+ """Validate ``messages`` against the contract, then issue a chat-completions POST.++ Use this for call sites in ``services/agents`` that talk to an+ OpenAI-compatible chat-completions endpoint over raw HTTP (e.g.+ ``api/copilot.py``, ``nl_query/translator.py``) instead of LangChain.+ The contract is enforced **before** the network request, so a+ violation never leaks log data on the wire.++ Returns the parsed JSON response. Raises:++ * :class:`LLMContractViolation` if any message fails the contract.+ * ``ValueError`` if ``api_key`` is empty.+ * ``httpx.HTTPError`` for transport / non-2xx responses (callers+ decide whether to fall back).+ """+ if not api_key:+ raise ValueError("api_key is required for safe_chat_completions_request")++ materialised = list(messages)+ LLMInputContract.validate(materialised)++ try:+ import httpx+ except ImportError as exc: # pragma: no cover - httpx is a hard dep+ raise RuntimeError("httpx is required for safe_chat_completions_request") from exc++ headers: dict[str, str] = {+ "Authorization": f"Bearer {api_key}",+ "Content-Type": "application/json",+ }+ if extra_headers:+ headers.update(extra_headers)++ body: dict[str, Any] = {"model": model, "messages": materialised}+ body.update(extra_body)++ async with httpx.AsyncClient(timeout=timeout) as client:+ resp = await client.post(url, headers=headers, json=body)+ resp.raise_for_status()+ return resp.json()diff --git a/services/agents/app/nl_query/translator.py b/services/agents/app/nl_query/translator.pyindex 2e8d6ca2b..f8df9e9c6 100644--- a/services/agents/app/nl_query/translator.py+++ b/services/agents/app/nl_query/translator.py@@ -887,7 +887,7 @@ async def enhance_with_llm(import jsonimport textwrap- import httpx # local import — keeps the offline path zero-dependency+ from app.llm.contract import safe_chat_completions_requestprompt = textwrap.dedent(f"""@@ -900,25 +900,19 @@ async def enhance_with_llm(""").strip()- async with httpx.AsyncClient(timeout=timeout) as client:- resp = await client.post(- "https://api.openai.com/v1/chat/completions",- headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},- json={- "model": model,- "response_format": {"type": "json_object"},- "temperature": 0,- "messages": [- {"role": "system", "content": "You translate security questions into queries."},- {"role": "user", "content": prompt},- ],- },- )- if resp.status_code != 200:- logger.info("nl_query LLM call failed: status=%s", resp.status_code)- return deterministic-- payload = resp.json()+ messages = [+ {"role": "system", "content": "You translate security questions into queries."},+ {"role": "user", "content": prompt},+ ]++ payload = await safe_chat_completions_request(+ api_key=api_key,+ model=model,+ messages=messages,+ timeout=timeout,+ response_format={"type": "json_object"},+ temperature=0,+ )content = payload["choices"][0]["message"]["content"]parsed = json.loads(content)esql = str(parsed.get("esql", "")).strip()diff --git a/services/agents/tests/test_four_agent_facade.py b/services/agents/tests/test_four_agent_facade.pyindex 1b2ab4794..5d7971dd5 100644--- a/services/agents/tests/test_four_agent_facade.py+++ b/services/agents/tests/test_four_agent_facade.py@@ -171,7 +171,7 @@ async def test_triage_heuristic_delegates(monkeypatch: pytest.MonkeyPatch) -> No@pytest.mark.asyncioasync def test_triage_capability_dispatch(monkeypatch: pytest.MonkeyPatch) -> None:"""Each capability dispatches to its underlying ``run_*`` function."""- flags = {name: False for name in ("phishing", "identity", "cloud", "insider")}+ flags = dict.fromkeys(("phishing", "identity", "cloud", "insider"), False)def _make_fake(name: str):async def _runner(state: InvestigationState) -> InvestigationState:diff --git a/services/agents/tests/test_llm_contract_http.py b/services/agents/tests/test_llm_contract_http.pynew file mode 100644index 000000000..1f58dbbaa--- /dev/null+++ b/services/agents/tests/test_llm_contract_http.py@@ -0,0 +1,183 @@+"""Tests for ``safe_chat_completions_request`` — the raw-HTTP LLM wrapper.++These tests ensure the helper:++* validates messages against the LLM input contract *before* hitting the+ network (so a violation never leaks on the wire),+* forwards body / headers / extra kwargs correctly,+* surfaces httpx transport errors so callers can decide whether to+ fall back to deterministic behaviour,+* refuses to run without an API key.+"""++from __future__ import annotations++import json+from typing import Any+from unittest.mock import AsyncMock, MagicMock, patch++import httpx+import pytest+from app.llm.contract import (+ DEFAULT_OPENAI_CHAT_COMPLETIONS_URL,+ LLMContractViolation,+ safe_chat_completions_request,+ set_contract_enforcement,+)+++def contract_enforced():+ prev = set_contract_enforcement(True)+ try:+ yield+ finally:+ set_contract_enforcement(prev)+++def _mock_async_client(response_json: dict[str, Any], status: int = 200):+ """Return a patched ``httpx.AsyncClient`` whose ``.post`` yields ``response_json``."""+ response = MagicMock(spec=httpx.Response)+ response.status_code = status+ response.json = MagicMock(return_value=response_json)+ if status >= 400:+ response.raise_for_status = MagicMock(side_effect=httpx.HTTPStatusError(f"status {status}", request=MagicMock(), response=response))+ else:+ response.raise_for_status = MagicMock(return_value=None)++ client = MagicMock()+ client.post = AsyncMock(return_value=response)+ client.__aenter__ = AsyncMock(return_value=client)+ client.__aexit__ = AsyncMock(return_value=None)+ return client, response+++async def test_safe_chat_completions_request_success(contract_enforced) -> None:+ """Happy-path: contract passes, body forwarded, parsed JSON returned."""+ expected_response = {"choices": [{"message": {"content": "hello"}}]}+ client, _ = _mock_async_client(expected_response)++ with patch("httpx.AsyncClient", return_value=client):+ result = await safe_chat_completions_request(+ api_key="sk-test",+ model="gpt-4o-mini",+ messages=[+ {"role": "system", "content": "You are an analyst."},+ {"role": "user", "content": "Summarise this alert title: Suspicious login"},+ ],+ max_tokens=128,+ )++ assert result == expected_response+ # body forwarded with all expected fields+ call_kwargs = client.post.call_args+ assert call_kwargs.args[0] == DEFAULT_OPENAI_CHAT_COMPLETIONS_URL+ body = call_kwargs.kwargs["json"]+ assert body["model"] == "gpt-4o-mini"+ assert body["max_tokens"] == 128+ assert body["messages"][0]["role"] == "system"+ headers = call_kwargs.kwargs["headers"]+ assert headers["Authorization"] == "Bearer sk-test"+ assert headers["Content-Type"] == "application/json"+++async def test_safe_chat_completions_request_rejects_raw_ocsf(contract_enforced) -> None:+ """Contract must reject OCSF-shaped payloads before any network call happens."""+ raw_log = json.dumps({"class_uid": 1, "activity_id": 2, "metadata": {"product": "X"}})++ client, _ = _mock_async_client({"choices": []})+ with patch("httpx.AsyncClient", return_value=client):+ with pytest.raises(LLMContractViolation):+ await safe_chat_completions_request(+ api_key="sk-test",+ model="gpt-4o-mini",+ messages=[{"role": "user", "content": raw_log}],+ )++ # critical: no network call was attempted+ client.post.assert_not_called()+++async def test_safe_chat_completions_request_requires_api_key() -> None:+ """Empty API key must raise before any contract / network work."""+ with pytest.raises(ValueError, match="api_key is required"):+ await safe_chat_completions_request(+ api_key="",+ model="gpt-4o-mini",+ messages=[{"role": "user", "content": "hi"}],+ )+++async def test_safe_chat_completions_request_propagates_http_errors(+ contract_enforced,+) -> None:+ """Non-2xx responses surface as ``httpx.HTTPStatusError`` for caller fallback."""+ client, _ = _mock_async_client({"error": "rate limited"}, status=429)+ with patch("httpx.AsyncClient", return_value=client):+ with pytest.raises(httpx.HTTPStatusError):+ await safe_chat_completions_request(+ api_key="sk-test",+ model="gpt-4o-mini",+ messages=[{"role": "user", "content": "summarise"}],+ )+++async def test_safe_chat_completions_request_forwards_extra_body(+ contract_enforced,+) -> None:+ """Extra kwargs (response_format, temperature, etc.) make it into the body."""+ client, _ = _mock_async_client({"choices": []})+ with patch("httpx.AsyncClient", return_value=client):+ await safe_chat_completions_request(+ api_key="sk-test",+ model="gpt-4o-mini",+ messages=[{"role": "user", "content": "translate to ES|QL"}],+ response_format={"type": "json_object"},+ temperature=0,+ )++ body = client.post.call_args.kwargs["json"]+ assert body["response_format"] == {"type": "json_object"}+ assert body["temperature"] == 0+++async def test_safe_chat_completions_request_merges_extra_headers(+ contract_enforced,+) -> None:+ """``extra_headers`` should be merged on top of the defaults."""+ client, _ = _mock_async_client({"choices": []})+ with patch("httpx.AsyncClient", return_value=client):+ await safe_chat_completions_request(+ api_key="sk-test",+ model="gpt-4o-mini",+ messages=[{"role": "user", "content": "hi"}],+ extra_headers={"OpenAI-Beta": "assistants=v2"},+ )++ headers = client.post.call_args.kwargs["headers"]+ assert headers["Authorization"] == "Bearer sk-test"+ assert headers["OpenAI-Beta"] == "assistants=v2"+++async def test_safe_chat_completions_request_honours_custom_url(+ contract_enforced,+) -> None:+ """Caller-provided URL overrides the default (for compatible endpoints)."""+ client, _ = _mock_async_client({"choices": []})+ custom_url = "https://api.example.test/v1/chat/completions"+ with patch("httpx.AsyncClient", return_value=client):+ await safe_chat_completions_request(+ api_key="sk-test",+ model="local-model",+ messages=[{"role": "user", "content": "hello"}],+ url=custom_url,+ )++ assert client.post.call_args.args[0] == custom_urldiff --git a/services/api/app/_vendor/nl_query/translator.py b/services/api/app/_vendor/nl_query/translator.pyindex 2e8d6ca2b..f8df9e9c6 100644--- a/services/api/app/_vendor/nl_query/translator.py+++ b/services/api/app/_vendor/nl_query/translator.py@@ -887,7 +887,7 @@ async def enhance_with_llm(import jsonimport textwrap- import httpx # local import — keeps the offline path zero-dependency+ from app.llm.contract import safe_chat_completions_requestprompt = textwrap.dedent(f"""@@ -900,25 +900,19 @@ async def enhance_with_llm(""").strip()- async with httpx.AsyncClient(timeout=timeout) as client:- resp = await client.post(- "https://api.openai.com/v1/chat/completions",- headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},- json={- "model": model,- "response_format": {"type": "json_object"},- "temperature": 0,- "messages": [- {"role": "system", "content": "You translate security questions into queries."},- {"role": "user", "content": prompt},- ],- },- )- if resp.status_code != 200:- logger.info("nl_query LLM call failed: status=%s", resp.status_code)- return deterministic-- payload = resp.json()+ messages = [+ {"role": "system", "content": "You translate security questions into queries."},+ {"role": "user", "content": prompt},+ ]++ payload = await safe_chat_completions_request(+ api_key=api_key,+ model=model,+ messages=messages,+ timeout=timeout,+ response_format={"type": "json_object"},+ temperature=0,+ )content = payload["choices"][0]["message"]["content"]parsed = json.loads(content)esql = str(parsed.get("esql", "")).strip()diff --git a/services/api/app/scripts/seed_demo.py b/services/api/app/scripts/seed_demo.pyindex 995e7bdf7..21d5d0c20 100644--- a/services/api/app/scripts/seed_demo.py+++ b/services/api/app/scripts/seed_demo.py@@ -3154,7 +3154,7 @@ async def _run_quick_seed(clock: datetime) -> None:print(f"[seed] tenant: {tenant.id} ({tenant.slug})")print(f"[seed] user: {user.email} (role={user.role})")print(f"[seed] connectors upserted: {connectors}")- print("[seed] purged DEMO-* — cases:%d alerts:%d timelines:%d" % (deleted_cases, deleted_alerts, deleted_timelines))+ print(f"[seed] purged DEMO-* — cases:{deleted_cases} alerts:{deleted_alerts} timelines:{deleted_timelines}")print(f"[seed] DEMO-* cases seeded: {cases}")print(f"[seed] DEMO-* alerts seeded: {alerts}")print(f"[seed] DEMO-* timelines seeded: {timelines}")