Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

feat(agents,api): route LLM tasks through gateway aliases; drop default model (#478) (#491)

Aadit Singal460cc4f0fa6c
diff --git a/.env.example b/.env.example
index 67b079216..86ba02c3d 100644
--- a/.env.example
+++ b/.env.example
@@ -83,25 +83,33 @@ QDRANT_URL=http://localhost:6333
# ─── AI / LLM ─────────────────────────────────────────────────────────────────
OPENAI_API_KEY=sk-your-openai-api-key-here
+# OPENAI_MODEL now only affects the "explain this alert" / BYOK path. Task
+# workloads (triage, recon, investigation, copilot, summary, report, NL) request
+# a logical alias instead — see the LLM gateway block below.
OPENAI_MODEL=gpt-4-turbo-preview
# ─── LLM gateway (LiteLLM) — issue #478 ───────────────────────────────────────
-# LiteLLM is the single entry point for live LLM calls. AiSOC asks for a logical
-# task alias (aisoc-triage, aisoc-investigation, …); the alias → real-model map
-# lives in infra/litellm/config.yaml, so you can swap local/hosted models there
-# without changing AiSOC. The service is defined in docker-compose.yml.
+# Since #478 AiSOC asks for a logical task alias (aisoc-triage,
+# aisoc-investigation, …) for every live LLM call — there is no hardcoded default
+# model. The alias → real-model map lives in infra/litellm/config.yaml, so you
+# swap local/hosted models there without changing AiSOC. The gateway service is
+# defined in docker-compose.yml.
#
# Key that AiSOC uses to authenticate to the gateway (also LiteLLM's admin key):
LITELLM_MASTER_KEY=sk-aisoc-local
# In-network URL of the gateway (already set per-service in docker-compose.yml):
LLM_GATEWAY_URL=http://litellm:4000/v1
#
-# To ROUTE AiSOC through the gateway, uncomment the two lines below — they make
-# every service send its calls to LiteLLM instead of straight to the provider.
-# Put your real provider key in OPENAI_API_KEY above (LiteLLM reads it), then:
-# OPENAI_BASE_URL=http://litellm:4000/v1
-# OPENAI_API_KEY=${LITELLM_MASTER_KEY}
-# Leave them unset to keep calling the provider directly (unchanged behaviour).
+# ENABLE LIVE LLM — pick one:
+# (a) Route through the gateway (recommended). Put your real provider key in
+# OPENAI_API_KEY above (LiteLLM reads it), then uncomment:
+# OPENAI_BASE_URL=http://litellm:4000/v1
+# OPENAI_API_KEY=${LITELLM_MASTER_KEY}
+# (b) Call a provider directly without the gateway — pin each task role to a
+# concrete model (escape hatch), e.g.:
+# AISOC_MODEL_PIN_TRIAGE=gpt-4o-mini
+# AISOC_MODEL_PIN_INVESTIGATION=gpt-4o
+# With neither configured, AiSOC uses its deterministic offline path.
# ─── Threat Intelligence ──────────────────────────────────────────────────────
# Open-source / community feeds (work out of the box, no key required for some)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index f51bb257b..23ded556c 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -27,6 +27,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
`services/agents/tests/test_litellm_config.py`. (A follow-up PR wires the ~10
in-code callsites to request these aliases via `model_pins` and removes the
hardcoded `gpt-4o-mini` default, closing #478.)
+- **LLM task-alias routing — no more shipped default model
+ ([#478](https://github.com/beenuar/AiSOC/issues/478), PR2).** Every live LLM
+ call now asks for a **logical task alias** instead of a hardcoded model. New
+ `services/agents/app/llm/factory.py` (`make_chat_model` / `resolve_model_alias`)
+ resolves a task role to its `aisoc-<role>` alias + the gateway base URL;
+ `model_pins.py` now pins all seven roles (triage, recon, investigation, copilot,
+ summary, report, nl) to aliases with a `deterministic` floor. The ~10 scattered
+ `os.getenv("AISOC_LLM_MODEL"/"OPENAI_MODEL","gpt-4o-mini")` + `ChatOpenAI(...)` /
+ raw-HTTP callsites across the agents (auto-triage, cloud/identity/insider/
+ phishing, recon/forensic/responder/report-writer, copilot, contextual, NL
+ translator) and the API endpoints (translation, hunts, knowledge base, phishing;
+ via new `services/api/app/services/model_aliases.py`) now request aliases; the
+ hardcoded `gpt-4o-mini` default is gone. **Behaviour change:** live LLM now runs
+ through the gateway (`OPENAI_BASE_URL`), or pin a concrete model per role via
+ `AISOC_MODEL_PIN_<ROLE>` (escape hatch; the slim demo does this so keyed demos
+ keep working); with neither, the deterministic offline path is used. Tests:
+ `services/agents/tests/test_llm_factory.py` + tightened `test_litellm_config.py`.
+ Closes #478.
- **v8 P4 — Compounding Memory (verdicts that measurably improve).** New
`services/fusion/app/memory/`: a nightly-distillable institutional memory that
makes verdicts more accurate the longer an instance runs. **Distillation**
diff --git a/apps/docs/docs/operations/llm-gateway.md b/apps/docs/docs/operations/llm-gateway.md
index 53f463656..e2b149e93 100644
--- a/apps/docs/docs/operations/llm-gateway.md
+++ b/apps/docs/docs/operations/llm-gateway.md
@@ -62,9 +62,19 @@ OPENAI_BASE_URL=http://litellm:4000/v1 # send AiSOC's calls to the gateway
# OPENAI_API_KEY=${LITELLM_MASTER_KEY} # (in the AiSOC services' environment)
```
-Leave `OPENAI_BASE_URL` unset to keep calling the provider directly — the
-gateway then runs idle (still serving `/health` and `/metrics`) and behaviour is
-unchanged.
+AiSOC now requests a task **alias** for every live call, so an alias only
+resolves when it reaches the gateway. If you don't run the gateway, pin each
+role to a concrete provider model instead (the **escape hatch**):
+
+```bash
+AISOC_MODEL_PIN_TRIAGE=gpt-4o-mini
+AISOC_MODEL_PIN_INVESTIGATION=gpt-4o
+# … one per role: triage, recon, investigation, copilot, summary, report, nl
+```
+
+With neither the gateway nor pin overrides configured, AiSOC uses its
+deterministic offline path. (`OPENAI_MODEL` still applies to the separate
+"explain this alert" / BYOK path.)
## Re-point a task to a local model
diff --git a/infra/compose/docker-compose.demo.yml b/infra/compose/docker-compose.demo.yml
index 0b96fc72a..99c98041b 100644
--- a/infra/compose/docker-compose.demo.yml
+++ b/infra/compose/docker-compose.demo.yml
@@ -191,6 +191,17 @@ services:
OPENAI_API_KEY: ${OPENAI_API_KEY:-}
ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:-}
OPENAI_MODEL: ${OPENAI_MODEL:-gpt-4o-mini}
+ # Issue #478 — the slim demo runs without the LiteLLM gateway, so pin every
+ # task role to a concrete provider model (AiSOC calls the provider directly
+ # with OPENAI_API_KEY; deterministic fallback when no key is set). The full
+ # stack (root docker-compose.yml) routes these aliases through the gateway.
+ AISOC_MODEL_PIN_TRIAGE: ${OPENAI_MODEL:-gpt-4o-mini}
+ AISOC_MODEL_PIN_RECON: ${OPENAI_MODEL:-gpt-4o-mini}
+ AISOC_MODEL_PIN_INVESTIGATION: ${OPENAI_MODEL:-gpt-4o-mini}
+ AISOC_MODEL_PIN_COPILOT: ${OPENAI_MODEL:-gpt-4o-mini}
+ AISOC_MODEL_PIN_SUMMARY: ${OPENAI_MODEL:-gpt-4o-mini}
+ AISOC_MODEL_PIN_REPORT: ${OPENAI_MODEL:-gpt-4o-mini}
+ AISOC_MODEL_PIN_NL: ${OPENAI_MODEL:-gpt-4o-mini}
ENV: development
networks:
- aisoc-demo
diff --git a/services/agents/app/agents/auto_triage_agent.py b/services/agents/app/agents/auto_triage_agent.py
index 8647ef440..9081bb7a1 100644
--- a/services/agents/app/agents/auto_triage_agent.py
+++ b/services/agents/app/agents/auto_triage_agent.py
@@ -19,10 +19,10 @@ from typing import Any
import structlog
from langchain_core.messages import HumanMessage, SystemMessage
-from langchain_openai import ChatOpenAI
from app.investigator.prompt_sanitizer import sanitize_text, wrap_untrusted
from app.llm import safe_ainvoke
+from app.llm.factory import make_chat_model
from app.models.state import AgentStatus, InvestigationState
from app.prompt_serialization import format_extra_fields_for_llm
@@ -176,8 +176,7 @@ async def run_auto_triage(state: InvestigationState) -> InvestigationState:
alert_context = _build_alert_context(state)
- model_name = os.getenv("AISOC_LLM_MODEL", "gpt-4o-mini")
- llm = ChatOpenAI(model=model_name, temperature=0.0, max_tokens=512)
+ llm = make_chat_model("triage", temperature=0.0, max_tokens=512)
t0 = time.monotonic()
try:
diff --git a/services/agents/app/agents/cloud_agent.py b/services/agents/app/agents/cloud_agent.py
index 01b72f14d..7100044d3 100644
--- a/services/agents/app/agents/cloud_agent.py
+++ b/services/agents/app/agents/cloud_agent.py
@@ -10,17 +10,16 @@ a confidence-weighted verdict.
from __future__ import annotations
import json
-import os
import time
from typing import Any
import structlog
from langchain_core.messages import HumanMessage, SystemMessage
-from langchain_openai import ChatOpenAI
from app.context import ContextBundle
from app.investigator.prompt_sanitizer import sanitize_text, wrap_untrusted
from app.llm import safe_ainvoke
+from app.llm.factory import make_chat_model
from app.models.state import AgentStatus, InvestigationState
from app.prompt_serialization import format_extra_fields_for_llm, summarize_structure_for_llm
@@ -200,8 +199,7 @@ async def run_cloud(
bundle_lines = bundle.prompt_context_lines() if bundle is not None else []
prompt_context = base_context + (("\n" + "\n".join(bundle_lines)) if bundle_lines else "")
- model_name = os.getenv("AISOC_LLM_MODEL", "gpt-4o-mini")
- llm = ChatOpenAI(model=model_name, temperature=0.0, max_tokens=768)
+ llm = make_chat_model("investigation", temperature=0.0, max_tokens=768)
t0 = time.monotonic()
try:
diff --git a/services/agents/app/agents/identity_agent.py b/services/agents/app/agents/identity_agent.py
index cb43122ef..7fe0128e5 100644
--- a/services/agents/app/agents/identity_agent.py
+++ b/services/agents/app/agents/identity_agent.py
@@ -9,17 +9,16 @@ to classify the alert and assign a confidence-weighted verdict.
from __future__ import annotations
import json
-import os
import time
from typing import Any
import structlog
from langchain_core.messages import HumanMessage, SystemMessage
-from langchain_openai import ChatOpenAI
from app.context import ContextBundle
from app.investigator.prompt_sanitizer import sanitize_text, wrap_untrusted
from app.llm import safe_ainvoke
+from app.llm.factory import make_chat_model
from app.models.state import AgentStatus, InvestigationState
from app.prompt_serialization import format_extra_fields_for_llm, summarize_structure_for_llm
@@ -195,8 +194,7 @@ async def run_identity(
bundle_lines = bundle.prompt_context_lines() if bundle is not None else []
prompt_context = base_context + (("\n" + "\n".join(bundle_lines)) if bundle_lines else "")
- model_name = os.getenv("AISOC_LLM_MODEL", "gpt-4o-mini")
- llm = ChatOpenAI(model=model_name, temperature=0.0, max_tokens=768)
+ llm = make_chat_model("investigation", temperature=0.0, max_tokens=768)
t0 = time.monotonic()
try:
diff --git a/services/agents/app/agents/insider_threat_agent.py b/services/agents/app/agents/insider_threat_agent.py
index 340136a2d..42d2addda 100644
--- a/services/agents/app/agents/insider_threat_agent.py
+++ b/services/agents/app/agents/insider_threat_agent.py
@@ -10,17 +10,16 @@ assign a confidence-weighted verdict.
from __future__ import annotations
import json
-import os
import time
from typing import Any
import structlog
from langchain_core.messages import HumanMessage, SystemMessage
-from langchain_openai import ChatOpenAI
from app.context import ContextBundle
from app.investigator.prompt_sanitizer import sanitize_text, wrap_untrusted
from app.llm import safe_ainvoke
+from app.llm.factory import make_chat_model
from app.models.state import AgentStatus, InvestigationState
from app.prompt_serialization import format_extra_fields_for_llm, summarize_structure_for_llm
@@ -218,8 +217,7 @@ async def run_insider_threat(
bundle_lines = bundle.prompt_context_lines() if bundle is not None else []
prompt_context = base_context + (("\n" + "\n".join(bundle_lines)) if bundle_lines else "")
- model_name = os.getenv("AISOC_LLM_MODEL", "gpt-4o-mini")
- llm = ChatOpenAI(model=model_name, temperature=0.0, max_tokens=768)
+ llm = make_chat_model("investigation", temperature=0.0, max_tokens=768)
t0 = time.monotonic()
try:
diff --git a/services/agents/app/agents/phishing_agent.py b/services/agents/app/agents/phishing_agent.py
index e03f9f70f..8c1e2678a 100644
--- a/services/agents/app/agents/phishing_agent.py
+++ b/services/agents/app/agents/phishing_agent.py
@@ -10,17 +10,16 @@ indicators and sets a verdict with calibrated confidence.
from __future__ import annotations
import json
-import os
import time
from typing import Any
import structlog
from langchain_core.messages import HumanMessage, SystemMessage
-from langchain_openai import ChatOpenAI
from app.context import ContextBundle
from app.investigator.prompt_sanitizer import sanitize_text, wrap_untrusted
from app.llm import safe_ainvoke
+from app.llm.factory import make_chat_model
from app.models.state import AgentStatus, InvestigationState
from app.prompt_serialization import format_extra_fields_for_llm, summarize_structure_for_llm
@@ -187,8 +186,7 @@ async def run_phishing(
bundle_lines = bundle.prompt_context_lines() if bundle is not None else []
prompt_context = base_context + (("\n" + "\n".join(bundle_lines)) if bundle_lines else "")
- model_name = os.getenv("AISOC_LLM_MODEL", "gpt-4o-mini")
- llm = ChatOpenAI(model=model_name, temperature=0.0, max_tokens=768)
+ llm = make_chat_model("investigation", temperature=0.0, max_tokens=768)
t0 = time.monotonic()
try:
diff --git a/services/agents/app/api/contextual.py b/services/agents/app/api/contextual.py
index 2d3a271aa..cc0cbdbbc 100644
--- a/services/agents/app/api/contextual.py
+++ b/services/agents/app/api/contextual.py
@@ -46,6 +46,7 @@ from fastapi.responses import StreamingResponse
from pydantic import BaseModel, Field
from app.llm import safe_ainvoke, safe_astream
+from app.llm.factory import resolve_model_alias
from app.prompt_serialization import summarize_structure_for_llm
logger = structlog.get_logger()
@@ -418,10 +419,10 @@ def _fallback_response(system: str, user: str) -> str:
"You're seeing a deterministic placeholder response.\n\n"
"**To enable real contextual answers:**\n\n"
"1. Set `OPENAI_API_KEY` in your `.env` file\n"
- "2. Optionally set `OPENAI_MODEL` (default: `gpt-4o-mini`)\n"
+ "2. Point AiSOC at the LiteLLM gateway (`OPENAI_BASE_URL=http://litellm:4000/v1`), "
+ "which maps each task alias to a real model\n"
"3. Restart the `aisoc-agents` service\n\n"
- "Refer to [the docs](https://example.com/docs/copilot) for self-hosted "
- "model alternatives (Ollama, vLLM, Together)."
+ "See the LLM-gateway docs for self-hosted model alternatives (Ollama, vLLM, Together)."
)
@@ -445,7 +446,7 @@ async def list_actions() -> ContextualActionsCatalogue:
async def run_action(req: ContextualActionRequest) -> ContextualActionResponse:
started = time.monotonic()
system, user = _build_messages(req)
- model = os.getenv("OPENAI_MODEL", "gpt-4o-mini")
+ model = resolve_model_alias("copilot")
fallback = not bool(os.getenv("OPENAI_API_KEY"))
try:
@@ -503,7 +504,7 @@ async def run_action(req: ContextualActionRequest) -> ContextualActionResponse:
)
async def run_action_stream(req: ContextualActionRequest) -> StreamingResponse:
system, user = _build_messages(req)
- model = os.getenv("OPENAI_MODEL", "gpt-4o-mini")
+ model = resolve_model_alias("copilot")
title = _TITLES.get((req.page, req.action), f"{req.page} · {req.action}")
suggestions = _FOLLOW_UPS.get((req.page, req.action), [])
fallback = not bool(os.getenv("OPENAI_API_KEY"))
diff --git a/services/agents/app/api/copilot.py b/services/agents/app/api/copilot.py
index ec39ea4df..0a13a46d2 100644
--- a/services/agents/app/api/copilot.py
+++ b/services/agents/app/api/copilot.py
@@ -122,6 +122,7 @@ async def _get_openai_reply(
try:
from app.llm.contract import safe_chat_completions_request
+ from app.llm.factory import chat_completions_url, resolve_model_alias
messages: list[dict[str, str]] = [
{
@@ -140,8 +141,9 @@ async def _get_openai_reply(
body = await safe_chat_completions_request(
api_key=api_key,
- model="gpt-4o-mini",
+ model=resolve_model_alias("copilot"),
messages=messages,
+ url=chat_completions_url(),
max_tokens=512,
)
return body["choices"][0]["message"]["content"]
diff --git a/services/agents/app/investigator/forensic_agent.py b/services/agents/app/investigator/forensic_agent.py
index fc2f8d548..c5c94b9a5 100644
--- a/services/agents/app/investigator/forensic_agent.py
+++ b/services/agents/app/investigator/forensic_agent.py
@@ -17,10 +17,10 @@ from typing import Any
import structlog
from langchain_core.messages import HumanMessage, SystemMessage
-from langchain_openai import ChatOpenAI
from app.core.cost_telemetry import record_llm_call
from app.llm import safe_ainvoke
+from app.llm.factory import make_chat_model, resolve_model_alias
from app.prompt_serialization import summarize_structure_for_llm
from .bundle_prompt import format_bundle_prompt_append
@@ -54,10 +54,8 @@ Respond ONLY with a JSON object:
async def _llm_forensic(state: InvestigatorState) -> dict[str, Any]:
- import os
-
- model = os.getenv("OPENAI_MODEL", "gpt-4o-mini")
- llm = ChatOpenAI(model=model, temperature=0)
+ model = resolve_model_alias("investigation")
+ llm = make_chat_model("investigation", temperature=0)
# Defence-in-depth: alert_summary, recon.summary, and the enrichment cache
# can all carry attacker-controlled strings (banners, dark-web excerpts,
diff --git a/services/agents/app/investigator/recon_agent.py b/services/agents/app/investigator/recon_agent.py
index 309fb2740..832a13673 100644
--- a/services/agents/app/investigator/recon_agent.py
+++ b/services/agents/app/investigator/recon_agent.py
@@ -16,10 +16,10 @@ from typing import Any
import structlog
from langchain_core.messages import HumanMessage, SystemMessage
-from langchain_openai import ChatOpenAI
from app.core.cost_telemetry import record_llm_call
from app.llm import safe_ainvoke
+from app.llm.factory import make_chat_model, resolve_model_alias
from app.prompt_serialization import summarize_structure_for_llm
from .bundle_prompt import format_bundle_prompt_append
@@ -54,10 +54,9 @@ async def _llm_recon(state: InvestigatorState) -> dict[str, Any]:
reasoning trace is replayable.
"""
import json
- import os
- model = os.getenv("OPENAI_MODEL", "gpt-4o-mini")
- llm = ChatOpenAI(model=model, temperature=0)
+ model = resolve_model_alias("recon")
+ llm = make_chat_model("recon", temperature=0)
# Defence-in-depth: every field surfaced here can be attacker-influenced
# (alert_summary often echoes log lines; raw_alert is verbatim event data).
diff --git a/services/agents/app/investigator/report_writer_agent.py b/services/agents/app/investigator/report_writer_agent.py
index a60dc50d6..6bfb96d85 100644
--- a/services/agents/app/investigator/report_writer_agent.py
+++ b/services/agents/app/investigator/report_writer_agent.py
@@ -15,10 +15,10 @@ from typing import Any
import structlog
from langchain_core.messages import HumanMessage, SystemMessage
-from langchain_openai import ChatOpenAI
from app.core.cost_telemetry import record_llm_call
from app.llm import safe_ainvoke
+from app.llm.factory import make_chat_model, resolve_model_alias
from app.prompt_serialization import summarize_structure_for_llm
from .bundle_prompt import format_bundle_prompt_append
@@ -158,15 +158,13 @@ def _md_to_html(md: str, case_id: str) -> str:
async def run_report_writer(state_dict: dict[str, Any]) -> dict[str, Any]:
"""LangGraph node."""
- import os
-
state = InvestigatorState.from_dict(state_dict)
t0 = time.monotonic()
logger.info("report_writer.start", case_id=state.case_id)
- model = os.getenv("OPENAI_MODEL", "gpt-4o-mini")
- llm = ChatOpenAI(model=model, temperature=0)
+ model = resolve_model_alias("report")
+ llm = make_chat_model("report", temperature=0)
context = _build_context(state)
bundle_append = format_bundle_prompt_append(state.context_bundle)
diff --git a/services/agents/app/investigator/responder_agent.py b/services/agents/app/investigator/responder_agent.py
index 015862522..2ed9e4002 100644
--- a/services/agents/app/investigator/responder_agent.py
+++ b/services/agents/app/investigator/responder_agent.py
@@ -16,10 +16,10 @@ from typing import Any
import structlog
from langchain_core.messages import HumanMessage, SystemMessage
-from langchain_openai import ChatOpenAI
from app.core.cost_telemetry import record_llm_call
from app.llm import safe_ainvoke
+from app.llm.factory import make_chat_model, resolve_model_alias
from app.prompt_serialization import summarize_structure_for_llm
from .bundle_prompt import format_bundle_prompt_append
@@ -52,10 +52,8 @@ Respond ONLY with a JSON object:
async def _llm_responder(state: InvestigatorState) -> dict[str, Any]:
- import os
-
- model = os.getenv("OPENAI_MODEL", "gpt-4o-mini")
- llm = ChatOpenAI(model=model, temperature=0)
+ model = resolve_model_alias("investigation")
+ llm = make_chat_model("investigation", temperature=0)
# Defence-in-depth: every field surfaced here originated in attacker-
# influenced data (alert payloads, banners, dark-web excerpts, LLM
diff --git a/services/agents/app/llm/factory.py b/services/agents/app/llm/factory.py
new file mode 100644
index 000000000..b68f04d4d
--- /dev/null
+++ b/services/agents/app/llm/factory.py
@@ -0,0 +1,87 @@
+"""LLM factory — turn a task role into a ready chat model / model alias (#478).
+
+Every live LLM call in ``services/agents`` asks for a **logical task alias**
+(``aisoc-triage``, ``aisoc-recon``, …) rather than a concrete model. The LiteLLM
+gateway (``infra/litellm/config.yaml``) owns the alias → real-model mapping and
+any model-level fallback, so operators re-point models without touching code.
+This module is the single place that resolves a role to ``(alias, base_url)`` and
+hands back a configured chat model.
+
+Since #478 there is **no hardcoded default model**. The alias comes from
+:func:`app.llm.model_pins.get_pin`, which is env-overridable per role
+(``AISOC_MODEL_PIN_<ROLE>``) so a deployment that calls a provider directly
+instead of through the gateway can pin a concrete model. When no live LLM is
+reachable, callers fall back to their deterministic path exactly as before — the
+factory never forces a live call.
+
+Routing to the gateway is deliberate: set ``OPENAI_BASE_URL`` (or ``LLM_BASE_URL``)
+to the gateway and send ``OPENAI_API_KEY=$LITELLM_MASTER_KEY``. Left unset, the
+client talks to its provider default — where an alias only resolves if the
+operator has pinned a concrete model via ``AISOC_MODEL_PIN_<ROLE>``.
+"""
+
+from __future__ import annotations
+
+import os
+from typing import Any
+
+from langchain_openai import ChatOpenAI
+
+from app.llm.contract import DEFAULT_OPENAI_CHAT_COMPLETIONS_URL
+from app.llm.model_pins import get_pin
+
+
+def resolve_model_alias(role: str) -> str:
+ """Return the logical model alias AiSOC sends for a task ``role``.
+
+ ``aisoc-<role>`` by default; ``AISOC_MODEL_PIN_<ROLE>`` overrides it with a
+ concrete provider model for deployments that bypass the gateway.
+ """
+ return get_pin(role).primary_model
+
+
+def resolve_base_url() -> str | None:
+ """OpenAI-compatible base URL for live calls, or ``None`` for the client default.
+
+ Honours an explicit ``OPENAI_BASE_URL`` / ``LLM_BASE_URL`` — what an operator
+ sets to point AiSOC at the LiteLLM gateway. ``None`` means "use the client's
+ provider default" (the direct-to-provider path). We intentionally do **not**
+ auto-adopt the compose-provided ``LLM_GATEWAY_URL`` here: routing through the
+ gateway is an explicit choice so the bearer token (the gateway master key vs.
+ a provider key) is never ambiguous.
+ """
+ return os.getenv("OPENAI_BASE_URL", "").strip() or os.getenv("LLM_BASE_URL", "").strip() or None
+
+
+def chat_completions_url() -> str:
+ """Full chat-completions URL for the raw-HTTP path (:func:`safe_chat_completions_request`)."""
+ base = resolve_base_url()
+ if base:
+ return base.rstrip("/") + "/chat/completions"
+ return DEFAULT_OPENAI_CHAT_COMPLETIONS_URL
+
+
+def make_chat_model(
+ role: str,
+ *,
+ temperature: float = 0.0,
+ max_tokens: int | None = None,
+ **kwargs: Any,
+) -> ChatOpenAI:
+ """Build a :class:`ChatOpenAI` for a task ``role`` (alias + gateway base URL).
+
+ The returned model is **not** contract-guarded — every caller still routes the
+ invocation through :func:`app.llm.safe_ainvoke`, which enforces the input
+ contract. Extra ``kwargs`` pass straight through to ``ChatOpenAI``.
+ """
+ params: dict[str, Any] = {
+ "model": resolve_model_alias(role),
+ "temperature": temperature,
+ }
+ if max_tokens is not None:
+ params["max_tokens"] = max_tokens
+ base_url = resolve_base_url()
+ if base_url:
+ params["base_url"] = base_url
+ params.update(kwargs)
+ return ChatOpenAI(**params)
diff --git a/services/agents/app/llm/model_pins.py b/services/agents/app/llm/model_pins.py
index 93df06999..40871bd4e 100644
--- a/services/agents/app/llm/model_pins.py
+++ b/services/agents/app/llm/model_pins.py
@@ -36,13 +36,24 @@ class ModelPin:
return chain
-# Shipped pins. Model IDs are the pinned defaults; override via env
-# (AISOC_MODEL_PIN_<ROLE>, e.g. AISOC_MODEL_PIN_TRIAGE). Every chain terminates
-# in `deterministic`.
+# Shipped pins. Since #478 the pinned primary is a **logical gateway alias**
+# (`aisoc-<role>`), not a concrete model — the LiteLLM gateway
+# (`infra/litellm/config.yaml`) owns the alias → real-model mapping and any
+# model-level fallback, so each AiSOC-side chain only needs its `deterministic`
+# floor. One role per AiSOC LLM workload named in #478.
+#
+# Escape hatch for deployments not running the gateway: override any primary via
+# env (`AISOC_MODEL_PIN_<ROLE>`, e.g. `AISOC_MODEL_PIN_TRIAGE=gpt-4o-mini`) to
+# send a concrete provider model directly. Every chain still terminates in
+# `deterministic`.
_DEFAULT_PINS: dict[str, ModelPin] = {
- "triage": ModelPin("triage", "gpt-4o-mini", ["gpt-4o", DETERMINISTIC]),
- "recon": ModelPin("recon", "gpt-4o-mini", ["gpt-4o", DETERMINISTIC]),
- "summary": ModelPin("summary", "gpt-4o-mini", [DETERMINISTIC]),
+ "triage": ModelPin("triage", "aisoc-triage", [DETERMINISTIC]),
+ "recon": ModelPin("recon", "aisoc-recon", [DETERMINISTIC]),
+ "investigation": ModelPin("investigation", "aisoc-investigation", [DETERMINISTIC]),
+ "copilot": ModelPin("copilot", "aisoc-copilot", [DETERMINISTIC]),
+ "summary": ModelPin("summary", "aisoc-summary", [DETERMINISTIC]),
+ "report": ModelPin("report", "aisoc-report", [DETERMINISTIC]),
+ "nl": ModelPin("nl", "aisoc-nl", [DETERMINISTIC]),
}
diff --git a/services/agents/app/nl_query/translator.py b/services/agents/app/nl_query/translator.py
index 24f4a3c9f..19eb8006e 100644
--- a/services/agents/app/nl_query/translator.py
+++ b/services/agents/app/nl_query/translator.py
@@ -885,7 +885,7 @@ async def enhance_with_llm(
query: NLQuery,
*,
api_key: str,
- model: str = "gpt-4o-mini",
+ model: str | None = None,
timeout: float = 30.0,
fallback: TranslatedQuery | None = None,
) -> TranslatedQuery:
@@ -903,6 +903,7 @@ async def enhance_with_llm(
import textwrap
from app.llm.contract import safe_chat_completions_request
+ from app.llm.factory import chat_completions_url, resolve_model_alias
prompt = textwrap.dedent(
f"""
@@ -922,8 +923,9 @@ async def enhance_with_llm(
payload = await safe_chat_completions_request(
api_key=api_key,
- model=model,
+ model=model or resolve_model_alias("nl"),
messages=messages,
+ url=chat_completions_url(),
timeout=timeout,
response_format={"type": "json_object"},
temperature=0,
diff --git a/services/agents/tests/test_litellm_config.py b/services/agents/tests/test_litellm_config.py
index 85a03cc5c..051607f7f 100644
--- a/services/agents/tests/test_litellm_config.py
+++ b/services/agents/tests/test_litellm_config.py
@@ -79,9 +79,7 @@ def test_master_key_comes_from_env():
assert str(master_key).startswith("os.environ/"), "master_key must be an env reference, not a committed secret"
-def test_model_pins_roles_have_a_gateway_alias():
- # Ties the code's logical roles to the gateway. Subset today (model_pins
- # ships triage/recon/summary); becomes an equality guarantee once PR2 pins
- # all seven roles.
- for role in all_roles():
- assert f"aisoc-{role}" in EXPECTED_ALIASES, f"role '{role}' has no aisoc-{role} gateway alias"
+def test_model_pins_roles_match_gateway_aliases_exactly():
+ # Ties the code's logical roles to the gateway config: every pinned role has
+ # an ``aisoc-<role>`` alias in the config, and vice versa — no drift either way.
+ assert {f"aisoc-{role}" for role in all_roles()} == EXPECTED_ALIASES
diff --git a/services/agents/tests/test_llm_factory.py b/services/agents/tests/test_llm_factory.py
new file mode 100644
index 000000000..bed8d2783
--- /dev/null
+++ b/services/agents/tests/test_llm_factory.py
@@ -0,0 +1,88 @@
+"""Tests for the LLM factory — alias + base-URL resolution (#478)."""
+
+from __future__ import annotations
+
+import pytest
+from app.llm.factory import (
+ chat_completions_url,
+ make_chat_model,
+ resolve_base_url,
+ resolve_model_alias,
+)
+
+# app.llm.contract.DEFAULT_OPENAI_CHAT_COMPLETIONS_URL
+_DEFAULT_URL = "https://api.openai.com/v1/chat/completions"
+
+
[email protected](autouse=True)
+def _clean_llm_env(monkeypatch):
+ """Start each test from a known-empty LLM env."""
+ for var in (
+ "OPENAI_BASE_URL",
+ "LLM_BASE_URL",
+ "LLM_GATEWAY_URL",
+ "AISOC_MODEL_PIN_TRIAGE",
+ "AISOC_MODEL_PIN_NL",
+ ):
+ monkeypatch.delenv(var, raising=False)
+
+
+# ── resolve_model_alias ──────────────────────────────────────────────────────
+
+
+def test_alias_defaults_to_aisoc_role():
+ assert resolve_model_alias("triage") == "aisoc-triage"
+ assert resolve_model_alias("investigation") == "aisoc-investigation"
+ assert resolve_model_alias("nl") == "aisoc-nl"
+
+
+def test_alias_env_override_escape_hatch(monkeypatch):
+ monkeypatch.setenv("AISOC_MODEL_PIN_TRIAGE", "gpt-4o-mini")
+ assert resolve_model_alias("triage") == "gpt-4o-mini"
+ # Other roles are unaffected by a single-role override.
+ assert resolve_model_alias("nl") == "aisoc-nl"
+
+
+# ── resolve_base_url ─────────────────────────────────────────────────────────
+
+
+def test_base_url_none_by_default():
+ assert resolve_base_url() is None
+
+
+def test_base_url_prefers_openai_then_llm(monkeypatch):
+ monkeypatch.setenv("LLM_BASE_URL", "http://llm:9/v1")
+ assert resolve_base_url() == "http://llm:9/v1"
+ monkeypatch.setenv("OPENAI_BASE_URL", "http://gw:4000/v1")
+ assert resolve_base_url() == "http://gw:4000/v1"
+
+
+def test_base_url_ignores_gateway_url_env(monkeypatch):
+ # Routing through the gateway must be explicit (OPENAI_BASE_URL), never
+ # silently adopted from the compose-provided LLM_GATEWAY_URL.
+ monkeypatch.setenv("LLM_GATEWAY_URL", "http://litellm:4000/v1")
+ assert resolve_base_url() is None
+
+
+# ── chat_completions_url ─────────────────────────────────────────────────────
+
+
+def test_chat_completions_url_default():
+ assert chat_completions_url() == _DEFAULT_URL
+
+
+def test_chat_completions_url_appends_path(monkeypatch):
+ monkeypatch.setenv("OPENAI_BASE_URL", "http://litellm:4000/v1/")
+ assert chat_completions_url() == "http://litellm:4000/v1/chat/completions"
+
+
+# ── make_chat_model ──────────────────────────────────────────────────────────
+
+
+def test_make_chat_model_uses_alias_and_base_url(monkeypatch):
+ monkeypatch.setenv("OPENAI_API_KEY", "sk-test")
+ monkeypatch.setenv("OPENAI_BASE_URL", "http://litellm:4000/v1")
+ llm = make_chat_model("triage", temperature=0.0, max_tokens=256)
+ # langchain_openai exposes the model id as ``model_name``.
+ assert llm.model_name == "aisoc-triage"
+ assert str(llm.openai_api_base).rstrip("/") == "http://litellm:4000/v1"
diff --git a/services/api/app/_vendor/nl_query/translator.py b/services/api/app/_vendor/nl_query/translator.py
index 24f4a3c9f..19eb8006e 100644
--- a/services/api/app/_vendor/nl_query/translator.py
+++ b/services/api/app/_vendor/nl_query/translator.py
@@ -885,7 +885,7 @@ async def enhance_with_llm(
query: NLQuery,
*,
api_key: str,
- model: str = "gpt-4o-mini",
+ model: str | None = None,
timeout: float = 30.0,
fallback: TranslatedQuery | None = None,
) -> TranslatedQuery:
@@ -903,6 +903,7 @@ async def enhance_with_llm(
import textwrap
from app.llm.contract import safe_chat_completions_request
+ from app.llm.factory import chat_completions_url, resolve_model_alias
prompt = textwrap.dedent(
f"""
@@ -922,8 +923,9 @@ async def enhance_with_llm(
payload = await safe_chat_completions_request(
api_key=api_key,
- model=model,
+ model=model or resolve_model_alias("nl"),
messages=messages,
+ url=chat_completions_url(),
timeout=timeout,
response_format={"type": "json_object"},
temperature=0,
diff --git a/services/api/app/api/v1/endpoints/hunts.py b/services/api/app/api/v1/endpoints/hunts.py
index 702ca5cbf..b1e19e7b2 100644
--- a/services/api/app/api/v1/endpoints/hunts.py
+++ b/services/api/app/api/v1/endpoints/hunts.py
@@ -38,6 +38,7 @@ from sqlalchemy import text
from app.api.v1.deps import AuthUser, DBSession
from app.core.airgap import AirgapViolation, enforce_airgap_for_url
+from app.services.model_aliases import resolve_model_alias
logger = logging.getLogger(__name__)
@@ -139,7 +140,7 @@ async def _generate_queries(hypothesis: str, mitre: str | None) -> dict[str, str
if not api_key:
return None
base_url = os.getenv("LLM_BASE_URL", "https://api.openai.com/v1")
- model = os.getenv("LLM_MODEL", "gpt-4o-mini")
+ model = os.getenv("LLM_MODEL") or resolve_model_alias("nl")
user_msg = f"HYPOTHESIS: {hypothesis}"
if mitre:
user_msg += f"\nMITRE TECHNIQUE: {mitre}"
diff --git a/services/api/app/api/v1/endpoints/knowledge_base.py b/services/api/app/api/v1/endpoints/knowledge_base.py
index 1f1db2f62..c1c5a5001 100644
--- a/services/api/app/api/v1/endpoints/knowledge_base.py
+++ b/services/api/app/api/v1/endpoints/knowledge_base.py
@@ -29,6 +29,7 @@ from sqlalchemy import text
from app.api.v1.deps import AuthUser, DBSession
from app.core.airgap import AirgapViolation, enforce_airgap_for_url
from app.services.kb_chunking import chunk_text
+from app.services.model_aliases import resolve_model_alias
logger = logging.getLogger(__name__)
@@ -115,7 +116,7 @@ async def _synthesise(question: str, chunks: list[KBChunk]) -> str | None:
if not api_key:
return None
base_url = os.getenv("LLM_BASE_URL", "https://api.openai.com/v1")
- model = os.getenv("LLM_MODEL", "gpt-4o-mini")
+ model = os.getenv("LLM_MODEL") or resolve_model_alias("nl")
context = "\n\n".join(f"[{c.title}] chunk {c.chunk_index}:\n{c.content}" for c in chunks)
completions_url = f"{base_url}/chat/completions"
enforce_airgap_for_url(completions_url)
diff --git a/services/api/app/api/v1/endpoints/phishing.py b/services/api/app/api/v1/endpoints/phishing.py
index e54310228..4705c39f1 100644
--- a/services/api/app/api/v1/endpoints/phishing.py
+++ b/services/api/app/api/v1/endpoints/phishing.py
@@ -28,6 +28,7 @@ from sqlalchemy import text
from app.api.v1.deps import AuthUser, DBSession
from app.core.airgap import AirgapViolation, enforce_airgap_for_url
+from app.services.model_aliases import resolve_model_alias
logger = logging.getLogger(__name__)
@@ -92,7 +93,7 @@ async def _triage(artifact_kind: str, content: str, urls: list[str]) -> TriageRe
if not api_key:
return None
base_url = os.getenv("LLM_BASE_URL", "https://api.openai.com/v1")
- model = os.getenv("LLM_MODEL", "gpt-4o-mini")
+ model = os.getenv("LLM_MODEL") or resolve_model_alias("investigation")
user_msg = f"ARTIFACT TYPE: {artifact_kind}\n"
if urls:
user_msg += f"URLS: {', '.join(urls[:10])}\n"
diff --git a/services/api/app/api/v1/endpoints/translation.py b/services/api/app/api/v1/endpoints/translation.py
index 588f16a41..066b7b2ff 100644
--- a/services/api/app/api/v1/endpoints/translation.py
+++ b/services/api/app/api/v1/endpoints/translation.py
@@ -22,6 +22,7 @@ from fastapi import APIRouter, HTTPException, status
from pydantic import BaseModel, Field
from app.core.airgap import AirgapViolation, enforce_airgap_for_url
+from app.services.model_aliases import resolve_model_alias
router = APIRouter(prefix="/translation", tags=["translation"])
@@ -111,7 +112,7 @@ async def _llm_translate(req: TranslateRequest) -> dict[str, Any] | None:
if not api_key:
return None
base_url = os.getenv("LLM_BASE_URL", "https://api.openai.com/v1")
- model = os.getenv("LLM_MODEL", "gpt-4o-mini")
+ model = os.getenv("LLM_MODEL") or resolve_model_alias("nl")
completions_url = f"{base_url}/chat/completions"
# Air-gap check: refuses the call entirely (rather than silently
# falling back to template-substitution) so misconfigurations are
diff --git a/services/api/app/services/model_aliases.py b/services/api/app/services/model_aliases.py
new file mode 100644
index 000000000..94983cde8
--- /dev/null
+++ b/services/api/app/services/model_aliases.py
@@ -0,0 +1,26 @@
+"""Task role → LiteLLM gateway alias for the API service (#478).
+
+The API service is a separate package from ``services/agents``, so it can't
+import ``app.llm.factory``. This is the small mirror of
+:func:`services.agents.app.llm.factory.resolve_model_alias`: each LLM-backed API
+endpoint (translation, hunts, knowledge base, phishing) asks for a **logical
+alias**; the LiteLLM gateway (``infra/litellm/config.yaml``) owns the alias →
+real-model mapping. There is no hardcoded default model.
+
+Escape hatch for deployments not running the gateway: override any role with a
+concrete provider model via ``AISOC_MODEL_PIN_<ROLE>`` (kept in lockstep with the
+agents-side pins), or set a global ``LLM_MODEL`` at the call site.
+"""
+
+from __future__ import annotations
+
+import os
+
+# Mirrors the roles in services/agents/app/llm/model_pins.py.
+ROLES = frozenset({"triage", "recon", "investigation", "copilot", "summary", "report", "nl"})
+
+
+def resolve_model_alias(role: str) -> str:
+ """Return the ``aisoc-<role>`` alias, honouring an ``AISOC_MODEL_PIN_<ROLE>`` override."""
+ override = os.environ.get(f"AISOC_MODEL_PIN_{role.upper()}", "").strip()
+ return override or f"aisoc-{role}"