Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
feat(agents,api): route LLM tasks through gateway aliases; drop default model (#478) (#491)
Aadit Singal460cc4f0fa6c
diff --git a/.env.example b/.env.exampleindex 67b079216..86ba02c3d 100644--- a/.env.example+++ b/.env.example@@ -83,25 +83,33 @@ QDRANT_URL=http://localhost:6333# ─── AI / LLM ─────────────────────────────────────────────────────────────────OPENAI_API_KEY=sk-your-openai-api-key-here+# OPENAI_MODEL now only affects the "explain this alert" / BYOK path. Task+# workloads (triage, recon, investigation, copilot, summary, report, NL) request+# a logical alias instead — see the LLM gateway block below.OPENAI_MODEL=gpt-4-turbo-preview# ─── LLM gateway (LiteLLM) — issue #478 ───────────────────────────────────────-# LiteLLM is the single entry point for live LLM calls. AiSOC asks for a logical-# task alias (aisoc-triage, aisoc-investigation, …); the alias → real-model map-# lives in infra/litellm/config.yaml, so you can swap local/hosted models there-# without changing AiSOC. The service is defined in docker-compose.yml.+# Since #478 AiSOC asks for a logical task alias (aisoc-triage,+# aisoc-investigation, …) for every live LLM call — there is no hardcoded default+# model. The alias → real-model map lives in infra/litellm/config.yaml, so you+# swap local/hosted models there without changing AiSOC. The gateway service is+# defined in docker-compose.yml.## Key that AiSOC uses to authenticate to the gateway (also LiteLLM's admin key):LITELLM_MASTER_KEY=sk-aisoc-local# In-network URL of the gateway (already set per-service in docker-compose.yml):LLM_GATEWAY_URL=http://litellm:4000/v1#-# To ROUTE AiSOC through the gateway, uncomment the two lines below — they make-# every service send its calls to LiteLLM instead of straight to the provider.-# Put your real provider key in OPENAI_API_KEY above (LiteLLM reads it), then:-# OPENAI_BASE_URL=http://litellm:4000/v1-# OPENAI_API_KEY=${LITELLM_MASTER_KEY}-# Leave them unset to keep calling the provider directly (unchanged behaviour).+# ENABLE LIVE LLM — pick one:+# (a) Route through the gateway (recommended). Put your real provider key in+# OPENAI_API_KEY above (LiteLLM reads it), then uncomment:+# OPENAI_BASE_URL=http://litellm:4000/v1+# OPENAI_API_KEY=${LITELLM_MASTER_KEY}+# (b) Call a provider directly without the gateway — pin each task role to a+# concrete model (escape hatch), e.g.:+# AISOC_MODEL_PIN_TRIAGE=gpt-4o-mini+# AISOC_MODEL_PIN_INVESTIGATION=gpt-4o+# With neither configured, AiSOC uses its deterministic offline path.# ─── Threat Intelligence ──────────────────────────────────────────────────────# Open-source / community feeds (work out of the box, no key required for some)diff --git a/CHANGELOG.md b/CHANGELOG.mdindex f51bb257b..23ded556c 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -27,6 +27,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0`services/agents/tests/test_litellm_config.py`. (A follow-up PR wires the ~10in-code callsites to request these aliases via `model_pins` and removes thehardcoded `gpt-4o-mini` default, closing #478.)+- **LLM task-alias routing — no more shipped default model+ ([#478](https://github.com/beenuar/AiSOC/issues/478), PR2).** Every live LLM+ call now asks for a **logical task alias** instead of a hardcoded model. New+ `services/agents/app/llm/factory.py` (`make_chat_model` / `resolve_model_alias`)+ resolves a task role to its `aisoc-<role>` alias + the gateway base URL;+ `model_pins.py` now pins all seven roles (triage, recon, investigation, copilot,+ summary, report, nl) to aliases with a `deterministic` floor. The ~10 scattered+ `os.getenv("AISOC_LLM_MODEL"/"OPENAI_MODEL","gpt-4o-mini")` + `ChatOpenAI(...)` /+ raw-HTTP callsites across the agents (auto-triage, cloud/identity/insider/+ phishing, recon/forensic/responder/report-writer, copilot, contextual, NL+ translator) and the API endpoints (translation, hunts, knowledge base, phishing;+ via new `services/api/app/services/model_aliases.py`) now request aliases; the+ hardcoded `gpt-4o-mini` default is gone. **Behaviour change:** live LLM now runs+ through the gateway (`OPENAI_BASE_URL`), or pin a concrete model per role via+ `AISOC_MODEL_PIN_<ROLE>` (escape hatch; the slim demo does this so keyed demos+ keep working); with neither, the deterministic offline path is used. Tests:+ `services/agents/tests/test_llm_factory.py` + tightened `test_litellm_config.py`.+ Closes #478.- **v8 P4 — Compounding Memory (verdicts that measurably improve).** New`services/fusion/app/memory/`: a nightly-distillable institutional memory thatmakes verdicts more accurate the longer an instance runs. **Distillation**diff --git a/apps/docs/docs/operations/llm-gateway.md b/apps/docs/docs/operations/llm-gateway.mdindex 53f463656..e2b149e93 100644--- a/apps/docs/docs/operations/llm-gateway.md+++ b/apps/docs/docs/operations/llm-gateway.md@@ -62,9 +62,19 @@ OPENAI_BASE_URL=http://litellm:4000/v1 # send AiSOC's calls to the gateway# OPENAI_API_KEY=${LITELLM_MASTER_KEY} # (in the AiSOC services' environment)```-Leave `OPENAI_BASE_URL` unset to keep calling the provider directly — the-gateway then runs idle (still serving `/health` and `/metrics`) and behaviour is-unchanged.+AiSOC now requests a task **alias** for every live call, so an alias only+resolves when it reaches the gateway. If you don't run the gateway, pin each+role to a concrete provider model instead (the **escape hatch**):++```bash+AISOC_MODEL_PIN_TRIAGE=gpt-4o-mini+AISOC_MODEL_PIN_INVESTIGATION=gpt-4o+# … one per role: triage, recon, investigation, copilot, summary, report, nl+```++With neither the gateway nor pin overrides configured, AiSOC uses its+deterministic offline path. (`OPENAI_MODEL` still applies to the separate+"explain this alert" / BYOK path.)## Re-point a task to a local modeldiff --git a/infra/compose/docker-compose.demo.yml b/infra/compose/docker-compose.demo.ymlindex 0b96fc72a..99c98041b 100644--- a/infra/compose/docker-compose.demo.yml+++ b/infra/compose/docker-compose.demo.yml@@ -191,6 +191,17 @@ services:OPENAI_API_KEY: ${OPENAI_API_KEY:-}ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:-}OPENAI_MODEL: ${OPENAI_MODEL:-gpt-4o-mini}+ # Issue #478 — the slim demo runs without the LiteLLM gateway, so pin every+ # task role to a concrete provider model (AiSOC calls the provider directly+ # with OPENAI_API_KEY; deterministic fallback when no key is set). The full+ # stack (root docker-compose.yml) routes these aliases through the gateway.+ AISOC_MODEL_PIN_TRIAGE: ${OPENAI_MODEL:-gpt-4o-mini}+ AISOC_MODEL_PIN_RECON: ${OPENAI_MODEL:-gpt-4o-mini}+ AISOC_MODEL_PIN_INVESTIGATION: ${OPENAI_MODEL:-gpt-4o-mini}+ AISOC_MODEL_PIN_COPILOT: ${OPENAI_MODEL:-gpt-4o-mini}+ AISOC_MODEL_PIN_SUMMARY: ${OPENAI_MODEL:-gpt-4o-mini}+ AISOC_MODEL_PIN_REPORT: ${OPENAI_MODEL:-gpt-4o-mini}+ AISOC_MODEL_PIN_NL: ${OPENAI_MODEL:-gpt-4o-mini}ENV: developmentnetworks:- aisoc-demodiff --git a/services/agents/app/agents/auto_triage_agent.py b/services/agents/app/agents/auto_triage_agent.pyindex 8647ef440..9081bb7a1 100644--- a/services/agents/app/agents/auto_triage_agent.py+++ b/services/agents/app/agents/auto_triage_agent.py@@ -19,10 +19,10 @@ from typing import Anyimport structlogfrom langchain_core.messages import HumanMessage, SystemMessage-from langchain_openai import ChatOpenAIfrom app.investigator.prompt_sanitizer import sanitize_text, wrap_untrustedfrom app.llm import safe_ainvoke+from app.llm.factory import make_chat_modelfrom app.models.state import AgentStatus, InvestigationStatefrom app.prompt_serialization import format_extra_fields_for_llm@@ -176,8 +176,7 @@ async def run_auto_triage(state: InvestigationState) -> InvestigationState:alert_context = _build_alert_context(state)- model_name = os.getenv("AISOC_LLM_MODEL", "gpt-4o-mini")- llm = ChatOpenAI(model=model_name, temperature=0.0, max_tokens=512)+ llm = make_chat_model("triage", temperature=0.0, max_tokens=512)t0 = time.monotonic()try:diff --git a/services/agents/app/agents/cloud_agent.py b/services/agents/app/agents/cloud_agent.pyindex 01b72f14d..7100044d3 100644--- a/services/agents/app/agents/cloud_agent.py+++ b/services/agents/app/agents/cloud_agent.py@@ -10,17 +10,16 @@ a confidence-weighted verdict.from __future__ import annotationsimport json-import osimport timefrom typing import Anyimport structlogfrom langchain_core.messages import HumanMessage, SystemMessage-from langchain_openai import ChatOpenAIfrom app.context import ContextBundlefrom app.investigator.prompt_sanitizer import sanitize_text, wrap_untrustedfrom app.llm import safe_ainvoke+from app.llm.factory import make_chat_modelfrom app.models.state import AgentStatus, InvestigationStatefrom app.prompt_serialization import format_extra_fields_for_llm, summarize_structure_for_llm@@ -200,8 +199,7 @@ async def run_cloud(bundle_lines = bundle.prompt_context_lines() if bundle is not None else []prompt_context = base_context + (("\n" + "\n".join(bundle_lines)) if bundle_lines else "")- model_name = os.getenv("AISOC_LLM_MODEL", "gpt-4o-mini")- llm = ChatOpenAI(model=model_name, temperature=0.0, max_tokens=768)+ llm = make_chat_model("investigation", temperature=0.0, max_tokens=768)t0 = time.monotonic()try:diff --git a/services/agents/app/agents/identity_agent.py b/services/agents/app/agents/identity_agent.pyindex cb43122ef..7fe0128e5 100644--- a/services/agents/app/agents/identity_agent.py+++ b/services/agents/app/agents/identity_agent.py@@ -9,17 +9,16 @@ to classify the alert and assign a confidence-weighted verdict.from __future__ import annotationsimport json-import osimport timefrom typing import Anyimport structlogfrom langchain_core.messages import HumanMessage, SystemMessage-from langchain_openai import ChatOpenAIfrom app.context import ContextBundlefrom app.investigator.prompt_sanitizer import sanitize_text, wrap_untrustedfrom app.llm import safe_ainvoke+from app.llm.factory import make_chat_modelfrom app.models.state import AgentStatus, InvestigationStatefrom app.prompt_serialization import format_extra_fields_for_llm, summarize_structure_for_llm@@ -195,8 +194,7 @@ async def run_identity(bundle_lines = bundle.prompt_context_lines() if bundle is not None else []prompt_context = base_context + (("\n" + "\n".join(bundle_lines)) if bundle_lines else "")- model_name = os.getenv("AISOC_LLM_MODEL", "gpt-4o-mini")- llm = ChatOpenAI(model=model_name, temperature=0.0, max_tokens=768)+ llm = make_chat_model("investigation", temperature=0.0, max_tokens=768)t0 = time.monotonic()try:diff --git a/services/agents/app/agents/insider_threat_agent.py b/services/agents/app/agents/insider_threat_agent.pyindex 340136a2d..42d2addda 100644--- a/services/agents/app/agents/insider_threat_agent.py+++ b/services/agents/app/agents/insider_threat_agent.py@@ -10,17 +10,16 @@ assign a confidence-weighted verdict.from __future__ import annotationsimport json-import osimport timefrom typing import Anyimport structlogfrom langchain_core.messages import HumanMessage, SystemMessage-from langchain_openai import ChatOpenAIfrom app.context import ContextBundlefrom app.investigator.prompt_sanitizer import sanitize_text, wrap_untrustedfrom app.llm import safe_ainvoke+from app.llm.factory import make_chat_modelfrom app.models.state import AgentStatus, InvestigationStatefrom app.prompt_serialization import format_extra_fields_for_llm, summarize_structure_for_llm@@ -218,8 +217,7 @@ async def run_insider_threat(bundle_lines = bundle.prompt_context_lines() if bundle is not None else []prompt_context = base_context + (("\n" + "\n".join(bundle_lines)) if bundle_lines else "")- model_name = os.getenv("AISOC_LLM_MODEL", "gpt-4o-mini")- llm = ChatOpenAI(model=model_name, temperature=0.0, max_tokens=768)+ llm = make_chat_model("investigation", temperature=0.0, max_tokens=768)t0 = time.monotonic()try:diff --git a/services/agents/app/agents/phishing_agent.py b/services/agents/app/agents/phishing_agent.pyindex e03f9f70f..8c1e2678a 100644--- a/services/agents/app/agents/phishing_agent.py+++ b/services/agents/app/agents/phishing_agent.py@@ -10,17 +10,16 @@ indicators and sets a verdict with calibrated confidence.from __future__ import annotationsimport json-import osimport timefrom typing import Anyimport structlogfrom langchain_core.messages import HumanMessage, SystemMessage-from langchain_openai import ChatOpenAIfrom app.context import ContextBundlefrom app.investigator.prompt_sanitizer import sanitize_text, wrap_untrustedfrom app.llm import safe_ainvoke+from app.llm.factory import make_chat_modelfrom app.models.state import AgentStatus, InvestigationStatefrom app.prompt_serialization import format_extra_fields_for_llm, summarize_structure_for_llm@@ -187,8 +186,7 @@ async def run_phishing(bundle_lines = bundle.prompt_context_lines() if bundle is not None else []prompt_context = base_context + (("\n" + "\n".join(bundle_lines)) if bundle_lines else "")- model_name = os.getenv("AISOC_LLM_MODEL", "gpt-4o-mini")- llm = ChatOpenAI(model=model_name, temperature=0.0, max_tokens=768)+ llm = make_chat_model("investigation", temperature=0.0, max_tokens=768)t0 = time.monotonic()try:diff --git a/services/agents/app/api/contextual.py b/services/agents/app/api/contextual.pyindex 2d3a271aa..cc0cbdbbc 100644--- a/services/agents/app/api/contextual.py+++ b/services/agents/app/api/contextual.py@@ -46,6 +46,7 @@ from fastapi.responses import StreamingResponsefrom pydantic import BaseModel, Fieldfrom app.llm import safe_ainvoke, safe_astream+from app.llm.factory import resolve_model_aliasfrom app.prompt_serialization import summarize_structure_for_llmlogger = structlog.get_logger()@@ -418,10 +419,10 @@ def _fallback_response(system: str, user: str) -> str:"You're seeing a deterministic placeholder response.\n\n""**To enable real contextual answers:**\n\n""1. Set `OPENAI_API_KEY` in your `.env` file\n"- "2. Optionally set `OPENAI_MODEL` (default: `gpt-4o-mini`)\n"+ "2. Point AiSOC at the LiteLLM gateway (`OPENAI_BASE_URL=http://litellm:4000/v1`), "+ "which maps each task alias to a real model\n""3. Restart the `aisoc-agents` service\n\n"- "Refer to [the docs](https://example.com/docs/copilot) for self-hosted "- "model alternatives (Ollama, vLLM, Together)."+ "See the LLM-gateway docs for self-hosted model alternatives (Ollama, vLLM, Together).")@@ -445,7 +446,7 @@ async def list_actions() -> ContextualActionsCatalogue:async def run_action(req: ContextualActionRequest) -> ContextualActionResponse:started = time.monotonic()system, user = _build_messages(req)- model = os.getenv("OPENAI_MODEL", "gpt-4o-mini")+ model = resolve_model_alias("copilot")fallback = not bool(os.getenv("OPENAI_API_KEY"))try:@@ -503,7 +504,7 @@ async def run_action(req: ContextualActionRequest) -> ContextualActionResponse:)async def run_action_stream(req: ContextualActionRequest) -> StreamingResponse:system, user = _build_messages(req)- model = os.getenv("OPENAI_MODEL", "gpt-4o-mini")+ model = resolve_model_alias("copilot")title = _TITLES.get((req.page, req.action), f"{req.page} · {req.action}")suggestions = _FOLLOW_UPS.get((req.page, req.action), [])fallback = not bool(os.getenv("OPENAI_API_KEY"))diff --git a/services/agents/app/api/copilot.py b/services/agents/app/api/copilot.pyindex ec39ea4df..0a13a46d2 100644--- a/services/agents/app/api/copilot.py+++ b/services/agents/app/api/copilot.py@@ -122,6 +122,7 @@ async def _get_openai_reply(try:from app.llm.contract import safe_chat_completions_request+ from app.llm.factory import chat_completions_url, resolve_model_aliasmessages: list[dict[str, str]] = [{@@ -140,8 +141,9 @@ async def _get_openai_reply(body = await safe_chat_completions_request(api_key=api_key,- model="gpt-4o-mini",+ model=resolve_model_alias("copilot"),messages=messages,+ url=chat_completions_url(),max_tokens=512,)return body["choices"][0]["message"]["content"]diff --git a/services/agents/app/investigator/forensic_agent.py b/services/agents/app/investigator/forensic_agent.pyindex fc2f8d548..c5c94b9a5 100644--- a/services/agents/app/investigator/forensic_agent.py+++ b/services/agents/app/investigator/forensic_agent.py@@ -17,10 +17,10 @@ from typing import Anyimport structlogfrom langchain_core.messages import HumanMessage, SystemMessage-from langchain_openai import ChatOpenAIfrom app.core.cost_telemetry import record_llm_callfrom app.llm import safe_ainvoke+from app.llm.factory import make_chat_model, resolve_model_aliasfrom app.prompt_serialization import summarize_structure_for_llmfrom .bundle_prompt import format_bundle_prompt_append@@ -54,10 +54,8 @@ Respond ONLY with a JSON object:async def _llm_forensic(state: InvestigatorState) -> dict[str, Any]:- import os-- model = os.getenv("OPENAI_MODEL", "gpt-4o-mini")- llm = ChatOpenAI(model=model, temperature=0)+ model = resolve_model_alias("investigation")+ llm = make_chat_model("investigation", temperature=0)# Defence-in-depth: alert_summary, recon.summary, and the enrichment cache# can all carry attacker-controlled strings (banners, dark-web excerpts,diff --git a/services/agents/app/investigator/recon_agent.py b/services/agents/app/investigator/recon_agent.pyindex 309fb2740..832a13673 100644--- a/services/agents/app/investigator/recon_agent.py+++ b/services/agents/app/investigator/recon_agent.py@@ -16,10 +16,10 @@ from typing import Anyimport structlogfrom langchain_core.messages import HumanMessage, SystemMessage-from langchain_openai import ChatOpenAIfrom app.core.cost_telemetry import record_llm_callfrom app.llm import safe_ainvoke+from app.llm.factory import make_chat_model, resolve_model_aliasfrom app.prompt_serialization import summarize_structure_for_llmfrom .bundle_prompt import format_bundle_prompt_append@@ -54,10 +54,9 @@ async def _llm_recon(state: InvestigatorState) -> dict[str, Any]:reasoning trace is replayable."""import json- import os- model = os.getenv("OPENAI_MODEL", "gpt-4o-mini")- llm = ChatOpenAI(model=model, temperature=0)+ model = resolve_model_alias("recon")+ llm = make_chat_model("recon", temperature=0)# Defence-in-depth: every field surfaced here can be attacker-influenced# (alert_summary often echoes log lines; raw_alert is verbatim event data).diff --git a/services/agents/app/investigator/report_writer_agent.py b/services/agents/app/investigator/report_writer_agent.pyindex a60dc50d6..6bfb96d85 100644--- a/services/agents/app/investigator/report_writer_agent.py+++ b/services/agents/app/investigator/report_writer_agent.py@@ -15,10 +15,10 @@ from typing import Anyimport structlogfrom langchain_core.messages import HumanMessage, SystemMessage-from langchain_openai import ChatOpenAIfrom app.core.cost_telemetry import record_llm_callfrom app.llm import safe_ainvoke+from app.llm.factory import make_chat_model, resolve_model_aliasfrom app.prompt_serialization import summarize_structure_for_llmfrom .bundle_prompt import format_bundle_prompt_append@@ -158,15 +158,13 @@ def _md_to_html(md: str, case_id: str) -> str:async def run_report_writer(state_dict: dict[str, Any]) -> dict[str, Any]:"""LangGraph node."""- import os-state = InvestigatorState.from_dict(state_dict)t0 = time.monotonic()logger.info("report_writer.start", case_id=state.case_id)- model = os.getenv("OPENAI_MODEL", "gpt-4o-mini")- llm = ChatOpenAI(model=model, temperature=0)+ model = resolve_model_alias("report")+ llm = make_chat_model("report", temperature=0)context = _build_context(state)bundle_append = format_bundle_prompt_append(state.context_bundle)diff --git a/services/agents/app/investigator/responder_agent.py b/services/agents/app/investigator/responder_agent.pyindex 015862522..2ed9e4002 100644--- a/services/agents/app/investigator/responder_agent.py+++ b/services/agents/app/investigator/responder_agent.py@@ -16,10 +16,10 @@ from typing import Anyimport structlogfrom langchain_core.messages import HumanMessage, SystemMessage-from langchain_openai import ChatOpenAIfrom app.core.cost_telemetry import record_llm_callfrom app.llm import safe_ainvoke+from app.llm.factory import make_chat_model, resolve_model_aliasfrom app.prompt_serialization import summarize_structure_for_llmfrom .bundle_prompt import format_bundle_prompt_append@@ -52,10 +52,8 @@ Respond ONLY with a JSON object:async def _llm_responder(state: InvestigatorState) -> dict[str, Any]:- import os-- model = os.getenv("OPENAI_MODEL", "gpt-4o-mini")- llm = ChatOpenAI(model=model, temperature=0)+ model = resolve_model_alias("investigation")+ llm = make_chat_model("investigation", temperature=0)# Defence-in-depth: every field surfaced here originated in attacker-# influenced data (alert payloads, banners, dark-web excerpts, LLMdiff --git a/services/agents/app/llm/factory.py b/services/agents/app/llm/factory.pynew file mode 100644index 000000000..b68f04d4d--- /dev/null+++ b/services/agents/app/llm/factory.py@@ -0,0 +1,87 @@+"""LLM factory — turn a task role into a ready chat model / model alias (#478).++Every live LLM call in ``services/agents`` asks for a **logical task alias**+(``aisoc-triage``, ``aisoc-recon``, …) rather than a concrete model. The LiteLLM+gateway (``infra/litellm/config.yaml``) owns the alias → real-model mapping and+any model-level fallback, so operators re-point models without touching code.+This module is the single place that resolves a role to ``(alias, base_url)`` and+hands back a configured chat model.++Since #478 there is **no hardcoded default model**. The alias comes from+:func:`app.llm.model_pins.get_pin`, which is env-overridable per role+(``AISOC_MODEL_PIN_<ROLE>``) so a deployment that calls a provider directly+instead of through the gateway can pin a concrete model. When no live LLM is+reachable, callers fall back to their deterministic path exactly as before — the+factory never forces a live call.++Routing to the gateway is deliberate: set ``OPENAI_BASE_URL`` (or ``LLM_BASE_URL``)+to the gateway and send ``OPENAI_API_KEY=$LITELLM_MASTER_KEY``. Left unset, the+client talks to its provider default — where an alias only resolves if the+operator has pinned a concrete model via ``AISOC_MODEL_PIN_<ROLE>``.+"""++from __future__ import annotations++import os+from typing import Any++from langchain_openai import ChatOpenAI++from app.llm.contract import DEFAULT_OPENAI_CHAT_COMPLETIONS_URL+from app.llm.model_pins import get_pin+++def resolve_model_alias(role: str) -> str:+ """Return the logical model alias AiSOC sends for a task ``role``.++ ``aisoc-<role>`` by default; ``AISOC_MODEL_PIN_<ROLE>`` overrides it with a+ concrete provider model for deployments that bypass the gateway.+ """+ return get_pin(role).primary_model+++def resolve_base_url() -> str | None:+ """OpenAI-compatible base URL for live calls, or ``None`` for the client default.++ Honours an explicit ``OPENAI_BASE_URL`` / ``LLM_BASE_URL`` — what an operator+ sets to point AiSOC at the LiteLLM gateway. ``None`` means "use the client's+ provider default" (the direct-to-provider path). We intentionally do **not**+ auto-adopt the compose-provided ``LLM_GATEWAY_URL`` here: routing through the+ gateway is an explicit choice so the bearer token (the gateway master key vs.+ a provider key) is never ambiguous.+ """+ return os.getenv("OPENAI_BASE_URL", "").strip() or os.getenv("LLM_BASE_URL", "").strip() or None+++def chat_completions_url() -> str:+ """Full chat-completions URL for the raw-HTTP path (:func:`safe_chat_completions_request`)."""+ base = resolve_base_url()+ if base:+ return base.rstrip("/") + "/chat/completions"+ return DEFAULT_OPENAI_CHAT_COMPLETIONS_URL+++def make_chat_model(+ role: str,+ *,+ temperature: float = 0.0,+ max_tokens: int | None = None,+ **kwargs: Any,+) -> ChatOpenAI:+ """Build a :class:`ChatOpenAI` for a task ``role`` (alias + gateway base URL).++ The returned model is **not** contract-guarded — every caller still routes the+ invocation through :func:`app.llm.safe_ainvoke`, which enforces the input+ contract. Extra ``kwargs`` pass straight through to ``ChatOpenAI``.+ """+ params: dict[str, Any] = {+ "model": resolve_model_alias(role),+ "temperature": temperature,+ }+ if max_tokens is not None:+ params["max_tokens"] = max_tokens+ base_url = resolve_base_url()+ if base_url:+ params["base_url"] = base_url+ params.update(kwargs)+ return ChatOpenAI(**params)diff --git a/services/agents/app/llm/model_pins.py b/services/agents/app/llm/model_pins.pyindex 93df06999..40871bd4e 100644--- a/services/agents/app/llm/model_pins.py+++ b/services/agents/app/llm/model_pins.py@@ -36,13 +36,24 @@ class ModelPin:return chain-# Shipped pins. Model IDs are the pinned defaults; override via env-# (AISOC_MODEL_PIN_<ROLE>, e.g. AISOC_MODEL_PIN_TRIAGE). Every chain terminates-# in `deterministic`.+# Shipped pins. Since #478 the pinned primary is a **logical gateway alias**+# (`aisoc-<role>`), not a concrete model — the LiteLLM gateway+# (`infra/litellm/config.yaml`) owns the alias → real-model mapping and any+# model-level fallback, so each AiSOC-side chain only needs its `deterministic`+# floor. One role per AiSOC LLM workload named in #478.+#+# Escape hatch for deployments not running the gateway: override any primary via+# env (`AISOC_MODEL_PIN_<ROLE>`, e.g. `AISOC_MODEL_PIN_TRIAGE=gpt-4o-mini`) to+# send a concrete provider model directly. Every chain still terminates in+# `deterministic`._DEFAULT_PINS: dict[str, ModelPin] = {- "triage": ModelPin("triage", "gpt-4o-mini", ["gpt-4o", DETERMINISTIC]),- "recon": ModelPin("recon", "gpt-4o-mini", ["gpt-4o", DETERMINISTIC]),- "summary": ModelPin("summary", "gpt-4o-mini", [DETERMINISTIC]),+ "triage": ModelPin("triage", "aisoc-triage", [DETERMINISTIC]),+ "recon": ModelPin("recon", "aisoc-recon", [DETERMINISTIC]),+ "investigation": ModelPin("investigation", "aisoc-investigation", [DETERMINISTIC]),+ "copilot": ModelPin("copilot", "aisoc-copilot", [DETERMINISTIC]),+ "summary": ModelPin("summary", "aisoc-summary", [DETERMINISTIC]),+ "report": ModelPin("report", "aisoc-report", [DETERMINISTIC]),+ "nl": ModelPin("nl", "aisoc-nl", [DETERMINISTIC]),}diff --git a/services/agents/app/nl_query/translator.py b/services/agents/app/nl_query/translator.pyindex 24f4a3c9f..19eb8006e 100644--- a/services/agents/app/nl_query/translator.py+++ b/services/agents/app/nl_query/translator.py@@ -885,7 +885,7 @@ async def enhance_with_llm(query: NLQuery,*,api_key: str,- model: str = "gpt-4o-mini",+ model: str | None = None,timeout: float = 30.0,fallback: TranslatedQuery | None = None,) -> TranslatedQuery:@@ -903,6 +903,7 @@ async def enhance_with_llm(import textwrapfrom app.llm.contract import safe_chat_completions_request+ from app.llm.factory import chat_completions_url, resolve_model_aliasprompt = textwrap.dedent(f"""@@ -922,8 +923,9 @@ async def enhance_with_llm(payload = await safe_chat_completions_request(api_key=api_key,- model=model,+ model=model or resolve_model_alias("nl"),messages=messages,+ url=chat_completions_url(),timeout=timeout,response_format={"type": "json_object"},temperature=0,diff --git a/services/agents/tests/test_litellm_config.py b/services/agents/tests/test_litellm_config.pyindex 85a03cc5c..051607f7f 100644--- a/services/agents/tests/test_litellm_config.py+++ b/services/agents/tests/test_litellm_config.py@@ -79,9 +79,7 @@ def test_master_key_comes_from_env():assert str(master_key).startswith("os.environ/"), "master_key must be an env reference, not a committed secret"-def test_model_pins_roles_have_a_gateway_alias():- # Ties the code's logical roles to the gateway. Subset today (model_pins- # ships triage/recon/summary); becomes an equality guarantee once PR2 pins- # all seven roles.- for role in all_roles():- assert f"aisoc-{role}" in EXPECTED_ALIASES, f"role '{role}' has no aisoc-{role} gateway alias"+def test_model_pins_roles_match_gateway_aliases_exactly():+ # Ties the code's logical roles to the gateway config: every pinned role has+ # an ``aisoc-<role>`` alias in the config, and vice versa — no drift either way.+ assert {f"aisoc-{role}" for role in all_roles()} == EXPECTED_ALIASESdiff --git a/services/agents/tests/test_llm_factory.py b/services/agents/tests/test_llm_factory.pynew file mode 100644index 000000000..bed8d2783--- /dev/null+++ b/services/agents/tests/test_llm_factory.py@@ -0,0 +1,88 @@+"""Tests for the LLM factory — alias + base-URL resolution (#478)."""++from __future__ import annotations++import pytest+from app.llm.factory import (+ chat_completions_url,+ make_chat_model,+ resolve_base_url,+ resolve_model_alias,+)++# app.llm.contract.DEFAULT_OPENAI_CHAT_COMPLETIONS_URL+_DEFAULT_URL = "https://api.openai.com/v1/chat/completions"++[email protected](autouse=True)+def _clean_llm_env(monkeypatch):+ """Start each test from a known-empty LLM env."""+ for var in (+ "OPENAI_BASE_URL",+ "LLM_BASE_URL",+ "LLM_GATEWAY_URL",+ "AISOC_MODEL_PIN_TRIAGE",+ "AISOC_MODEL_PIN_NL",+ ):+ monkeypatch.delenv(var, raising=False)+++# ── resolve_model_alias ──────────────────────────────────────────────────────+++def test_alias_defaults_to_aisoc_role():+ assert resolve_model_alias("triage") == "aisoc-triage"+ assert resolve_model_alias("investigation") == "aisoc-investigation"+ assert resolve_model_alias("nl") == "aisoc-nl"+++def test_alias_env_override_escape_hatch(monkeypatch):+ monkeypatch.setenv("AISOC_MODEL_PIN_TRIAGE", "gpt-4o-mini")+ assert resolve_model_alias("triage") == "gpt-4o-mini"+ # Other roles are unaffected by a single-role override.+ assert resolve_model_alias("nl") == "aisoc-nl"+++# ── resolve_base_url ─────────────────────────────────────────────────────────+++def test_base_url_none_by_default():+ assert resolve_base_url() is None+++def test_base_url_prefers_openai_then_llm(monkeypatch):+ monkeypatch.setenv("LLM_BASE_URL", "http://llm:9/v1")+ assert resolve_base_url() == "http://llm:9/v1"+ monkeypatch.setenv("OPENAI_BASE_URL", "http://gw:4000/v1")+ assert resolve_base_url() == "http://gw:4000/v1"+++def test_base_url_ignores_gateway_url_env(monkeypatch):+ # Routing through the gateway must be explicit (OPENAI_BASE_URL), never+ # silently adopted from the compose-provided LLM_GATEWAY_URL.+ monkeypatch.setenv("LLM_GATEWAY_URL", "http://litellm:4000/v1")+ assert resolve_base_url() is None+++# ── chat_completions_url ─────────────────────────────────────────────────────+++def test_chat_completions_url_default():+ assert chat_completions_url() == _DEFAULT_URL+++def test_chat_completions_url_appends_path(monkeypatch):+ monkeypatch.setenv("OPENAI_BASE_URL", "http://litellm:4000/v1/")+ assert chat_completions_url() == "http://litellm:4000/v1/chat/completions"+++# ── make_chat_model ──────────────────────────────────────────────────────────+++def test_make_chat_model_uses_alias_and_base_url(monkeypatch):+ monkeypatch.setenv("OPENAI_API_KEY", "sk-test")+ monkeypatch.setenv("OPENAI_BASE_URL", "http://litellm:4000/v1")+ llm = make_chat_model("triage", temperature=0.0, max_tokens=256)+ # langchain_openai exposes the model id as ``model_name``.+ assert llm.model_name == "aisoc-triage"+ assert str(llm.openai_api_base).rstrip("/") == "http://litellm:4000/v1"diff --git a/services/api/app/_vendor/nl_query/translator.py b/services/api/app/_vendor/nl_query/translator.pyindex 24f4a3c9f..19eb8006e 100644--- a/services/api/app/_vendor/nl_query/translator.py+++ b/services/api/app/_vendor/nl_query/translator.py@@ -885,7 +885,7 @@ async def enhance_with_llm(query: NLQuery,*,api_key: str,- model: str = "gpt-4o-mini",+ model: str | None = None,timeout: float = 30.0,fallback: TranslatedQuery | None = None,) -> TranslatedQuery:@@ -903,6 +903,7 @@ async def enhance_with_llm(import textwrapfrom app.llm.contract import safe_chat_completions_request+ from app.llm.factory import chat_completions_url, resolve_model_aliasprompt = textwrap.dedent(f"""@@ -922,8 +923,9 @@ async def enhance_with_llm(payload = await safe_chat_completions_request(api_key=api_key,- model=model,+ model=model or resolve_model_alias("nl"),messages=messages,+ url=chat_completions_url(),timeout=timeout,response_format={"type": "json_object"},temperature=0,diff --git a/services/api/app/api/v1/endpoints/hunts.py b/services/api/app/api/v1/endpoints/hunts.pyindex 702ca5cbf..b1e19e7b2 100644--- a/services/api/app/api/v1/endpoints/hunts.py+++ b/services/api/app/api/v1/endpoints/hunts.py@@ -38,6 +38,7 @@ from sqlalchemy import textfrom app.api.v1.deps import AuthUser, DBSessionfrom app.core.airgap import AirgapViolation, enforce_airgap_for_url+from app.services.model_aliases import resolve_model_aliaslogger = logging.getLogger(__name__)@@ -139,7 +140,7 @@ async def _generate_queries(hypothesis: str, mitre: str | None) -> dict[str, strif not api_key:return Nonebase_url = os.getenv("LLM_BASE_URL", "https://api.openai.com/v1")- model = os.getenv("LLM_MODEL", "gpt-4o-mini")+ model = os.getenv("LLM_MODEL") or resolve_model_alias("nl")user_msg = f"HYPOTHESIS: {hypothesis}"if mitre:user_msg += f"\nMITRE TECHNIQUE: {mitre}"diff --git a/services/api/app/api/v1/endpoints/knowledge_base.py b/services/api/app/api/v1/endpoints/knowledge_base.pyindex 1f1db2f62..c1c5a5001 100644--- a/services/api/app/api/v1/endpoints/knowledge_base.py+++ b/services/api/app/api/v1/endpoints/knowledge_base.py@@ -29,6 +29,7 @@ from sqlalchemy import textfrom app.api.v1.deps import AuthUser, DBSessionfrom app.core.airgap import AirgapViolation, enforce_airgap_for_urlfrom app.services.kb_chunking import chunk_text+from app.services.model_aliases import resolve_model_aliaslogger = logging.getLogger(__name__)@@ -115,7 +116,7 @@ async def _synthesise(question: str, chunks: list[KBChunk]) -> str | None:if not api_key:return Nonebase_url = os.getenv("LLM_BASE_URL", "https://api.openai.com/v1")- model = os.getenv("LLM_MODEL", "gpt-4o-mini")+ model = os.getenv("LLM_MODEL") or resolve_model_alias("nl")context = "\n\n".join(f"[{c.title}] chunk {c.chunk_index}:\n{c.content}" for c in chunks)completions_url = f"{base_url}/chat/completions"enforce_airgap_for_url(completions_url)diff --git a/services/api/app/api/v1/endpoints/phishing.py b/services/api/app/api/v1/endpoints/phishing.pyindex e54310228..4705c39f1 100644--- a/services/api/app/api/v1/endpoints/phishing.py+++ b/services/api/app/api/v1/endpoints/phishing.py@@ -28,6 +28,7 @@ from sqlalchemy import textfrom app.api.v1.deps import AuthUser, DBSessionfrom app.core.airgap import AirgapViolation, enforce_airgap_for_url+from app.services.model_aliases import resolve_model_aliaslogger = logging.getLogger(__name__)@@ -92,7 +93,7 @@ async def _triage(artifact_kind: str, content: str, urls: list[str]) -> TriageReif not api_key:return Nonebase_url = os.getenv("LLM_BASE_URL", "https://api.openai.com/v1")- model = os.getenv("LLM_MODEL", "gpt-4o-mini")+ model = os.getenv("LLM_MODEL") or resolve_model_alias("investigation")user_msg = f"ARTIFACT TYPE: {artifact_kind}\n"if urls:user_msg += f"URLS: {', '.join(urls[:10])}\n"diff --git a/services/api/app/api/v1/endpoints/translation.py b/services/api/app/api/v1/endpoints/translation.pyindex 588f16a41..066b7b2ff 100644--- a/services/api/app/api/v1/endpoints/translation.py+++ b/services/api/app/api/v1/endpoints/translation.py@@ -22,6 +22,7 @@ from fastapi import APIRouter, HTTPException, statusfrom pydantic import BaseModel, Fieldfrom app.core.airgap import AirgapViolation, enforce_airgap_for_url+from app.services.model_aliases import resolve_model_aliasrouter = APIRouter(prefix="/translation", tags=["translation"])@@ -111,7 +112,7 @@ async def _llm_translate(req: TranslateRequest) -> dict[str, Any] | None:if not api_key:return Nonebase_url = os.getenv("LLM_BASE_URL", "https://api.openai.com/v1")- model = os.getenv("LLM_MODEL", "gpt-4o-mini")+ model = os.getenv("LLM_MODEL") or resolve_model_alias("nl")completions_url = f"{base_url}/chat/completions"# Air-gap check: refuses the call entirely (rather than silently# falling back to template-substitution) so misconfigurations arediff --git a/services/api/app/services/model_aliases.py b/services/api/app/services/model_aliases.pynew file mode 100644index 000000000..94983cde8--- /dev/null+++ b/services/api/app/services/model_aliases.py@@ -0,0 +1,26 @@+"""Task role → LiteLLM gateway alias for the API service (#478).++The API service is a separate package from ``services/agents``, so it can't+import ``app.llm.factory``. This is the small mirror of+:func:`services.agents.app.llm.factory.resolve_model_alias`: each LLM-backed API+endpoint (translation, hunts, knowledge base, phishing) asks for a **logical+alias**; the LiteLLM gateway (``infra/litellm/config.yaml``) owns the alias →+real-model mapping. There is no hardcoded default model.++Escape hatch for deployments not running the gateway: override any role with a+concrete provider model via ``AISOC_MODEL_PIN_<ROLE>`` (kept in lockstep with the+agents-side pins), or set a global ``LLM_MODEL`` at the call site.+"""++from __future__ import annotations++import os++# Mirrors the roles in services/agents/app/llm/model_pins.py.+ROLES = frozenset({"triage", "recon", "investigation", "copilot", "summary", "report", "nl"})+++def resolve_model_alias(role: str) -> str:+ """Return the ``aisoc-<role>`` alias, honouring an ``AISOC_MODEL_PIN_<ROLE>`` override."""+ override = os.environ.get(f"AISOC_MODEL_PIN_{role.upper()}", "").strip()+ return override or f"aisoc-{role}"