Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
AI triage could not reach a model, and the variable that would have fixed it was read by nothing (#829)
Beenu Aroraad775e8df38f
diff --git a/.env.example b/.env.exampleindex b881e5d1d..f612be8a6 100644--- a/.env.example+++ b/.env.example@@ -105,10 +105,20 @@ QDRANT_URL=http://localhost:6333# ─── AI / LLM ─────────────────────────────────────────────────────────────────OPENAI_API_KEY=sk-your-openai-api-key-here-# OPENAI_MODEL now only affects the "explain this alert" / BYOK path. Task-# workloads (triage, recon, investigation, copilot, summary, report, NL) request-# a logical alias instead — see the LLM gateway block below.-OPENAI_MODEL=gpt-4-turbo-preview+# OPENAI_MODEL affects the "explain this alert" / BYOK path only. Task workloads+# (triage, recon, investigation, copilot, summary, report, NL) request a logical+# alias instead — see the LLM gateway block below.+#+# It ships as an alias, not a concrete model, because the default deployment+# routes through the bundled gateway and the gateway 400s anything+# infra/litellm/config.yaml does not define. It used to read+# `gpt-4-turbo-preview`, which no gateway config here has ever defined, and the+# auto-triage worker treated it as a per-tenant BYOK override — so it replaced+# `aisoc-triage` on the highest-volume path in the product.+# scripts/check_llm_model_routing.py fails the build if this names a model the+# gateway does not define. Set a concrete model only when calling a provider+# directly (see "ENABLE LIVE LLM" option (b) below).+OPENAI_MODEL=aisoc-summary# ─── LLM gateway (LiteLLM) — issue #478 ───────────────────────────────────────# Since #478 AiSOC asks for a logical task alias (aisoc-triage,@@ -119,19 +129,30 @@ OPENAI_MODEL=gpt-4-turbo-preview## Key that AiSOC uses to authenticate to the gateway (also LiteLLM's admin key):LITELLM_MASTER_KEY=sk-aisoc-local-# In-network URL of the gateway (already set per-service in docker-compose.yml):+# In-network URL of the gateway. docker-compose.yml sets this on the api and+# agents services, and both resolvers READ it — `resolve_base_url()` returns it+# for any aisoc-<role> alias, and `resolve_api_key()` returns LITELLM_MASTER_KEY+# to pair with it. That wire was missing: compose set this variable, the+# resolvers deliberately ignored it in favour of OPENAI_BASE_URL (which nothing+# set), and so the product's headline capability could not reach a model in the+# default deployment.LLM_GATEWAY_URL=http://litellm:4000/v1#-# ENABLE LIVE LLM — pick one:-# (a) Route through the gateway (recommended). Put your real provider key in-# OPENAI_API_KEY above (LiteLLM reads it), then uncomment:-# OPENAI_BASE_URL=http://litellm:4000/v1-# OPENAI_API_KEY=${LITELLM_MASTER_KEY}-# (b) Call a provider directly without the gateway — pin each task role to a-# concrete model (escape hatch), e.g.:+# ENABLE LIVE LLM:+# (a) Through the gateway — the default, and already wired. Put your real+# provider key in OPENAI_API_KEY above (the gateway reads it) and bring up+# the `full` profile, which is where the litellm service lives:+# docker compose --profile full up -d+# Nothing else to set. To point at a gateway somewhere else, or to send a+# different bearer, set OPENAI_BASE_URL/OPENAI_API_KEY and they win.+# (b) Direct to a provider, no gateway — pin each task role to a concrete+# model. A concrete pin is NOT rerouted to the gateway, so this keeps+# working with LLM_GATEWAY_URL set:# AISOC_MODEL_PIN_TRIAGE=gpt-4o-mini# AISOC_MODEL_PIN_INVESTIGATION=gpt-4o-# With neither configured, AiSOC uses its deterministic offline path.+# OPENAI_MODEL=gpt-4o-mini # the BYOK/explain path, direct too+# With neither configured, AiSOC uses its deterministic offline path — which+# reports zero model calls and zero tokens rather than pretending to triage.# ─── Threat Intelligence ──────────────────────────────────────────────────────# Open-source / community feeds (work out of the box, no key required for some)diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex 1a12343e9..ce9cd480f 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -412,6 +412,22 @@ jobs:run: python3 scripts/check_connector_profiles.py --self-test- name: Connector profile drift gaterun: python3 scripts/check_connector_profiles.py+ # Same question one layer up: a connector name has to reach a profile,+ # and a model name has to reach a gateway that defines it. AI triage+ # could not reach a model in the default deployment because+ # docker-compose.yml set LLM_GATEWAY_URL and both resolvers ignored it,+ # and because .env.example shipped a model no gateway config here has+ # ever defined. Neither is visible in a diff, and both are structural.+ #+ # The self-test runs first and on the same tree: it injects drift in+ # each direction and feeds the parsers prose shaped like the thing they+ # look for — a commented-out alias in the config's Ollama example, a+ # commented-out pin in .env.example's escape-hatch block, a compose+ # comment naming a variable two lines above the key it describes.+ - name: LLM model routing gate — self-test+ run: python3 scripts/check_llm_model_routing.py --self-test+ - name: LLM model routing gate+ run: python3 scripts/check_llm_model_routing.py# The count claim above says how many connectors exist; this says# whether a reader can find out how to use them. 35 of 84 had no page# at all and the sidebar listed 34 of 96, so most connectordiff --git a/CHANGELOG.md b/CHANGELOG.mdindex 4b9c7c49a..92fcb5032 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -75,8 +75,105 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0the breaking-change gate does not see this; it is recorded here because adropped enum value is a break whether or not a workflow notices.+### Fixed++- **AI triage could not reach a model in the default deployment, and it was+ not a missing key.** `docker-compose.yml` set `LLM_GATEWAY_URL` on the `api`+ and `agents` services. Nothing read it. Both resolvers honoured only+ `OPENAI_BASE_URL`, which compose never set, and each carried a comment+ explaining that the refusal was deliberate — routing through the gateway+ should be an explicit choice so it is never ambiguous whether the bearer is+ the gateway master key or a provider key. The reasoning was sound and the+ result was that every `aisoc-<role>` alias went to `api.openai.com`, which+ cannot resolve one, and the caller's `except` rendered that as "no LLM+ available". `preflight_llm()` logged exactly this condition at boot with the+ remedy, so the diagnosis existed and the wiring did not.++ `LLM_GATEWAY_URL` is now the lowest-precedence base URL, and the ambiguity is+ resolved rather than avoided. **The model decides the route:** the gateway is+ adopted only for a gateway alias, which resolves nowhere else, while a+ concrete `AISOC_MODEL_PIN_<ROLE>` — the documented direct-to-provider escape+ hatch — is left pointing at its provider. **The route decides the bearer:**+ when AiSOC picks the gateway itself it sends `LITELLM_MASTER_KEY`, which+ compose now supplies to both services, and never a provider key. An explicit+ `OPENAI_BASE_URL` still outranks both. The rule lives once, in+ `services/agents/app/llm/routing.py`, shared with the BYOK resolver and+ mirrored in `services/api/app/services/model_aliases.py` for the service that+ cannot import it.++ Consumers audited and fixed alongside it: the API copilot and the NL-query+ translator (both already went through the shared helpers); the agents+ **contextual copilot**, which built `ChatOpenAI(model=…)` with no base URL at+ all and reported "LLM not configured" from a key check that could not see the+ gateway's; the **detection builder** (`/nl-detection`), the only consumer that+ could not be pointed anywhere — it POSTed to a hardcoded+ `https://api.openai.com/v1/chat/completions` with `OPENAI_MODEL`; the four API+ endpoints (translation, hunts, knowledge base, phishing) that read+ `LLM_BASE_URL` and defaulted to OpenAI without consulting `OPENAI_BASE_URL`;+ the BYOK / "explain this alert" resolvers in both services, which defaulted+ straight to `https://api.openai.com` and took an alias there; and+ `GET /llm/status`, so the indicator describes the route the explain path+ actually takes. The **MITRE RAG embedding** path is a deliberate exclusion —+ `infra/litellm/config.yaml` declares chat aliases only, so an embedding call+ routed there would 400 on every batch. It has its own+ `AISOC_EMBEDDING_BASE_URL` / `AISOC_EMBEDDING_MODEL` pair and the gate checks+ the exclusion in both directions.++- **`OPENAI_MODEL` replaced the triage role's model on the highest-volume path+ in the product.** `.env.example` shipped `OPENAI_MODEL=gpt-4-turbo-preview`,+ a model no gateway config in this tree has ever defined. The auto-triage+ worker layers a tenant's BYOK configuration over the role pin — but the+ resolver's `LlmConfig` collapsed provenance into a single `source` word, so+ the worker could not tell a per-tenant override from the process-wide env+ baseline and bound all three fields unconditionally. Once traffic reached the+ gateway that would have produced `Invalid model name` on every triage call:+ the failure the commercial deployment already hit, where the copilot and every+ triage agent degraded to empty output.++ `LlmConfig` now carries `model_from_tenant` / `base_url_from_tenant` /+ `api_key_from_tenant`, computed from state the resolvers already tracked, and+ only a field the tenant actually set is treated as an override. A real BYOK+ model still wins. `.env.example` ships `OPENAI_MODEL=aisoc-summary` — an alias+ the bundled gateway defines — and the variable is documented as the BYOK /+ explain path only, never a task role.++- **A model with nowhere to go is now an error instead of a fallback.**+ `make_chat_model` and `chat_completions_url` raise `UnroutableModelError`,+ naming the alias and the remedy, when a gateway alias is requested with no+ gateway configured — rather than building a client destined to 404 and+ letting a caller read that as "the LLM is unavailable". `run_auto_triage`+ constructs its model inside its own `try` so the failure arrives as+ `AutoTriageError` like every other LLM failure; `auto_triage_node` catches+ that specifically, so constructing outside it would have failed a whole graph+ run over a configuration problem the deterministic path handles. The+ deterministic floor is unchanged and still reports `call_count=0 models=[]+ total_tokens=0` with no `recommended_actions`. `preflight_llm()` now reports+ both directions at boot: an alias with no gateway, and a concrete model+ pointed at the bundled gateway.+### Added+- **A model name must resolve, and the variable that routes it must be read —+ gated in both directions.** `scripts/check_llm_model_routing.py` reconciles+ `infra/litellm/config.yaml`, the role pins, the API-side mirror,+ `.env.example` and `docker-compose.yml`: every pin's alias must be defined by+ the gateway and every alias claimed by a pin; every model named in an env or+ compose file that wires the bundled gateway must be one the gateway defines;+ every variable compose points at the gateway must be read by *both* resolvers+ and every gateway variable the resolvers read must be set by compose; and a+ service handed the gateway URL must be handed the gateway key. Wired into+ `ci.yml :: python-lint`, with the self-test running first.++ Every parser is AST- or comment-aware, and the self-test proves each refusal+ rather than asserting it, because all three corpora contain text shaped+ exactly like what the gate looks for: the gateway config's commented+ Ollama/vLLM examples repeat `model_name: aisoc-triage` verbatim,+ `.env.example` documents the escape hatch as a commented+ `AISOC_MODEL_PIN_TRIAGE=gpt-4o-mini`, and `docker-compose.yml` explains the+ gateway in prose containing `OPENAI_BASE_URL=http://litellm:4000/v1` two lines+ above the key it describes. A line regex credits all three. 13 injected+ defects and 7 parser blind spots, each caught by its own code, plus the shared+ empty-tree refusal.- **`GET /api/v1/graph` exists.** The console's Attack Graph view has alwayscalled it and always got a 404, while `/graph/neighbors/…` and`/graph/mitre-coverage` returned 200 from the same Neo4j instance holdingdiff --git a/apps/docs/docs/compliance/evidence-pack.md b/apps/docs/docs/compliance/evidence-pack.mdindex 48e148abf..a44a97c0f 100644--- a/apps/docs/docs/compliance/evidence-pack.md+++ b/apps/docs/docs/compliance/evidence-pack.md@@ -33,7 +33,7 @@ and says so.| Prompt-injection resistance | 32-payload adversarial corpus with a recall floor and a false-positive ceiling | `services/agents/tests/adversarial/` || Claim-to-gate traceability | Every product claim mapped to the CI job that fails when it stops being true | `docs/audit/CLAIM_TO_GATE_MATRIX.md` |-The last row is the one worth reading first. It is the honest index: 129 rows+The last row is the one worth reading first. It is the honest index: 130 rows`GATED`, 8 `PARTIAL` with the specific gap named, and a ratchet that refusesto let a row sit at `NO GATE`.diff --git a/apps/docs/docs/deployment/env-vars.md b/apps/docs/docs/deployment/env-vars.mdindex 6e355f33b..2083c54d7 100644--- a/apps/docs/docs/deployment/env-vars.md+++ b/apps/docs/docs/deployment/env-vars.md@@ -210,8 +210,13 @@ Source: [`services/agents/app/`](https://github.com/beenuar/AiSOC/tree/main/serv| Variable | Default | Description ||----------|---------|-------------|-| `OPENAI_API_KEY` | — | **Required.** OpenAI key used by the investigator and copilot agents |-| `OPENAI_MODEL` | `gpt-4o` | LLM identifier — set per agent if you need different models |+| `OPENAI_API_KEY` | — | Provider key. With the bundled gateway running, the *gateway* uses this to reach the upstream model; AiSOC itself authenticates with `LITELLM_MASTER_KEY`. |+| `LLM_GATEWAY_URL` | `http://litellm:4000/v1` (set by compose) | In-network URL of the bundled LiteLLM gateway. Both resolvers read it for any `aisoc-<role>` alias — an alias resolves nowhere else. A concrete `AISOC_MODEL_PIN_<ROLE>` is left pointing at its provider. |+| `LITELLM_MASTER_KEY` | `sk-aisoc-local` in compose | Bearer AiSOC sends when it routes to the gateway itself. Resolved *with* the URL, so a provider key is never sent to the gateway (it rejects one). |+| `OPENAI_BASE_URL` / `LLM_BASE_URL` | — | Explicit override. Outranks `LLM_GATEWAY_URL` for every role; you own the key pairing when you set it. |+| `AISOC_MODEL_PIN_<ROLE>` | `aisoc-<role>` | Per-role model. Set a concrete provider model to call a provider directly with no gateway. Roles: `triage`, `recon`, `investigation`, `copilot`, `summary`, `report`, `nl`. |+| `OPENAI_MODEL` | `aisoc-summary` | BYOK / "explain this alert" path **only** — never a task role's model. Ships as an alias because the default deployment routes through the gateway, which 400s anything `infra/litellm/config.yaml` does not define. |+| `AISOC_EMBEDDING_BASE_URL` / `AISOC_EMBEDDING_MODEL` | — / `text-embedding-3-large` | MITRE RAG embeddings. Deliberately outside the gateway, whose model list is chat aliases only. || `DATABASE_URL` | `postgresql+asyncpg://aisoc:aisoc@localhost:5432/aisoc` | Postgres DSN for the Investigation Ledger || `QDRANT_URL` | `http://localhost:6333` | Vector store for case memory and RAG || `ENRICHMENT_SERVICE_URL` | `http://enrichment:8011` | URL of the enrichment service (Go) |@@ -449,8 +454,11 @@ AISOC_CORS_ORIGINS=http://localhost:3000JWT_SECRET=$(openssl rand -hex 32)# --- Agents ---+# The gateway reads OPENAI_API_KEY; AiSOC authenticates to the gateway with+# LITELLM_MASTER_KEY. Routing is already wired — compose sets LLM_GATEWAY_URL+# on api and agents, and both resolvers read it.OPENAI_API_KEY=sk-...-OPENAI_MODEL=gpt-4o+LITELLM_MASTER_KEY=$(openssl rand -hex 32)ENRICHMENT_SERVICE_URL=http://enrichment:8011# --- Realtime ---diff --git a/apps/docs/docs/operations/faq.md b/apps/docs/docs/operations/faq.mdindex d9040aca7..368f8d90c 100644--- a/apps/docs/docs/operations/faq.md+++ b/apps/docs/docs/operations/faq.md@@ -101,7 +101,7 @@ Postgres and Kafka are the natural ceiling; both scale predictably under standar### Which model does the agent use?-Whichever you configure. `OPENAI_MODEL` (and per-agent overrides) is read at startup. Default is `gpt-4o`. Anthropic Claude works via the agents service's provider abstraction. Self-hosted Llama / Mistral works via OpenAI-compatible endpoints (vLLM, Ollama, llama.cpp server).+Whichever you configure, and the choice lives in one file. Each workload asks for a logical alias (`aisoc-triage`, `aisoc-investigation`, …) and `infra/litellm/config.yaml` maps aliases to real models — so you re-point a task without touching AiSOC. Hosted providers, Anthropic Claude, and self-hosted Llama / Mistral behind an OpenAI-compatible endpoint (vLLM, Ollama, llama.cpp server) are all just a different line in that file. To skip the gateway entirely, pin a concrete model per role with `AISOC_MODEL_PIN_<ROLE>`. See [LLM gateway](./llm-gateway.md).### Will the agent do things without my permission?diff --git a/apps/docs/docs/operations/llm-gateway.md b/apps/docs/docs/operations/llm-gateway.mdindex e2b149e93..a6dc267be 100644--- a/apps/docs/docs/operations/llm-gateway.md+++ b/apps/docs/docs/operations/llm-gateway.md@@ -51,30 +51,73 @@ point is that you change it. The alias names stay constant.## Enable the gateway-The `litellm` service is defined in `docker-compose.yml` and starts with the-stack. To route AiSOC through it, set in your `.env`:+The `litellm` service is part of the `full` profile in `docker-compose.yml`, and+routing to it is already wired. Put your provider key in `.env` and bring the+profile up:```bash+OPENAI_API_KEY=<your-real-provider-key> # the gateway uses this to reach the upstream modelLITELLM_MASTER_KEY=<a-strong-key> # AiSOC authenticates to the gateway with this-OPENAI_API_KEY=<your-real-provider-key> # LiteLLM uses this to reach the upstream model-OPENAI_BASE_URL=http://litellm:4000/v1 # send AiSOC's calls to the gateway-# and set AiSOC's client key to the gateway key:-# OPENAI_API_KEY=${LITELLM_MASTER_KEY} # (in the AiSOC services' environment)```-AiSOC now requests a task **alias** for every live call, so an alias only-resolves when it reaches the gateway. If you don't run the gateway, pin each-role to a concrete provider model instead (the **escape hatch**):+```bash+docker compose --profile full up -d+```++That is the whole configuration. `docker-compose.yml` sets `LLM_GATEWAY_URL` on+the `api` and `agents` services, and both services' resolvers read it for any+`aisoc-<role>` alias — an alias resolves nowhere else, so the gateway is the+only correct destination for one. The bearer token is resolved with the route:+when AiSOC picks the gateway itself it sends `LITELLM_MASTER_KEY`, never a+provider key, which the gateway would reject as an invalid proxy token.++To send AiSOC somewhere else — a gateway on another host, or your own+OpenAI-compatible endpoint — set `OPENAI_BASE_URL` (or `LLM_BASE_URL`) and it+outranks `LLM_GATEWAY_URL` for every role.++### Calling a provider directly, with no gateway++Pin each role to a concrete provider model. A concrete pin is **not** rerouted+to the gateway, so this keeps working with `LLM_GATEWAY_URL` set:```bashAISOC_MODEL_PIN_TRIAGE=gpt-4o-miniAISOC_MODEL_PIN_INVESTIGATION=gpt-4o# … one per role: triage, recon, investigation, copilot, summary, report, nl+OPENAI_MODEL=gpt-4o-mini # the BYOK / "explain this alert" path, direct too```With neither the gateway nor pin overrides configured, AiSOC uses its-deterministic offline path. (`OPENAI_MODEL` still applies to the separate-"explain this alert" / BYOK path.)+deterministic offline path, which reports zero model calls and zero tokens+rather than presenting a heuristic as triage.++### A model the gateway does not define is an error, not a fallback++Two failures used to be indistinguishable from "no LLM available", because both+arrived as an exception a caller turned into a deterministic result:++- An `aisoc-<role>` alias sent to a provider default, which does not know the+ name. AiSOC now raises `UnroutableModelError` naming the alias and the remedy+ instead of building a client destined to fail.+- A concrete model sent to the gateway, which answers `Invalid model name`.+ `preflight_llm()` reports this at boot, in both directions, and+ `scripts/check_llm_model_routing.py` fails the build when a model named in+ `.env.example` or `docker-compose.yml` is not one `infra/litellm/config.yaml`+ defines — or when the gateway defines an alias no role requests.++`OPENAI_MODEL` applies to the BYOK / "explain this alert" path **only**. It is+never a task role's model: the auto-triage worker layers a tenant's BYOK+configuration over the role pin, and only the fields the *tenant* set are+treated as overrides, so an environment default cannot displace an alias.++### Embeddings++The MITRE RAG path calls an embedding endpoint, and `infra/litellm/config.yaml`+declares chat aliases only — so embeddings deliberately do **not** follow chat+traffic to the gateway. They have their own pair of variables,+`AISOC_EMBEDDING_BASE_URL` and `AISOC_EMBEDDING_MODEL` (default+`text-embedding-3-large`), and default to the provider. The routing gate checks+that exclusion in both directions.## Re-point a task to a local modeldiff --git a/docker-compose.yml b/docker-compose.ymlindex 7d12268bf..2e57cd11d 100644--- a/docker-compose.yml+++ b/docker-compose.yml@@ -344,10 +344,17 @@ services:SECRET_KEY: ${SECRET_KEY:-dev_secret_key_change_in_production}ENVIRONMENT: developmentLOG_LEVEL: info- # Issue #478 — LiteLLM gateway. Route this service's live LLM endpoints- # (translation, hunts, KB, phishing, NL-detection) through the gateway by- # setting OPENAI_BASE_URL=http://litellm:4000/v1 + OPENAI_API_KEY=$LITELLM_MASTER_KEY.+ # Issue #478 — LiteLLM gateway. This service's live LLM endpoints+ # (translation, hunts, KB, phishing, NL-detection) route here by default:+ # `app.services.model_aliases.resolve_base_url` reads LLM_GATEWAY_URL for+ # any aisoc-<role> alias. It was previously read by nothing, so the+ # variable was set, documented, and dead — every alias went to the+ # provider default and 404'd. OPENAI_BASE_URL still overrides it.LLM_GATEWAY_URL: http://litellm:4000/v1+ # The bearer that pairs with the URL above. Without it AiSOC would send a+ # provider key to the gateway, which rejects it as an invalid proxy token.+ LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY:-sk-aisoc-local}+ OPENAI_API_KEY: ${OPENAI_API_KEY:-}# services/actions — where an approved response action is executed.# The API had no pointer to it at all: the one endpoint that called it# read the URL from os.environ and fell back to `aisoc-actions`, which@@ -516,13 +523,20 @@ services:NEO4J_PASSWORD: neo4j_dev_secretOPENAI_API_KEY: ${OPENAI_API_KEY:-}ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:-}- OPENAI_MODEL: ${OPENAI_MODEL:-gpt-4o-mini}+ # BYOK / "explain this alert" only — never a task role. A task role asks+ # `app.llm.model_pins` for its aisoc-<role> alias, and this leaking into+ # that path is what replaced `aisoc-triage` with a model the gateway does+ # not define. Defaults to the summary alias so the explain path resolves+ # at the gateway too; set a concrete model only for direct-to-provider.+ OPENAI_MODEL: ${OPENAI_MODEL:-aisoc-summary}# Issue #478 — the LiteLLM gateway (below) is the single entry point for- # live LLM calls when enabled. To route this service through it, set in- # .env: OPENAI_BASE_URL=http://litellm:4000/v1 and- # OPENAI_API_KEY=$LITELLM_MASTER_KEY. Left unset, calls go direct to the- # provider (unchanged behaviour).+ # live LLM calls. `app.llm.factory.resolve_base_url` reads this for any+ # aisoc-<role> alias, so the default deployment reaches the gateway with+ # no further configuration; OPENAI_BASE_URL still overrides it, and a+ # concrete AISOC_MODEL_PIN_<ROLE> is left pointing at its provider.LLM_GATEWAY_URL: http://litellm:4000/v1+ # The bearer that pairs with the URL above — see the api service.+ LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY:-sk-aisoc-local}ATTCK_DATA_PATH: /app/data/enterprise-attack.json# Where to post an approval card when triage raises one. Unset means# no ChatOps notification; the approval is still durable and visible indiff --git a/docs/audit/CLAIM_TO_GATE_MATRIX.md b/docs/audit/CLAIM_TO_GATE_MATRIX.mdindex fa2c73cb5..9bed5da5a 100644--- a/docs/audit/CLAIM_TO_GATE_MATRIX.md+++ b/docs/audit/CLAIM_TO_GATE_MATRIX.md@@ -144,11 +144,12 @@ Statuses: `GATED` (a CI job fails when the claim stops being true) · `PARTIAL`| What counts as a check is decided by what a script does, not what it is named | `scripts/check_gate_coverage.py` (the inventory it reports on) | `ci.yml` (three structural signals, all read from the tree: a declared verdict flag, an exit status derived from findings the script accumulates, and a workflow job whose output another job branches on. The first two are intrinsic, so an uncalled gate is still inventoried and the `SCRIPT -> WORKFLOW` direction is not vacuous. The classifier is also checked in reverse — a script CI runs with `--check` that the classifier does not inventory fails as `classifier-blind-spot`) + `tests/test_gate_coverage_classifier.py` (10 cases, including a gate planted under a name no convention would reveal, and a generator that aborts on an empty read asserted *not* to be a check) | GATED | - || No check reports OK over a repository with no content | every row above (a gate that credits an empty tree proves nothing about a full one) | `ci.yml :: python-test` (`scripts/check_gate_contract.py` — runs all 72 inventoried checks the way a workflow runs each one, inside a git repository holding only `scripts/`, and requires a non-zero exit. Two tree shapes, because they are two questions: **bare** omits the directories a gate judges, **skeleton** creates them empty. The second is the one that happens — a renamed package, a glob that stopped matching — and a gate opening with `if not X.is_dir(): return 2` answers the first for a reason that says nothing about its corpus. Measured before it was added: 64 of 71 checks already refused the skeleton, so the exception list stayed readable rather than doubling. A run that failed for a reason the empty tree did not cause — an argparse usage error, or an import of a module the repository does not supply — is INCONCLUSIVE and fails rather than counting as a refusal, so the gate cannot credit a failure it did not cause. Exceptions are recorded with the disposition they are excused for and checked in both directions: an entry naming nothing, and an entry whose gate has started behaving differently, both fail) + `tests/test_gate_contract.py` (45 cases) | GATED | 5 recorded exceptions, keyed by tree shape; the probe cannot pose the question to a gate whose only input is a path named on the command line, or one that lives inside `scripts/`. One entry exists because its dispositions legitimately differ: `check_logger_kwargs.py` scans every Python file in the repository, so against a skeleton it has a real corpus in `scripts/` and correctly finds nothing wrong || Every check resolves its root from git and carries a `--self-test` | `scripts/` (the gates themselves) | `ci.yml :: python-test` (`scripts/check_gate_contract.py` — a check that derives a repository root from `__file__` without consulting git fails as `root-from-file`, and one that declares no `--self-test` fails as `no-self-test`. Both read from the syntax tree: the first spelling matched `repo_root` anywhere in the text and `check_gate_coverage.py` emits it as a JSON key, so a non-compliant gate read as compliant on the strength of a dictionary key in its own output. Both exception lists are empty — all 72 comply — and an entry added to either must carry a reason) | GATED | - |+| Every model AiSOC sends has a gateway that defines it, and the variable compose supplies is the variable the resolvers read | `apps/docs/docs/operations/llm-gateway.md`, `.env.example`, README (AI triage) | `ci.yml :: python-lint` (`scripts/check_llm_model_routing.py` — both directions on every name: each role pin's alias must be defined by `infra/litellm/config.yaml` and each alias must be claimed by a pin; every model named in `.env.example` or `docker-compose.yml`, in a file that wires the bundled gateway, must be one the gateway defines; every variable compose points at the gateway must be read by *both* resolvers, and every gateway variable the resolvers read must be set by compose — the defect that shipped was compose supplying `LLM_GATEWAY_URL` and both resolvers deliberately ignoring it, so AI triage could not reach a model at all; a service given the gateway URL must also get `LITELLM_MASTER_KEY`, because a provider key sent there is rejected. Every parser is AST- or comment-aware and the self-test proves it: the config's commented Ollama example repeats `model_name: aisoc-triage` verbatim, `.env.example` documents the escape hatch as a commented `AISOC_MODEL_PIN_TRIAGE=`, and compose explains the gateway in prose two lines above the key it describes — a line regex credits all three) + `services/agents/tests/test_llm_factory.py` + `services/api/tests/test_model_aliases_routing.py` | GATED | Embeddings are a recorded exclusion — the gateway's model list is chat aliases only — and the gate checks that in both directions rather than trusting the comment || Every test file in the tree is executed by a workflow | `CONTRIBUTING.md` / the suites themselves | `ci.yml :: python-test` (`scripts/check_test_discovery.py` — the corpus is `git ls-files`, so a service added tomorrow is scanned with no edit, and an untracked install tree is out by construction rather than by a name somebody remembered. Reach is modelled the way pytest collects: directory arguments, `--ignore`, `conftest` `collect_ignore`, and `-m` marker deselection, because a file whose every test is deselected runs nothing however plainly it is named. Both directions — an invocation naming a path that is not in the tree fails too. Quarantine is a shrink-only list where each entry carries a reason and an entry that has become reachable fails as stale. Two blind spots in this parser were found by counting what it **credited**: a step that `cd`s into a service and then names `tests/` had its paths resolved against the repository root, which reported all 82 agents files unreached *and* credited the root tree with a package's invocation; and `-m` was first skipped as "a filter, not a collection rule", which would have credited a file in full under `-m "not integration"`) + `tests/test_test_discovery_gate.py` | GATED | - |## Summary-- GATED: 129+- GATED: 130- PARTIAL: 8- NO GATE: 0 (**every claim is backed by a failing test.** The last NO GATE — the weekly benchmark scoreboard running live against `main` — closed in Phase E1: `scripts/check_scoreboard.py` ties the published scoreboard to a deterministic per-PR live-agent MITRE-accuracy run, while the funded weekly `wet-eval.yml` appends the LLM-tier rows. The remaining PARTIAL rows are honest, named deferrals — each states the specific gap and what closes it — not unproven claims.)diff --git a/scripts/check_llm_model_routing.py b/scripts/check_llm_model_routing.pynew file mode 100644index 000000000..02b05b626--- /dev/null+++ b/scripts/check_llm_model_routing.py@@ -0,0 +1,858 @@+#!/usr/bin/env python3+"""A model name must have somewhere to go, and the variable that sends it there must be read.++Why this exists+---------------+AI triage is this product's headline capability and it could not reach a model+in the default deployment. Not for want of a key — for two wiring defects that+each looked fine in isolation:++``docker-compose.yml`` set ``LLM_GATEWAY_URL`` on the api and agents services.+Both resolvers deliberately ignored it, honouring only ``OPENAI_BASE_URL``,+which compose never set. So the one variable the deployment supplied was read by+nothing, every ``aisoc-<role>`` alias went to ``api.openai.com``, and the 404+came back through a caller's ``except`` as "no LLM available". The service even+logged the diagnosis at boot; the diagnosis existed and the wiring did not.++And ``.env.example`` shipped ``OPENAI_MODEL=gpt-4-turbo-preview`` — a model no+gateway config in this tree has ever defined — which the auto-triage worker+picked up as a per-tenant BYOK override and sent in place of ``aisoc-triage``.+The commercial deployment hit the same shape from the other end: the model names+its LiteLLM did not know produced "Invalid model name" on every call and the+whole thing degraded to empty output across the copilot and every triage agent.++Both are unreachability, and both are structural: a name that resolves nowhere,+or a variable nothing reads. Neither is visible in a diff.++What it checks, in both directions+----------------------------------+The dominant bug shape in this repository is the one-directional check: it+compares A against B and never B against A, so drift in the direction things+actually change slips through while the check prints OK. Every rule below is a+resolution requirement on a specific name, and both name spaces are enumerated.++ PIN -> GW every role pin's shipped alias is defined by the gateway+ GW -> PIN every gateway alias is claimed by a role pin; an orphan is a+ model the operator maintains and nothing requests+ AGENTS<->API the role set and the routing surface agree across the two+ packages, which cannot import one another and so drift+ CONFIG-> GW every router fallback names a defined alias, in both the key+ and the value position+ ENV -> GW every model an env/compose file names, in a file that wires+ the bundled gateway, is one the gateway defines+ CMP -> CODE every variable compose points at the bundled gateway is read+ by both resolvers. This is the defect above, stated as a rule+ CODE -> CMP every gateway variable the resolvers read is set by compose on+ the services that call models. The same defect, mirrored: a+ read with nothing to read is as dead as a write nobody reads+ PAIRING a service handed the gateway URL is handed the gateway's key.+ Resolving the two independently is what made adopting the URL+ look ambiguous; a provider key sent to LiteLLM is rejected+ SCOPE embeddings are recorded as outside the gateway (its model list+ is seven chat aliases), and the exclusion is checked in both+ directions rather than asserted in a comment++Usage+-----+ python3 scripts/check_llm_model_routing.py # gate+ python3 scripts/check_llm_model_routing.py --json+ python3 scripts/check_llm_model_routing.py --self-test # prove it bites++``--repo-root`` overrides the tree under inspection. The resolved root comes+from ``git rev-parse``, every file read and every count is printed before the+verdict, and a missing or empty input is a hard error rather than a quiet zero:+a gate that reports OK about a tree it never opened is worse than no gate.++Stdlib only. The lint job that runs this installs ruff and mypy and nothing+else, and a gate that needs a ``pip install`` to render a verdict is a gate that+can be skipped.++Exit codes: 0 clean, 1 findings, 2 the scan itself could not run.+"""++from __future__ import annotations++import argparse+import ast+import copy+import json+import re+import sys+from collections.abc import Callable+from dataclasses import dataclass, field+from pathlib import Path++# `scripts/` is on sys.path when this file is run as a program, but not when a+# test loads it by path with importlib. gate_toolkit sits beside it either way.+sys.path.insert(0, str(Path(__file__).resolve().parent))++from gate_toolkit import repo_root, self_test_main # noqa: E402++CONFIG_REL = "infra/litellm/config.yaml"+PINS_REL = "services/agents/app/llm/model_pins.py"+AGENTS_ROUTING_REL = "services/agents/app/llm/routing.py"+API_ROUTING_REL = "services/api/app/services/model_aliases.py"+ENV_EXAMPLE_REL = ".env.example"+COMPOSE_REL = "docker-compose.yml"+EMBEDDING_REL = "services/agents/app/tools/mitre_full.py"++#: Env vars whose value is a model name. Read from the tree where possible;+#: this is the shape that identifies one in a compose or dotenv file, where+#: there is no code to parse.+MODEL_VAR_RE = re.compile(r"^(OPENAI_MODEL|LLM_MODEL|AISOC_LLM_MODEL|AISOC_MODEL_PIN_[A-Z0-9_]+)$")++#: How the bundled gateway is recognised in a compose or dotenv value. Matched+#: on the service name compose gives it, so a port change does not blind this.+BUNDLED_GATEWAY_RE = re.compile(r"https?://litellm(:\d+)?(/|$)")++#: Compose services that make LLM calls, and so need both halves of the pair.+#: Derived below from the services that set a gateway variable *or* a model+#: variable, so adding a third LLM service does not require editing this file.+GATEWAY_KEY_VAR = "LITELLM_MASTER_KEY"++#: Embeddings are out of the gateway's scope: ``infra/litellm/config.yaml``+#: declares chat aliases only, so routing an embedding call there 400s on every+#: batch. Recorded here so the exclusion is a checked property rather than a+#: comment, and enforced in both directions below.+EMBEDDING_MODEL_SHAPES = ("embedding", "embed-")+++class GateError(RuntimeError):+ """The scan could not be performed — distinct from the scan finding nothing."""+++@dataclass+class RoutingModule:+ """What one routing module defines, declares as the rule, and reads."""++ defined: set[str] = field(default_factory=set)+ declared: set[str] = field(default_factory=set)+ reads: dict[str, set[str]] = field(default_factory=dict)++ @property+ def all_reads(self) -> set[str]:+ return set().union(*self.reads.values()) if self.reads else set()+++@dataclass+class Corpus:+ """Everything the gate read, named, so a caller cannot lose track of which tree."""++ gateway_aliases: set[str] = field(default_factory=set)+ fallbacks: dict[str, list[str]] = field(default_factory=dict)+ role_pins: dict[str, str] = field(default_factory=dict)+ api_roles: set[str] = field(default_factory=set)+ agents_routing: RoutingModule = field(default_factory=RoutingModule)+ api_routing: RoutingModule = field(default_factory=RoutingModule)+ env_example: dict[str, str] = field(default_factory=dict)+ compose: dict[str, dict[str, str]] = field(default_factory=dict)+ embedding_default: str = ""+++# --------------------------------------------------------------------------+# Parsing. Every parser below is comment-aware or AST-based, never a bare+# regex over raw source, because both of this gate's corpora contain text that+# looks exactly like the thing it is looking for:+#+# * ``infra/litellm/config.yaml`` ends with commented Ollama / vLLM /+# Anthropic examples that repeat ``model_name: aisoc-triage`` verbatim. A+# line regex credits three aliases the gateway does not serve.+# * ``.env.example`` documents the direct-to-provider escape hatch as+# ``# AISOC_MODEL_PIN_TRIAGE=gpt-4o-mini``. A line regex reads that+# as a live assignment and fails the build over documentation.+# * ``docker-compose.yml`` explains the gateway in prose containing+# ``OPENAI_BASE_URL=http://litellm:4000/v1`` two lines above the key it is+# describing.+#+# The self-test injects each of those three shapes and requires the parser to+# decline to credit it.+# --------------------------------------------------------------------------+++def strip_comment(line: str) -> str:+ """Drop a trailing ``#`` comment, respecting quotes."""+ out: list[str] = []+ quote: str | None = None+ for ch in line:+ if quote:+ out.append(ch)+ if ch == quote:+ quote = None+ continue+ if ch in "\"'":+ quote = ch+ out.append(ch)+ continue+ if ch == "#":+ break+ out.append(ch)+ return "".join(out).rstrip()+++def _unquote(value: str) -> str:+ value = value.strip()+ if len(value) >= 2 and value[0] == value[-1] and value[0] in "\"'":+ return value[1:-1]+ return value+++def parse_gateway_aliases(text: str) -> set[str]:+ """Every ``model_name`` the gateway actually serves."""+ aliases: set[str] = set()+ for raw in text.splitlines():+ line = strip_comment(raw)+ m = re.match(r"^\s*-?\s*model_name:\s*(\S+)\s*$", line)+ if m:+ aliases.add(_unquote(m.group(1)))+ return aliases+++def parse_gateway_fallbacks(text: str) -> dict[str, list[str]]:+ """``router_settings.fallbacks`` as ``{alias: [alias, ...]}``."""+ fallbacks: dict[str, list[str]] = {}+ in_block = False+ for raw in text.splitlines():+ line = strip_comment(raw)+ if not line.strip():+ continue+ if re.match(r"^\s*fallbacks:\s*$", line):+ in_block = True+ continue+ if in_block:+ m = re.match(r"^\s*-\s*([A-Za-z0-9_.\-]+):\s*\[(.*)\]\s*$", line)+ if not m:+ # Any non-item line ends the block; a `fallbacks:` entry is a+ # flat list of one-key maps in this file.+ if not re.match(r"^\s*-\s", line):+ in_block = False+ continue+ targets = [_unquote(t) for t in m.group(2).split(",") if t.strip()]+ fallbacks[m.group(1)] = targets+ return fallbacks+++def parse_role_pins(source: str) -> dict[str, str]:+ """``_DEFAULT_PINS`` as ``{role: shipped primary model}``, read as code.++ AST rather than regex: ``model_pins.py`` names ``aisoc-triage`` in its+ module docstring and in an inline comment explaining the escape hatch, and+ a gate that credits a variable mentioned in prose is a gate that credits+ documentation for the behaviour it describes.+ """+ tree = ast.parse(source)+ pins: dict[str, str] = {}+ for node in ast.walk(tree):+ if not isinstance(node, ast.AnnAssign | ast.Assign):+ continue+ targets = [node.target] if isinstance(node, ast.AnnAssign) else node.targets+ if not any(isinstance(t, ast.Name) and t.id == "_DEFAULT_PINS" for t in targets):+ continue+ if not isinstance(node.value, ast.Dict):+ continue+ for key, value in zip(node.value.keys, node.value.values, strict=False):+ if not (isinstance(key, ast.Constant) and isinstance(key.value, str)):+ continue+ if not isinstance(value, ast.Call) or len(value.args) < 2:+ continue+ primary = value.args[1]+ if isinstance(primary, ast.Constant) and isinstance(primary.value, str):+ pins[key.value] = primary.value+ return pins+++def parse_api_roles(source: str) -> set[str]:+ """``ROLES`` from the API-side mirror, read as code."""+ tree = ast.parse(source)+ for node in ast.walk(tree):+ if not isinstance(node, ast.Assign):+ continue+ if not any(isinstance(t, ast.Name) and t.id == "ROLES" for t in node.targets):+ continue+ for sub in ast.walk(node.value):+ if isinstance(sub, ast.Set):+ return {e.value for e in sub.elts if isinstance(e, ast.Constant) and isinstance(e.value, str)}+ return set()+++def _env_reads(node: ast.AST) -> set[str]:+ """Env var names read by an actual call under ``node``, not mentioned in prose."""+ names: set[str] = set()+ for sub in ast.walk(node):+ if isinstance(sub, ast.Call):+ func = sub.func+ target = None+ if isinstance(func, ast.Attribute) and func.attr in ("getenv", "get"):+ owner = func.value+ if isinstance(owner, ast.Name) and owner.id == "os":+ target = sub.args[0] if sub.args else None+ elif isinstance(owner, ast.Attribute) and owner.attr == "environ":+ target = sub.args[0] if sub.args else None+ if isinstance(target, ast.Constant) and isinstance(target.value, str):+ names.add(target.value)+ elif isinstance(sub, ast.Subscript):+ owner = sub.value+ if isinstance(owner, ast.Attribute) and owner.attr == "environ" and isinstance(sub.slice, ast.Constant):+ if isinstance(sub.slice.value, str):+ names.add(sub.slice.value)+ return names+++def parse_routing_module(source: str) -> RoutingModule:+ """Names defined, names declared in ``__all__``, and the env vars each function reads."""+ tree = ast.parse(source)+ functions: dict[str, set[str]] = {}+ defined: set[str] = set()+ declared: set[str] = set()+ for node in tree.body:+ if isinstance(node, ast.FunctionDef | ast.AsyncFunctionDef):+ functions[node.name] = _env_reads(node)+ defined.add(node.name)+ elif isinstance(node, ast.ClassDef):+ defined.add(node.name)+ elif isinstance(node, ast.Assign):+ names = {t.id for t in node.targets if isinstance(t, ast.Name)}+ defined |= names+ if "__all__" in names and isinstance(node.value, ast.List | ast.Tuple):+ declared = {e.value for e in node.value.elts if isinstance(e, ast.Constant) and isinstance(e.value, str)}+ return RoutingModule(defined=defined, declared=declared, reads=functions)+++def parse_env_file(text: str) -> dict[str, str]:+ """Live ``KEY=VALUE`` assignments. A commented example is documentation."""+ out: dict[str, str] = {}+ for raw in text.splitlines():+ line = raw.strip()+ if not line or line.startswith("#"):+ continue+ m = re.match(r"^(?:export\s+)?([A-Za-z_][A-Za-z0-9_]*)=(.*)$", line)+ if m:+ out[m.group(1)] = _unquote(strip_comment(m.group(2)))+ return out+++def parse_compose_env(text: str) -> dict[str, dict[str, str]]:+ """``{service: {VAR: value}}`` from a compose file's ``environment:`` blocks."""+ services: dict[str, dict[str, str]] = {}+ in_services = False+ service: str | None = None+ in_env = False+ for raw in text.splitlines():+ line = strip_comment(raw)+ if not line.strip():+ continue+ if re.match(r"^services:\s*$", line):+ in_services, service, in_env = True, None, False+ continue+ if re.match(r"^[A-Za-z_]", line): # another top-level key+ in_services, service, in_env = False, None, False+ continue+ if not in_services:+ continue+ m = re.match(r"^ ([A-Za-z0-9_.\-]+):\s*$", line)+ if m:+ service, in_env = m.group(1), False+ services.setdefault(service, {})+ continue+ if service is None:+ continue+ if re.match(r"^ environment:\s*$", line):+ in_env = True+ continue+ if re.match(r"^ [A-Za-z0-9_.\-]+:", line): # sibling of environment:+ in_env = False+ continue+ if not in_env:+ continue+ m = re.match(r"^ ([A-Z_][A-Z0-9_]*):\s*(.*)$", line)+ if m:+ services[service][m.group(1)] = _unquote(m.group(2))+ continue+ m = re.match(r"^ -\s*([A-Z_][A-Z0-9_]*)=(.*)$", line)+ if m:+ services[service][m.group(1)] = _unquote(m.group(2))+ return services+++def parse_embedding_default(source: str) -> str:+ """The model the RAG embedding path sends when nothing overrides it."""+ tree = ast.parse(source)+ for node in ast.walk(tree):+ if isinstance(node, ast.Assign) and any(isinstance(t, ast.Name) and t.id == "_DEFAULT_EMBEDDING_MODEL" for t in node.targets):+ if isinstance(node.value, ast.Constant) and isinstance(node.value.value, str):+ return node.value.value+ return ""+++# --------------------------------------------------------------------------+# Rules+# --------------------------------------------------------------------------+++def _resolve_value(value: str, env: dict[str, str]) -> str:+ """Resolve ``${VAR:-default}`` / ``${VAR}`` against a dotenv baseline."""++ def sub(m: re.Match[str]) -> str:+ name, default = m.group(1), m.group(3) or ""+ return env.get(name) or default++ return re.sub(r"\$\{([A-Za-z_][A-Za-z0-9_]*)(:-([^}]*))?\}", sub, value).strip()+++def evaluate(c: Corpus) -> list[tuple[str, str]]:+ """Return ``(code, detail)`` for every unresolvable name. Empty means clean."""+ out: list[tuple[str, str]] = []+ gateway_aliases = c.gateway_aliases+ fallbacks = c.fallbacks+ role_pins = c.role_pins+ api_roles = c.api_roles+ agents_routing = c.agents_routing+ api_routing = c.api_routing+ env_example = c.env_example+ compose = c.compose+ embedding_default = c.embedding_default++ # PIN -> GW / GW -> PIN.+ claimed = set(role_pins.values())+ for role, primary in sorted(role_pins.items()):+ if primary not in gateway_aliases:+ out.append(("pin-alias-undefined", f"role '{role}' ships primary '{primary}', which {CONFIG_REL} does not define"))+ for alias in sorted(gateway_aliases - claimed):+ out.append(("gateway-alias-unclaimed", f"{CONFIG_REL} defines '{alias}', which no role pin in {PINS_REL} requests"))++ # AGENTS <-> API.+ for role in sorted(set(role_pins) - api_roles):+ out.append(("role-set-drift", f"'{role}' is pinned in {PINS_REL} but absent from ROLES in {API_ROUTING_REL}"))+ for role in sorted(api_roles - set(role_pins)):+ out.append(("role-set-drift", f"'{role}' is in ROLES in {API_ROUTING_REL} but has no pin in {PINS_REL}"))++ # The mirror compared against the rule the agents module declares, in both+ # directions. Equality of *every* name would be wrong — the API module also+ # carries alias resolution, which the agents side gets from model_pins — so+ # the contract is ``__all__``: the mirror may not lose a rule, and may not+ # grow a routing rule the agents side does not have. Growth is detected by+ # the reads, not by the name, because a new rule is one that consults the+ # environment.+ declared = agents_routing.declared+ routing_vars = agents_routing.all_reads+ api_defined = api_routing.defined+ agents_defined = agents_routing.defined+ for name in sorted(declared - agents_defined):+ out.append(("routing-surface-drift", f"{AGENTS_ROUTING_REL} declares '{name}' in __all__ and does not define it"))+ for name in sorted(declared - api_defined):+ out.append(+ ("routing-surface-drift", f"{AGENTS_ROUTING_REL} declares '{name}' as a routing rule and the {API_ROUTING_REL} mirror lacks it")+ )+ for name, reads in sorted(api_routing.reads.items()):+ if reads & routing_vars and name.lstrip("_") not in declared:+ out.append(+ (+ "routing-surface-drift",+ f"{API_ROUTING_REL} defines '{name}', which reads {', '.join(sorted(reads & routing_vars))}, "+ f"and {AGENTS_ROUTING_REL} does not declare it — the two services would route differently",+ )+ )++ agents_reads = agents_routing.all_reads+ api_reads = api_routing.all_reads+ for name in sorted(agents_reads ^ api_reads):+ side = AGENTS_ROUTING_REL if name in agents_reads else API_ROUTING_REL+ other = API_ROUTING_REL if name in agents_reads else AGENTS_ROUTING_REL+ out.append(("routing-read-drift", f"'{name}' is read by {side} and not by {other}; the two services would route differently"))++ # CONFIG -> GW.+ for key, targets in sorted(fallbacks.items()):+ if key not in gateway_aliases:+ out.append(("fallback-undefined", f"router fallback is keyed on '{key}', which {CONFIG_REL} does not define"))+ for target in targets:+ if target not in gateway_aliases:+ out.append(("fallback-undefined", f"router fallback '{key}' falls back to '{target}', which {CONFIG_REL} does not define"))++ # ENV -> GW. Only in a file that wires the bundled gateway: an operator+ # pointing at their own vLLM with concrete model names is correct, and only+ # this gateway's contents are knowable from here.+ env_wires_gateway = any(BUNDLED_GATEWAY_RE.search(v) for v in env_example.values())+ if env_wires_gateway:+ for name, value in sorted(env_example.items()):+ if MODEL_VAR_RE.match(name) and value and value not in gateway_aliases:+ out.append(+ (+ "env-model-undefined",+ f"{ENV_EXAMPLE_REL} sets {name}={value} while wiring the bundled gateway, which does not define it",+ )+ )+ for service, env in sorted(compose.items()):+ if not any(BUNDLED_GATEWAY_RE.search(v) for v in env.values()):+ continue+ for name, value in sorted(env.items()):+ if not MODEL_VAR_RE.match(name):+ continue+ resolved = _resolve_value(value, env_example)+ if resolved and resolved not in gateway_aliases:+ out.append(+ (+ "env-model-undefined",+ f"{COMPOSE_REL} service '{service}' sets {name}={value} (resolves to '{resolved}') "+ f"while wiring the bundled gateway, which does not define it",+ )+ )++ # CMP -> CODE and CODE -> CMP. The variable compose points at the gateway+ # and the variable the resolvers read have to be the same variable.+ read_everywhere = agents_reads & api_reads+ gateway_services = {s for s, env in compose.items() if any(BUNDLED_GATEWAY_RE.search(v) for v in env.values())}+ compose_gateway_vars: set[str] = set()+ for service in gateway_services:+ for name, value in compose[service].items():+ if BUNDLED_GATEWAY_RE.search(value):+ compose_gateway_vars.add(name)+ for name in sorted(compose_gateway_vars - read_everywhere):+ out.append(+ (+ "compose-var-unread",+ f"{COMPOSE_REL} points {name} at the bundled gateway and neither resolver reads it; "+ "the deployment supplies a route nothing can follow",+ )+ )+ # The mirror. Every gateway variable the resolvers read has to be supplied,+ # or the read is as dead as the write was.+ code_gateway_vars = {n for n in read_everywhere if n.endswith("GATEWAY_URL")}+ if not code_gateway_vars:+ out.append(("code-var-unset", f"no *_GATEWAY_URL variable is read by both {AGENTS_ROUTING_REL} and {API_ROUTING_REL}"))+ for name in sorted(code_gateway_vars):+ if not any(name in compose[s] for s in compose):+ out.append(("code-var-unset", f"both resolvers read {name} and no {COMPOSE_REL} service sets it"))++ # PAIRING. The route decides the bearer; a provider key sent to LiteLLM is+ # rejected as an invalid proxy token.+ for service in sorted(gateway_services):+ if GATEWAY_KEY_VAR not in compose[service]:+ out.append(+ (+ "gateway-without-key",+ f"{COMPOSE_REL} service '{service}' is pointed at the bundled gateway without {GATEWAY_KEY_VAR}; "+ "it would authenticate with a provider key the gateway rejects",+ )+ )++ # SCOPE. Embeddings are excluded from the gateway, in both directions.+ if not embedding_default:+ out.append(("embedding-scope", f"{EMBEDDING_REL} declares no default embedding model to classify"))+ elif embedding_default.startswith("aisoc-"):+ out.append(+ (+ "embedding-scope",+ f"{EMBEDDING_REL} defaults to '{embedding_default}', a gateway alias — but {CONFIG_REL} "+ "declares chat models only, so every embedding batch would 400",+ )+ )+ for alias in sorted(gateway_aliases):+ if any(shape in alias for shape in EMBEDDING_MODEL_SHAPES):+ out.append(+ (+ "embedding-scope",+ f"{CONFIG_REL} defines '{alias}', which looks like an embedding model; the exclusion "+ f"recorded in {EMBEDDING_REL} says the gateway carries chat aliases only",+ )+ )+ return out+++# --------------------------------------------------------------------------+# I/O+# --------------------------------------------------------------------------+++def load(root: Path) -> Corpus:+ paths = {+ "config": root / CONFIG_REL,+ "pins": root / PINS_REL,+ "agents_routing": root / AGENTS_ROUTING_REL,+ "api_routing": root / API_ROUTING_REL,+ "env_example": root / ENV_EXAMPLE_REL,+ "compose": root / COMPOSE_REL,+ "embedding": root / EMBEDDING_REL,+ }+ for name, path in paths.items():+ if not path.is_file():+ raise GateError(f"expected input does not exist: {path} ({name})")++ config_text = paths["config"].read_text(encoding="utf-8")+ corpus = Corpus(+ gateway_aliases=parse_gateway_aliases(config_text),+ fallbacks=parse_gateway_fallbacks(config_text),+ role_pins=parse_role_pins(paths["pins"].read_text(encoding="utf-8")),+ api_roles=parse_api_roles(paths["api_routing"].read_text(encoding="utf-8")),+ agents_routing=parse_routing_module(paths["agents_routing"].read_text(encoding="utf-8")),+ api_routing=parse_routing_module(paths["api_routing"].read_text(encoding="utf-8")),+ env_example=parse_env_file(paths["env_example"].read_text(encoding="utf-8")),+ compose=parse_compose_env(paths["compose"].read_text(encoding="utf-8")),+ embedding_default=parse_embedding_default(paths["embedding"].read_text(encoding="utf-8")),+ )+ # An empty parse means the format moved, not that the tree is clean. Found+ # nothing and scanned nothing print the same word unless one of them refuses.+ empty = [+ name for name in ("gateway_aliases", "fallbacks", "role_pins", "api_roles", "env_example", "compose") if not getattr(corpus, name)+ ]+ if empty:+ raise GateError(f"parsed zero {empty[0]} — refusing to report a clean tree from an empty read")+ for side in ("agents_routing", "api_routing"):+ module: RoutingModule = getattr(corpus, side)+ if not module.defined:+ raise GateError(f"parsed zero definitions from {side} — refusing to report a clean tree from an empty read")+ if not corpus.agents_routing.declared:+ raise GateError(f"{AGENTS_ROUTING_REL} declares no __all__ — the routing contract the mirror is checked against is missing")+ return corpus+++def _summary(c: Corpus, root: Path) -> list[str]:+ gateway_services = sorted(s for s, env in c.compose.items() if any(BUNDLED_GATEWAY_RE.search(v) for v in env.values()))+ reads = ", ".join(sorted(c.agents_routing.all_reads))+ wired = ", ".join(gateway_services)+ return [+ f"repo root {root}",+ f"gateway config {CONFIG_REL} ({len(c.gateway_aliases)} aliases, {len(c.fallbacks)} router fallbacks)",+ f"role pins {PINS_REL} ({len(c.role_pins)} roles)",+ f"agents routing {AGENTS_ROUTING_REL} ({len(c.agents_routing.declared)} declared rules, reads {reads})",+ f"api routing {API_ROUTING_REL} ({len(c.api_routing.defined)} definitions, {len(c.api_roles)} roles)",+ f"env example {ENV_EXAMPLE_REL} ({len(c.env_example)} live assignments)",+ f"compose {COMPOSE_REL} ({len(c.compose)} services, {len(gateway_services)} on the bundled gateway: {wired})",+ f"embedding scope {EMBEDDING_REL} (default '{c.embedding_default}', excluded from the gateway)",+ ]+++# --------------------------------------------------------------------------+# Self-test+# --------------------------------------------------------------------------+++def self_test(root: Path) -> int:+ """Prove the gate still catches what it claims, and still declines prose.++ Two kinds of case. The first injects a defect into the parsed data and+ requires the matching code. The second feeds the *parsers* text shaped like+ the thing they look for but commented out, and requires them not to credit+ it — because every blind spot found in this tree so far was something a+ gate credited, not something it failed to flag.+ """+ try:+ base = load(root)+ except GateError as exc:+ print(f"self-test cannot run: {exc}", file=sys.stderr)+ return 2++ baseline = evaluate(base)+ if baseline:+ print("self-test cannot run: the clean tree already has findings:", file=sys.stderr)+ for code, detail in baseline:+ print(f" [{code}] {detail}", file=sys.stderr)+ return 2++ def drop_gateway_read(c: Corpus) -> None:+ for reads in c.api_routing.reads.values():+ reads.discard("LLM_GATEWAY_URL")++ def add_unset_gateway_var(c: Corpus) -> None:+ for module in (c.agents_routing, c.api_routing):+ module.reads.setdefault("gateway_url", set()).add("AISOC_GATEWAY_URL")++ def drop_gateway_key(c: Corpus) -> None:+ c.compose["agents"].pop(GATEWAY_KEY_VAR, None)++ def use_embedding_alias(c: Corpus) -> None:+ c.embedding_default = "aisoc-summary"++ # Each case is a defect injected into a deep copy of the real corpus, and+ # the code it must produce. deepcopy, not a shallow one: the corpora are+ # nested (compose is service -> var -> value), and the first draft let one+ # case mutate the baseline every later case was measured against, so every+ # case "caught" three codes it had inherited. A self-test that passes for+ # the wrong reason is the defect this gate exists to find, one level up.+ cases: list[tuple[str, str, Callable[[Corpus], None]]] = [+ (+ "PIN -> GW: a role pinned to an alias the gateway does not define",+ "pin-alias-undefined",+ lambda c: c.role_pins.update({"triage": "aisoc-triage-v2"}),+ ),+ (+ "GW -> PIN: an alias the gateway serves that no role requests",+ "gateway-alias-unclaimed",+ lambda c: c.gateway_aliases.add("aisoc-orphan"),+ ),+ (+ "AGENTS <-> API: a role pinned on one side only",+ "role-set-drift",+ lambda c: c.api_roles.discard("triage"),+ ),+ (+ "AGENTS <-> API: the mirror loses a routing function",+ "routing-surface-drift",+ lambda c: c.api_routing.defined.discard("resolve_api_key"),+ ),+ (+ "AGENTS <-> API: the mirror stops reading a variable the other reads",+ "routing-read-drift",+ drop_gateway_read,+ ),+ (+ "CONFIG -> GW: a router fallback naming an undefined alias",+ "fallback-undefined",+ lambda c: c.fallbacks.update({"aisoc-report": ["aisoc-nonexistent"]}),+ ),+ (+ "ENV -> GW: .env.example ships a model the gateway does not define",+ "env-model-undefined",+ lambda c: c.env_example.update({"OPENAI_MODEL": "gpt-4-turbo-preview"}),+ ),+ (+ "ENV -> GW: a compose service pins a role to a model the gateway lacks",+ "env-model-undefined",+ lambda c: c.compose["agents"].update({"AISOC_MODEL_PIN_TRIAGE": "gpt-4o-mini"}),+ ),+ (+ "CMP -> CODE: compose supplies a gateway route no resolver reads",+ "compose-var-unread",+ lambda c: c.compose["agents"].update({"LLM_PROXY_URL": "http://litellm:4000/v1"}),+ ),+ (+ "CODE -> CMP: both resolvers read a gateway variable compose never sets",+ "code-var-unset",+ add_unset_gateway_var,+ ),+ (+ "PAIRING: a service given the gateway URL but not the gateway key",+ "gateway-without-key",+ drop_gateway_key,+ ),+ (+ "SCOPE: the gateway grows an embedding alias it cannot serve as chat",+ "embedding-scope",+ lambda c: c.gateway_aliases.add("aisoc-embedding"),+ ),+ (+ "SCOPE: the RAG path defaults to a gateway alias for an embedding call",+ "embedding-scope",+ use_embedding_alias,+ ),+ ]++ print(f"self-test against {root}")+ print("clean tree: 0 findings (the baseline every case below perturbs)\n")+ results: list[tuple[str, bool]] = []+ for description, expected, mutate in cases:+ perturbed = copy.deepcopy(base)+ mutate(perturbed)+ codes = {code for code, _ in evaluate(perturbed)}+ caught = expected in codes+ results.append((f"{description}\n expected [{expected}] got {sorted(codes) or 'nothing'}", caught))++ # Parser blind spots: text that looks like the thing, and is not it.+ blind: list[tuple[str, bool]] = [+ (+ "PARSER: a commented-out model_name in the Ollama example is not a served alias",+ "aisoc-never-served"+ not in parse_gateway_aliases("model_list:\n - model_name: aisoc-triage\n # - model_name: aisoc-never-served\n"),+ ),+ (+ "PARSER: a model_name inside a quoted value is not mistaken for a comment boundary",+ parse_gateway_aliases('model_list:\n - model_name: "aisoc-triage" # the high-volume path\n') == {"aisoc-triage"},+ ),+ (+ "PARSER: a commented-out .env assignment is documentation, not configuration",+ parse_env_file("# AISOC_MODEL_PIN_TRIAGE=gpt-4o-mini\nOPENAI_MODEL=aisoc-summary\n")+ == {"OPENAI_MODEL": "aisoc-summary"},+ ),+ (+ "PARSER: a compose comment naming a variable two lines above is not an assignment",+ parse_compose_env(+ "services:\n agents:\n environment:\n"+ " # set OPENAI_BASE_URL=http://litellm:4000/v1 to route here\n"+ " LLM_GATEWAY_URL: http://litellm:4000/v1\n"+ )+ == {"agents": {"LLM_GATEWAY_URL": "http://litellm:4000/v1"}},+ ),+ (+ "PARSER: a role pin named only in a docstring is not a shipped pin",+ parse_role_pins('"""_DEFAULT_PINS maps triage to aisoc-triage."""\n_DEFAULT_PINS = {}\n') == {},+ ),+ (+ "PARSER: an env var named only in a docstring is not read",+ "LLM_GATEWAY_URL"+ not in parse_routing_module('def gateway_url():\n """Reads LLM_GATEWAY_URL."""\n return None\n').all_reads,+ ),+ (+ "PARSER: an env var read through os.environ[...] counts as read",+ parse_routing_module('import os\ndef f():\n return os.environ["LLM_GATEWAY_URL"]\n').all_reads == {"LLM_GATEWAY_URL"},+ ),+ ]+ results.extend(blind)+ results.append(+ (+ f"{len(cases)} injected defects and {len(blind)} parser blind spots, each caught by its own code",+ all(passed for _, passed in results),+ )+ )+ # Delegated so the empty-tree probe is the shared one every gate answers,+ # run here rather than restated: a gate that refuses nothing is the failure+ # scripts/check_gate_contract.py exists to make impossible.+ return self_test_main(Path(__file__).name, [], extra=results)+++def main(argv: list[str] | None = None) -> int:+ parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)+ parser.add_argument("--repo-root", type=Path, default=None)+ parser.add_argument("--json", action="store_true")+ parser.add_argument("--self-test", action="store_true", help="prove the gate detects injected drift in each direction")+ args = parser.parse_args(argv)++ root = (args.repo_root or repo_root()).resolve()+ if args.self_test:+ return self_test(root)++ try:+ corpus = load(root)+ except GateError as exc:+ print(f"check_llm_model_routing: FAILED to read the tree: {exc}", file=sys.stderr)+ return 2+ except SyntaxError as exc:+ print(f"check_llm_model_routing: FAILED to parse the tree: {exc}", file=sys.stderr)+ return 2++ findings = evaluate(corpus)++ if args.json:+ print(+ json.dumps(+ {+ "repo_root": str(root),+ "gateway_aliases": sorted(corpus.gateway_aliases),+ "role_pins": corpus.role_pins,+ "api_roles": sorted(corpus.api_roles),+ "agents_reads": sorted(corpus.agents_routing.all_reads),+ "api_reads": sorted(corpus.api_routing.all_reads),+ "embedding_default": corpus.embedding_default,+ "findings": [{"code": c, "detail": d} for c, d in findings],+ },+ indent=2,+ sort_keys=True,+ )+ )+ return 1 if findings else 0++ for line in _summary(corpus, root):+ print(line)+ print()+ if findings:+ print(f"FAIL: {len(findings)} model name(s) or routing variable(s) that resolve nowhere\n", file=sys.stderr)+ for code, detail in findings:+ print(f" [{code}] {detail}", file=sys.stderr)+ return 1+ print(+ f"OK: {len(corpus.role_pins)} role pins, {len(corpus.gateway_aliases)} gateway aliases and "+ f"every model named in {ENV_EXAMPLE_REL} / {COMPOSE_REL} resolve at the gateway, "+ "and the variable compose supplies is the variable both resolvers read"+ )+ return 0+++if __name__ == "__main__":+ sys.exit(main())diff --git a/services/agents/app/agents/auto_triage_agent.py b/services/agents/app/agents/auto_triage_agent.pyindex 068a18c4d..4dd70e29c 100644--- a/services/agents/app/agents/auto_triage_agent.py+++ b/services/agents/app/agents/auto_triage_agent.py@@ -231,10 +231,15 @@ async def run_auto_triage(state: InvestigationState) -> InvestigationState:alert_context = _build_alert_context(state)- llm = make_chat_model("triage", temperature=0.0, max_tokens=512)-t0 = time.monotonic()try:+ # Inside the try: "the model could not be built" and "the call failed"+ # are the same condition to every caller, and only one of them used to+ # become an AutoTriageError. An unroutable gateway alias raises here,+ # and auto_triage_node catches AutoTriageError specifically — so+ # constructing outside would have failed the whole graph run over a+ # configuration problem the deterministic path handles fine.+ llm = make_chat_model("triage", temperature=0.0, max_tokens=512)response = await safe_ainvoke(llm,[diff --git a/services/agents/app/api/contextual.py b/services/agents/app/api/contextual.pyindex aeb8ffb3e..9773f0531 100644--- a/services/agents/app/api/contextual.py+++ b/services/agents/app/api/contextual.py@@ -33,7 +33,6 @@ unset, so the demo path never breaks.from __future__ import annotationsimport json-import osimport timeimport uuidfrom collections.abc import AsyncIterator@@ -46,7 +45,7 @@ from fastapi.responses import StreamingResponsefrom pydantic import BaseModel, Fieldfrom app.llm import safe_ainvoke, safe_astream-from app.llm.factory import resolve_model_alias+from app.llm.factory import make_chat_model, resolve_api_key, resolve_model_aliasfrom app.prompt_serialization import summarize_structure_for_llmfrom app.security.tenant_scope import require_console_or_service_auth@@ -369,17 +368,18 @@ async def _call_llm(system: str, user: str, model: str) -> tuple[str, int]:Falls back to a deterministic stub response when ``OPENAI_API_KEY`` ismissing so the demo never hard-fails."""- if not os.getenv("OPENAI_API_KEY"):+ if not resolve_api_key(model):return _fallback_response(system, user), 0try:from langchain_core.messages import HumanMessage, SystemMessage- from langchain_openai import ChatOpenAIexcept ImportError as exc:logger.warning("contextual.llm.import_failed", error=str(exc))return _fallback_response(system, user), 0- llm = ChatOpenAI(model=model, temperature=0.2)+ # Was a bare ``ChatOpenAI(model=model)`` — no base URL, so the copilot sent+ # `aisoc-copilot` to the provider default and took the 404 as "no LLM".+ llm = make_chat_model("copilot", temperature=0.2)response = await safe_ainvoke(llm, [SystemMessage(content=system), HumanMessage(content=user)])text = response.content if isinstance(response.content, str) else str(response.content)tokens = 0@@ -390,7 +390,7 @@ async def _call_llm(system: str, user: str, model: str) -> tuple[str, int]:async def _stream_llm(system: str, user: str, model: str) -> AsyncIterator[str]:"""Yield response delta chunks. Used by the NDJSON streaming endpoint."""- if not os.getenv("OPENAI_API_KEY"):+ if not resolve_api_key(model):# Fake-stream the fallback in 8-character chunks for a nice UX in the# demo path.text = _fallback_response(system, user)@@ -400,14 +400,13 @@ async def _stream_llm(system: str, user: str, model: str) -> AsyncIterator[str]:try:from langchain_core.messages import HumanMessage, SystemMessage- from langchain_openai import ChatOpenAIexcept ImportError:text = _fallback_response(system, user)for i in range(0, len(text), 8):yield text[i : i + 8]return- llm = ChatOpenAI(model=model, temperature=0.2, streaming=True)+ llm = make_chat_model("copilot", temperature=0.2, streaming=True)async for chunk in safe_astream(llm, [SystemMessage(content=system), HumanMessage(content=user)]):if hasattr(chunk, "content") and chunk.content:yield chunk.content if isinstance(chunk.content, str) else str(chunk.content)@@ -423,9 +422,9 @@ def _fallback_response(system: str, user: str) -> str:"so the contextual Copilot cannot reach a language model right now. ""You're seeing a deterministic placeholder response.\n\n""**To enable real contextual answers:**\n\n"- "1. Set `OPENAI_API_KEY` in your `.env` file\n"- "2. Point AiSOC at the LiteLLM gateway (`OPENAI_BASE_URL=http://litellm:4000/v1`), "- "which maps each task alias to a real model\n"+ "1. Set `OPENAI_API_KEY` in your `.env` file — the gateway reads it\n"+ "2. Bring up the `full` profile so the LiteLLM gateway runs; "+ "`LLM_GATEWAY_URL` already points every service at it\n""3. Restart the `aisoc-agents` service\n\n""See the LLM-gateway docs for self-hosted model alternatives (Ollama, vLLM, Together).")@@ -453,7 +452,10 @@ async def run_action(req: ContextualActionRequest) -> ContextualActionResponse:system, user = _build_messages(req)model = resolve_model_alias("copilot")- fallback = not bool(os.getenv("OPENAI_API_KEY"))+ # Same question _call_llm asks, so the flag the UI reads and the branch+ # actually taken cannot disagree — the gateway authenticates with+ # LITELLM_MASTER_KEY, which OPENAI_API_KEY alone does not see.+ fallback = not resolve_api_key(model)try:content, tokens = await _call_llm(system, user, model)except Exception as exc: # noqa: BLE001@@ -512,7 +514,10 @@ async def run_action_stream(req: ContextualActionRequest) -> StreamingResponse:model = resolve_model_alias("copilot")title = _TITLES.get((req.page, req.action), f"{req.page} · {req.action}")suggestions = _FOLLOW_UPS.get((req.page, req.action), [])- fallback = not bool(os.getenv("OPENAI_API_KEY"))+ # Same question _call_llm asks, so the flag the UI reads and the branch+ # actually taken cannot disagree — the gateway authenticates with+ # LITELLM_MASTER_KEY, which OPENAI_API_KEY alone does not see.+ fallback = not resolve_api_key(model)async def gen() -> AsyncIterator[bytes]:# Header frame so the UI can render the title before tokens arrive.diff --git a/services/agents/app/api/copilot.py b/services/agents/app/api/copilot.pyindex b335bc955..ec325b501 100644--- a/services/agents/app/api/copilot.py+++ b/services/agents/app/api/copilot.py@@ -16,7 +16,6 @@ from __future__ import annotationsimport itertoolsimport json-import osimport uuidfrom collections.abc import AsyncIteratorfrom datetime import UTC, datetime@@ -136,13 +135,18 @@ async def _get_openai_reply(alerts" as real analysis of their environment. The frontend had an honestfallback of its own that never fired, because the backend reported success."""- api_key = os.getenv("OPENAI_API_KEY", "")+ from app.llm.factory import resolve_api_key, resolve_model_alias++ model = resolve_model_alias("copilot")+ # Resolved with the route, not from OPENAI_API_KEY directly: when the call+ # goes to the bundled gateway the bearer has to be the gateway's master key.+ api_key = resolve_api_key(model) or ""if not api_key:return _synthetic_reply(user_message), "template"try:from app.llm.contract import safe_chat_completions_request- from app.llm.factory import chat_completions_url, resolve_model_alias+ from app.llm.factory import chat_completions_urlmessages: list[dict[str, str]] = [{@@ -161,9 +165,9 @@ async def _get_openai_reply(body = await safe_chat_completions_request(api_key=api_key,- model=resolve_model_alias("copilot"),+ model=model,messages=messages,- url=chat_completions_url(),+ url=chat_completions_url(model),max_tokens=512,)return body["choices"][0]["message"]["content"], "llm"diff --git a/services/agents/app/llm/factory.py b/services/agents/app/llm/factory.pyindex 46493baaf..f732a96c8 100644--- a/services/agents/app/llm/factory.py+++ b/services/agents/app/llm/factory.py@@ -14,10 +14,12 @@ instead of through the gateway can pin a concrete model. When no live LLM isreachable, callers fall back to their deterministic path exactly as before — thefactory never forces a live call.-Routing to the gateway is deliberate: set ``OPENAI_BASE_URL`` (or ``LLM_BASE_URL``)-to the gateway and send ``OPENAI_API_KEY=$LITELLM_MASTER_KEY``. Left unset, the-client talks to its provider default — where an alias only resolves if the-operator has pinned a concrete model via ``AISOC_MODEL_PIN_<ROLE>``.+Which URL a model goes to, and which bearer goes with it, is+:mod:`app.llm.routing` — shared with the BYOK resolver and mirrored in the API+service. The short version: the compose-provided ``LLM_GATEWAY_URL`` is adopted+for an ``aisoc-<role>`` alias (it can resolve nowhere else) and left alone for a+concrete pinned model (which the gateway would not define), and an explicit+``OPENAI_BASE_URL`` always wins."""from __future__ import annotations@@ -33,6 +35,42 @@ from langchain_openai import ChatOpenAIfrom app.llm.contract import DEFAULT_OPENAI_CHAT_COMPLETIONS_URLfrom app.llm.model_pins import get_pin+# The routing rule (which URL, which bearer, and when an alias has nowhere to+# go) lives in one stdlib-only module because the BYOK resolver needs it too+# and cannot import this one. Re-exported so existing call sites and tests keep+# importing these names from the factory.+from app.llm.routing import (+ BYOK_MODEL_ENV_VARS,+ GATEWAY_ALIAS_PREFIX,+ UnroutableModelError,+ adopted_gateway,+ assert_routable,+ at_bundled_gateway,+ gateway_url,+ is_gateway_alias,+ resolve_api_key,+ resolve_base_url,+)+from app.llm.routing import explicit_base_url as _explicit_base_url++__all__ = [+ "BYOK_MODEL_ENV_VARS",+ "GATEWAY_ALIAS_PREFIX",+ "UnroutableModelError",+ "adopted_gateway",+ "assert_routable",+ "at_bundled_gateway",+ "chat_completions_url",+ "gateway_url",+ "is_gateway_alias",+ "llm_override",+ "make_chat_model",+ "preflight_llm",+ "resolve_api_key",+ "resolve_base_url",+ "resolve_model_alias",+]+# Wave 1 — per-tenant BYOK override. The auto-triage / investigation paths# resolve a tenant's own (base_url, model, api_key) via# app.security.llm_resolver and bind it here so make_chat_model actually routes@@ -62,22 +100,16 @@ def resolve_model_alias(role: str) -> str:return get_pin(role).primary_model-def resolve_base_url() -> str | None:- """OpenAI-compatible base URL for live calls, or ``None`` for the client default.+def chat_completions_url(model: str | None = None) -> str:+ """Full chat-completions URL for the raw-HTTP path (:func:`safe_chat_completions_request`).- Honours an explicit ``OPENAI_BASE_URL`` / ``LLM_BASE_URL`` — what an operator- sets to point AiSOC at the LiteLLM gateway. ``None`` means "use the client's- provider default" (the direct-to-provider path). We intentionally do **not**- auto-adopt the compose-provided ``LLM_GATEWAY_URL`` here: routing through the- gateway is an explicit choice so the bearer token (the gateway master key vs.- a provider key) is never ambiguous.+ ``model`` is what the caller is about to send. Passing it lets the same+ routing rule as :func:`resolve_base_url` apply, and turns an alias with no+ gateway into :class:`UnroutableModelError` here rather than a 404 later."""- return os.getenv("OPENAI_BASE_URL", "").strip() or os.getenv("LLM_BASE_URL", "").strip() or None---def chat_completions_url() -> str:- """Full chat-completions URL for the raw-HTTP path (:func:`safe_chat_completions_request`)."""- base = resolve_base_url()+ base = resolve_base_url(model)+ if model is not None:+ assert_routable(model, base)if base:return base.rstrip("/") + "/chat/completions"return DEFAULT_OPENAI_CHAT_COMPLETIONS_URL@@ -97,39 +129,61 @@ def make_chat_model(contract. Extra ``kwargs`` pass straight through to ``ChatOpenAI``."""override = _llm_override.get()- params: dict[str, Any] = {- "model": (override or {}).get("model") or resolve_model_alias(role),- "temperature": temperature,- }+ model = (override or {}).get("model") or resolve_model_alias(role)+ params: dict[str, Any] = {"model": model, "temperature": temperature}if max_tokens is not None:params["max_tokens"] = max_tokens- base_url = (override or {}).get("base_url") or resolve_base_url()+ base_url = (override or {}).get("base_url") or resolve_base_url(model)+ assert_routable(model, base_url)if base_url:params["base_url"] = base_url- if override and override.get("api_key"):- params["api_key"] = override["api_key"]+ api_key = (override or {}).get("api_key") or resolve_api_key(model)+ if api_key:+ params["api_key"] = api_keyparams.update(kwargs)return ChatOpenAI(**params)def preflight_llm(roles: tuple[str, ...] = ("triage", "recon", "investigation", "report", "summary", "copilot", "nl")) -> list[str]:- """Return startup warnings about unroutable LLM aliases.+ """Return startup warnings about LLM config that cannot reach a model.- Catches the LiteLLM 400 gotcha: when no gateway base URL is configured, an- ``aisoc-<role>`` alias is sent straight to the provider default (e.g.- api.openai.com), which 400s with "model does not exist" — and the agents- then silently degrade to heuristics with no operator signal. This surfaces- the misconfiguration explicitly at boot. Empty list => config looks routable.+ Checked in both directions, because each one shipped broken:++ * **alias with no gateway** — an ``aisoc-<role>`` alias sent to a provider+ default 404s, and the caller's ``except`` renders that as a deterministic+ fallback. This condition was already reported here, correctly, while+ nothing read the variable that would have fixed it.+ * **concrete model at the gateway** — the reverse. A BYOK model name+ (``OPENAI_MODEL``) or a role pin naming something the gateway does not+ define 400s with "Invalid model name" once traffic is routed through it.++ Empty list => every role has somewhere to send its model."""warnings: list[str] = []- if resolve_base_url():- return warnings # a gateway/base URL is set — aliases resolve there.- unrouted = sorted({resolve_model_alias(r) for r in roles if resolve_model_alias(r).startswith("aisoc-")})+ aliases = {r: resolve_model_alias(r) for r in roles}++ unrouted = sorted({m for m in aliases.values() if is_gateway_alias(m) and not resolve_base_url(m)})if unrouted:warnings.append(- "No OPENAI_BASE_URL/LLM_BASE_URL is set, so LiteLLM aliases "- f"({', '.join(unrouted)}) will be sent to the provider default and 400. "- "Point OPENAI_BASE_URL at the LiteLLM gateway, or pin concrete models "- "via AISOC_MODEL_PIN_<ROLE>. Agents will otherwise run deterministic-only."+ f"No LLM gateway is configured, so LiteLLM aliases ({', '.join(unrouted)}) have "+ "nowhere to resolve and every live call will fail. Set LLM_GATEWAY_URL "+ "(docker-compose.yml already does) or OPENAI_BASE_URL to the gateway, or pin "+ "concrete models via AISOC_MODEL_PIN_<ROLE>. Agents run deterministic-only until then."+ )++ concrete_at_gateway = sorted({m for m in aliases.values() if not is_gateway_alias(m) and at_bundled_gateway(resolve_base_url(m))})+ if concrete_at_gateway:+ warnings.append(+ f"AISOC_MODEL_PIN_<ROLE> names concrete models ({', '.join(concrete_at_gateway)}) while traffic "+ "is routed at the bundled LiteLLM gateway, which 400s any model it does not define. Either add "+ "them to infra/litellm/config.yaml or drop the pins so the aisoc-<role> aliases apply."+ )++ byok = sorted({v for name in BYOK_MODEL_ENV_VARS if (v := os.getenv(name, "").strip()) and not is_gateway_alias(v)})+ if byok and at_bundled_gateway(_explicit_base_url() or gateway_url()):+ warnings.append(+ f"{'/'.join(BYOK_MODEL_ENV_VARS)} names a concrete model ({', '.join(byok)}) the bundled gateway "+ "does not define, so the BYOK / explain path will 400. These never apply to a task role — "+ "set one of them to an aisoc-<role> alias, or add the model to infra/litellm/config.yaml.")return warningsdiff --git a/services/agents/app/llm/routing.py b/services/agents/app/llm/routing.pynew file mode 100644index 000000000..49c3df77c--- /dev/null+++ b/services/agents/app/llm/routing.py@@ -0,0 +1,157 @@+"""Where a model name goes, and which bearer goes with it.++One rule, in one place, because it had been written three times and disagreed+each time. ``app.llm.factory`` resolves it for the LangChain path,+``app.security.llm_resolver`` for the BYOK / "explain this alert" path, and+``app.services.model_aliases`` is the byte-equivalent mirror in the API service+(a separate package that cannot import this one).++The defect this replaces+------------------------+``docker-compose.yml`` set ``LLM_GATEWAY_URL`` on the api and agents services.+Nothing read it. The resolvers honoured only ``OPENAI_BASE_URL`` /+``LLM_BASE_URL``, which compose never set, and declined to adopt the gateway+variable on the grounds that routing through the gateway should be an explicit+choice so the bearer token is never ambiguous. The reasoning was sound and the+result was that the product's headline capability could not reach a model in the+default deployment: an ``aisoc-triage`` alias went to ``api.openai.com``, which+404s, and the caller's ``except`` rendered that as "no LLM available".++Two rules make the adoption unambiguous rather than merely convenient:++**The model decides the route.** ``LLM_GATEWAY_URL`` is adopted only for a+gateway alias. An alias is unroutable anywhere else, so sending it to the+gateway cannot be wrong. A concrete model — ``AISOC_MODEL_PIN_TRIAGE=gpt-4o-mini``,+the documented direct-to-provider escape hatch — is left pointing at its+provider, because hijacking it to a gateway that does not define it would break+a deployment that works today.++**The route decides the bearer.** When AiSOC picks the gateway itself, the+bearer is the gateway's ``LITELLM_MASTER_KEY``; a provider key sent there is+rejected as an invalid proxy token. When the operator set ``OPENAI_BASE_URL``+by hand they own the pairing. That is the ambiguity the old refusal was about,+resolved instead of avoided.++Stdlib only: the BYOK resolver is imported by explain unit tests that avoid the+LangChain dependency graph, so this cannot reach for anything heavier.+"""++from __future__ import annotations++import os++#: The routing rule, named. ``scripts/check_llm_model_routing.py`` requires the+#: API-service mirror to define every one of these, and requires any function+#: in the mirror that reads a routing variable to appear here — so the mirror+#: can neither lose a rule nor grow one this module does not have.+__all__ = [+ "BYOK_MODEL_ENV_VARS",+ "GATEWAY_ALIAS_PREFIX",+ "UnroutableModelError",+ "adopted_gateway",+ "assert_routable",+ "at_bundled_gateway",+ "explicit_base_url",+ "gateway_url",+ "is_gateway_alias",+ "resolve_api_key",+ "resolve_base_url",+]++#: Prefix of every logical task alias the bundled gateway defines. The alias set+#: itself lives in ``infra/litellm/config.yaml``;+#: ``scripts/check_llm_model_routing.py`` reconciles that file against the role+#: pins in both directions. Only the *shape* is known here, because the service+#: images do not ship the gateway config.+GATEWAY_ALIAS_PREFIX = "aisoc-"++#: Env vars holding a model name for the BYOK / "explain this alert" path. They+#: are not role pins. A task role that adopts one has taken a global default for+#: a per-tenant override, which is how ``OPENAI_MODEL`` came to replace+#: ``aisoc-triage`` on the highest-volume path in the product.+BYOK_MODEL_ENV_VARS = ("OPENAI_MODEL", "LLM_MODEL", "AISOC_LLM_MODEL")+++class UnroutableModelError(RuntimeError):+ """A gateway alias was requested with nowhere to send it.++ An ``aisoc-<role>`` alias only means something to the LiteLLM gateway. Sent+ to a provider default it returns 404, and the caller's ``except`` turns that+ into a deterministic fallback — a configuration error wearing the costume of+ a degraded-but-working system. Raising names the actual fault.+ """+++def is_gateway_alias(model: str | None) -> bool:+ """Whether ``model`` is a logical alias that only the gateway can resolve."""+ return bool(model) and str(model).strip().startswith(GATEWAY_ALIAS_PREFIX)+++def explicit_base_url() -> str | None:+ """The base URL an operator set by hand, if any."""+ return os.environ.get("OPENAI_BASE_URL", "").strip() or os.environ.get("LLM_BASE_URL", "").strip() or None+++def gateway_url() -> str | None:+ """The bundled LiteLLM gateway's in-network URL, as compose supplies it."""+ return os.environ.get("LLM_GATEWAY_URL", "").strip() or None+++def resolve_base_url(model: str | None = None) -> str | None:+ """OpenAI-compatible base URL for ``model``, or ``None`` for the client default.++ Precedence, highest first: ``OPENAI_BASE_URL`` / ``LLM_BASE_URL``, then+ ``LLM_GATEWAY_URL`` when ``model`` is a gateway alias, then nothing.++ ``model=None`` means the caller has not said which model it will send.+ Every task role defaults to an alias, so the gateway is the right answer.+ """+ explicit = explicit_base_url()+ if explicit:+ return explicit+ if model is None or is_gateway_alias(model):+ return gateway_url()+ return None+++def adopted_gateway(model: str | None = None) -> bool:+ """Whether the base URL for ``model`` came from ``LLM_GATEWAY_URL``, not an operator."""+ return explicit_base_url() is None and resolve_base_url(model) is not None+++def at_bundled_gateway(base_url: str | None) -> bool:+ """Whether ``base_url`` is the gateway this repo ships a model list for.++ Narrower than "some base URL is set" on purpose: an operator pointing at+ their own vLLM with concrete model names is correct, and warning about it+ would be noise. Only ``infra/litellm/config.yaml``'s alias set is knowable+ from here, so only that gateway's contents can be reasoned about.+ """+ gateway = gateway_url()+ if not base_url or not gateway:+ return False+ return str(base_url).rstrip("/") == gateway.rstrip("/")+++def resolve_api_key(model: str | None = None) -> str | None:+ """Bearer token that pairs with :func:`resolve_base_url`'s answer for ``model``."""+ if adopted_gateway(model):+ return os.environ.get("LITELLM_MASTER_KEY", "").strip() or os.environ.get("OPENAI_API_KEY", "").strip() or None+ return os.environ.get("OPENAI_API_KEY", "").strip() or os.environ.get("LLM_API_KEY", "").strip() or None+++def assert_routable(model: str, base_url: str | None) -> None:+ """Raise :class:`UnroutableModelError` if ``model`` can never resolve at ``base_url``.++ One shape, checked at every entry point: a gateway alias with no gateway to+ send it to. Everything else is the provider's call to make — a wrong+ concrete model name comes back as a 404 from a host that can answer the+ question, which is a diagnosable result rather than a silent one.+ """+ if is_gateway_alias(model) and not base_url:+ raise UnroutableModelError(+ f"model '{model}' is a LiteLLM gateway alias and no gateway is configured. "+ "Set LLM_GATEWAY_URL (docker-compose.yml already does) or OPENAI_BASE_URL to the "+ "gateway, or pin a concrete provider model via AISOC_MODEL_PIN_<ROLE>. "+ "Sending an alias to a provider default returns 404 and degrades silently."+ )diff --git a/services/agents/app/nl_query/translator.py b/services/agents/app/nl_query/translator.pyindex 5de992be3..98e4f6eb5 100644--- a/services/agents/app/nl_query/translator.py+++ b/services/agents/app/nl_query/translator.py@@ -930,11 +930,12 @@ async def enhance_with_llm({"role": "user", "content": prompt},]+ resolved_model = model or resolve_model_alias("nl")payload = await safe_chat_completions_request(api_key=api_key,- model=model or resolve_model_alias("nl"),+ model=resolved_model,messages=messages,- url=chat_completions_url(),+ url=chat_completions_url(resolved_model),timeout=timeout,response_format={"type": "json_object"},temperature=0,diff --git a/services/agents/app/security/llm_resolver.py b/services/agents/app/security/llm_resolver.pyindex 9e6c69489..b8141ccb4 100644--- a/services/agents/app/security/llm_resolver.py+++ b/services/agents/app/security/llm_resolver.py@@ -42,8 +42,9 @@ For each of the three fields we resolve in this order:1. Tenant row (when present, ``enabled=true``, and successfullydecrypted in the ``api_key_vault`` case).2. Process env vars (``OPENAI_*`` / ``LLM_*``).-3. Platform default (``https://api.openai.com`` / ``gpt-4o-mini``)- so the explain path always has a deterministic config to log+3. Platform default — the bundled gateway and the ``summary`` alias when a+ gateway is configured, ``https://api.openai.com`` / ``gpt-4o-mini``+ otherwise — so the explain path always has a deterministic config to logagainst, even when it ultimately decides ``allowed=False``.Whichever field came from the tenant row is recorded so the resolver@@ -82,6 +83,7 @@ from urllib.parse import urlparseimport asyncpgimport structlog+from app.llm.routing import gateway_url, is_gateway_alias, resolve_api_keyfrom app.security.credential_vault import CredentialVaultError, get_vault# NOTE: ``app.investigator.ledger`` is imported lazily inside@@ -106,6 +108,10 @@ logger = structlog.get_logger()_DEFAULT_OPENAI_BASE = "https://api.openai.com"_DEFAULT_OPENAI_MODEL = "gpt-4o-mini"+#: The role the explain path's work belongs to. Its alias is the default model+#: when a gateway is configured, because ``gpt-4o-mini`` is not a model the+#: bundled gateway defines and sending it there returns "Invalid model name".+_EXPLAIN_ROLE_ALIAS = "aisoc-summary"@dataclass(frozen=True)@@ -138,6 +144,16 @@ class LlmConfig:api_key: str | Nonesource: strreason: str+ # Per-field provenance. ``source`` collapses the whole config into one+ # word, which is enough to log and not enough to act on: the auto-triage+ # worker binds this config as a per-tenant BYOK *override*, and without+ # knowing which fields the tenant actually set it overrode the task role's+ # model with whatever ``OPENAI_MODEL`` happened to be — shipped in+ # .env.example as `gpt-4-turbo-preview`, which no gateway here defines. An+ # env baseline is a default, never an override.+ model_from_tenant: bool = False+ base_url_from_tenant: bool = False+ api_key_from_tenant: bool = False# ---------------------------------------------------------------------------@@ -157,7 +173,15 @@ def _env_baseline() -> tuple[str, str, str | None]:"""base_url = os.getenv("OPENAI_BASE_URL", "").strip() or os.getenv("LLM_BASE_URL", "").strip()model = os.getenv("OPENAI_MODEL", "").strip() or os.getenv("LLM_MODEL", "").strip() or os.getenv("AISOC_LLM_MODEL", "").strip()- api_key = os.getenv("OPENAI_API_KEY", "").strip() or os.getenv("LLM_API_KEY", "").strip()+ if not base_url and gateway_url() and is_gateway_alias(model or _EXPLAIN_ROLE_ALIAS):+ # The compose-provided gateway, under the same rule every other caller+ # uses (``app.llm.routing``): adopted for an alias, never for a concrete+ # model. Without this the explain path was the one consumer that could+ # not reach the gateway at all — it defaulted straight to+ # ``https://api.openai.com`` and took an alias there.+ base_url = gateway_url() or ""+ model = model or _EXPLAIN_ROLE_ALIAS+ api_key = resolve_api_key(model) or ""return base_url, model, (api_key or None)@@ -388,6 +412,9 @@ async def resolve_llm_config(tenant_ref: str | None) -> LlmConfig:api_key=None,source=source if source != "none" else "none",reason="no API key configured (neither tenant BYOK nor env)",+ model_from_tenant=tenant_contributed_model,+ base_url_from_tenant=tenant_contributed_base_url,+ api_key_from_tenant=tenant_contributed_key,)blocked, reason = _airgap_blocks(base_url)@@ -399,6 +426,9 @@ async def resolve_llm_config(tenant_ref: str | None) -> LlmConfig:api_key=api_key, # safe: caller does not log this on failuresource=source,reason=reason,+ model_from_tenant=tenant_contributed_model,+ base_url_from_tenant=tenant_contributed_base_url,+ api_key_from_tenant=tenant_contributed_key,)return LlmConfig(@@ -408,6 +438,9 @@ async def resolve_llm_config(tenant_ref: str | None) -> LlmConfig:api_key=api_key,source=source,reason="",+ model_from_tenant=tenant_contributed_model,+ base_url_from_tenant=tenant_contributed_base_url,+ api_key_from_tenant=tenant_contributed_key,)diff --git a/services/agents/app/tools/mitre_full.py b/services/agents/app/tools/mitre_full.pyindex 40567a2e4..0db583d07 100644--- a/services/agents/app/tools/mitre_full.py+++ b/services/agents/app/tools/mitre_full.py@@ -34,6 +34,22 @@ ATTCK_DATA_PATH = os.getenv("ATTCK_DATA_PATH", "/data/enterprise-attack.json")_CACHE_TTL_HOURS = 24_EMBED_BATCH_SIZE = 50+#: Embeddings deliberately do NOT follow chat traffic to the bundled LiteLLM+#: gateway. ``infra/litellm/config.yaml`` declares seven *chat* aliases and no+#: embedding model, so routing here would 400 on every batch — and a gateway+#: master key sent to api.openai.com 401s just as surely. This path therefore+#: keeps its own pair of variables, defaulting to the provider.+#: ``scripts/check_llm_model_routing.py`` records embeddings as out of the+#: gateway's scope, so the exclusion is checked rather than assumed.+_DEFAULT_EMBEDDING_MODEL = "text-embedding-3-large"+++def _embedding_client(openai_module: Any, api_key: str) -> tuple[Any, str]:+ """Return ``(client, model)`` for the embedding endpoint."""+ base_url = os.getenv("AISOC_EMBEDDING_BASE_URL", "").strip() or None+ model = os.getenv("AISOC_EMBEDDING_MODEL", "").strip() or _DEFAULT_EMBEDDING_MODEL+ return openai_module.AsyncOpenAI(api_key=api_key, base_url=base_url), model+# ─── Data Model ───────────────────────────────────────────────────────────────@@ -339,8 +355,13 @@ async def embed_techniques_into_qdrant(logger.warning("Qdrant/OpenAI client not available for ATT&CK embedding", error=str(exc))return- oai = openai.AsyncOpenAI(api_key=openai_api_key)+ oai, embed_model = _embedding_client(openai, openai_api_key)qdrant = AsyncQdrantClient(url=qdrant_url)+ logger.info(+ "ATT&CK embedding target",+ base_url=os.getenv("AISOC_EMBEDDING_BASE_URL", "").strip() or "provider default",+ model=embed_model,+ )# Ensure collection existstry:@@ -362,7 +383,7 @@ async def embed_techniques_into_qdrant(texts = [f"{t.id} {t.name}: {t.description or ''} Tactics: {', '.join(t.tactic_names or [])}" for t in batch]try:resp = await oai.embeddings.create(- model="text-embedding-3-large",+ model=embed_model,input=texts,dimensions=3072,)@@ -412,12 +433,12 @@ async def semantic_technique_search(except ImportError:return []- oai = openai.AsyncOpenAI(api_key=openai_api_key)+ oai, embed_model = _embedding_client(openai, openai_api_key)qdrant = AsyncQdrantClient(url=qdrant_url)try:emb_resp = await oai.embeddings.create(- model="text-embedding-3-large",+ model=embed_model,input=[query],dimensions=3072,)diff --git a/services/agents/app/workers/fused_alert_consumer.py b/services/agents/app/workers/fused_alert_consumer.pyindex fd1b7ebcc..6ce70d98c 100644--- a/services/agents/app/workers/fused_alert_consumer.py+++ b/services/agents/app/workers/fused_alert_consumer.py@@ -427,7 +427,23 @@ class FusedAlertTriageWorker:if use_llm and cfg is not None:# Route the LLM call through the tenant's BYOK key/model so# auto-triage actually honours per-tenant credentials.- with llm_override(api_key=cfg.api_key, base_url=cfg.base_url, model=cfg.model):+ #+ # Only the fields the *tenant* set are overrides. The+ # resolver also carries an env baseline, and passing that+ # through here replaced the `triage` role's gateway alias+ # with whatever `OPENAI_MODEL` held — `gpt-4-turbo-preview`+ # in the shipped .env.example, which the gateway 400s. An+ # env default is not a per-tenant override.+ # The key needs the same treatment: the env baseline's key+ # is a *provider* key, and forcing it here would send it to+ # the gateway as a bearer token, which 401s. Left unset,+ # make_chat_model resolves the key that pairs with the route+ # it actually chose.+ with llm_override(+ api_key=cfg.api_key if getattr(cfg, "api_key_from_tenant", False) else None,+ base_url=cfg.base_url if getattr(cfg, "base_url_from_tenant", False) else None,+ model=cfg.model if getattr(cfg, "model_from_tenant", False) else None,+ ):state, tier = await self._llm_triage(state)else:state = await run_triage(state)@@ -701,10 +717,14 @@ class FusedAlertTriageWorker:"""Resolve the tenant's LLM config, or None to force deterministic triage."""try:cfg = await resolve_llm_config(str(tenant_id))- return cfg if (cfg.allowed and cfg.api_key) else Noneexcept Exception as exc: # noqa: BLE001 — resolver failure => deterministiclogger.debug("auto_triage_worker.llm_resolve_failed", error=str(exc))return None+ # The resolver's env baseline now resolves the bearer that pairs with+ # the route it chose, so a deployment doing exactly what the compose+ # file sets up — gateway URL and master key, provider key held by the+ # gateway — reports a usable key here instead of "none configured".+ return cfg if (cfg.allowed and cfg.api_key) else Noneasync def _llm_triage(self, state: InvestigationState) -> tuple[InvestigationState, str]:try:diff --git a/services/agents/tests/test_llm_factory.py b/services/agents/tests/test_llm_factory.pyindex bed8d2783..3c010a5c8 100644--- a/services/agents/tests/test_llm_factory.py+++ b/services/agents/tests/test_llm_factory.py@@ -4,11 +4,16 @@ from __future__ import annotationsimport pytestfrom app.llm.factory import (+ UnroutableModelError,chat_completions_url,+ llm_override,make_chat_model,+ preflight_llm,+ resolve_api_key,resolve_base_url,resolve_model_alias,)+from app.llm.model_pins import all_roles# app.llm.contract.DEFAULT_OPENAI_CHAT_COMPLETIONS_URL_DEFAULT_URL = "https://api.openai.com/v1/chat/completions"@@ -21,8 +26,13 @@ def _clean_llm_env(monkeypatch):"OPENAI_BASE_URL","LLM_BASE_URL","LLM_GATEWAY_URL",- "AISOC_MODEL_PIN_TRIAGE",- "AISOC_MODEL_PIN_NL",+ "LITELLM_MASTER_KEY",+ "OPENAI_API_KEY",+ "LLM_API_KEY",+ "OPENAI_MODEL",+ "LLM_MODEL",+ "AISOC_LLM_MODEL",+ *(f"AISOC_MODEL_PIN_{role.upper()}" for role in all_roles()),):monkeypatch.delenv(var, raising=False)@@ -57,11 +67,52 @@ def test_base_url_prefers_openai_then_llm(monkeypatch):assert resolve_base_url() == "http://gw:4000/v1"-def test_base_url_ignores_gateway_url_env(monkeypatch):- # Routing through the gateway must be explicit (OPENAI_BASE_URL), never- # silently adopted from the compose-provided LLM_GATEWAY_URL.+def test_base_url_adopts_gateway_url_for_an_alias(monkeypatch):+ # The compose-provided variable, which nothing read: an aisoc-<role> alias+ # resolves nowhere else, so the gateway is the only correct destination.+ # Sending it to the provider default is the 404 this whole module exists to+ # stop, and the fallback made it look like "no LLM available".monkeypatch.setenv("LLM_GATEWAY_URL", "http://litellm:4000/v1")- assert resolve_base_url() is None+ assert resolve_base_url("aisoc-triage") == "http://litellm:4000/v1"+ # No model named => every task role defaults to an alias, so same answer.+ assert resolve_base_url() == "http://litellm:4000/v1"+++def test_base_url_leaves_a_concrete_pinned_model_at_its_provider(monkeypatch):+ # The documented direct-to-provider escape hatch. Rerouting it to a gateway+ # that does not define `gpt-4o-mini` would break a working deployment.+ monkeypatch.setenv("LLM_GATEWAY_URL", "http://litellm:4000/v1")+ assert resolve_base_url("gpt-4o-mini") is None+++def test_explicit_base_url_outranks_the_gateway(monkeypatch):+ monkeypatch.setenv("LLM_GATEWAY_URL", "http://litellm:4000/v1")+ monkeypatch.setenv("OPENAI_BASE_URL", "http://my-vllm:8000/v1")+ assert resolve_base_url("aisoc-triage") == "http://my-vllm:8000/v1"+ assert resolve_base_url("gpt-4o-mini") == "http://my-vllm:8000/v1"+++# ── resolve_api_key ──────────────────────────────────────────────────────────+++def test_api_key_is_the_master_key_when_we_pick_the_gateway(monkeypatch):+ # The route decides the bearer. A provider key sent to LiteLLM is rejected+ # as an invalid proxy token, so resolving the two independently is what+ # made "adopt the gateway URL" look ambiguous in the first place.+ monkeypatch.setenv("LLM_GATEWAY_URL", "http://litellm:4000/v1")+ monkeypatch.setenv("LITELLM_MASTER_KEY", "sk-gateway")+ monkeypatch.setenv("OPENAI_API_KEY", "sk-provider")+ assert resolve_api_key("aisoc-triage") == "sk-gateway"+ # …and the provider key for a model that goes straight to the provider.+ assert resolve_api_key("gpt-4o-mini") == "sk-provider"+++def test_api_key_is_the_operators_when_they_set_the_base_url(monkeypatch):+ monkeypatch.setenv("LLM_GATEWAY_URL", "http://litellm:4000/v1")+ monkeypatch.setenv("LITELLM_MASTER_KEY", "sk-gateway")+ monkeypatch.setenv("OPENAI_BASE_URL", "http://my-vllm:8000/v1")+ monkeypatch.setenv("OPENAI_API_KEY", "sk-provider")+ assert resolve_api_key("aisoc-triage") == "sk-provider"# ── chat_completions_url ─────────────────────────────────────────────────────@@ -76,6 +127,17 @@ def test_chat_completions_url_appends_path(monkeypatch):assert chat_completions_url() == "http://litellm:4000/v1/chat/completions"+def test_chat_completions_url_refuses_an_alias_with_no_gateway():+ # The loud failure. Previously this returned api.openai.com and the caller+ # took the resulting 404 as evidence that no model was available.+ with pytest.raises(UnroutableModelError, match="aisoc-triage"):+ chat_completions_url("aisoc-triage")+++def test_chat_completions_url_allows_a_concrete_model_with_no_gateway():+ assert chat_completions_url("gpt-4o-mini") == _DEFAULT_URL++# ── make_chat_model ──────────────────────────────────────────────────────────@@ -86,3 +148,75 @@ def test_make_chat_model_uses_alias_and_base_url(monkeypatch):# langchain_openai exposes the model id as ``model_name``.assert llm.model_name == "aisoc-triage"assert str(llm.openai_api_base).rstrip("/") == "http://litellm:4000/v1"+++def test_make_chat_model_reaches_the_gateway_from_compose_alone(monkeypatch):+ # The default deployment: compose sets LLM_GATEWAY_URL + LITELLM_MASTER_KEY+ # and nothing else. This is the state that could not reach a model.+ monkeypatch.setenv("LLM_GATEWAY_URL", "http://litellm:4000/v1")+ monkeypatch.setenv("LITELLM_MASTER_KEY", "sk-aisoc-local")+ llm = make_chat_model("triage")+ assert llm.model_name == "aisoc-triage"+ assert str(llm.openai_api_base).rstrip("/") == "http://litellm:4000/v1"+ assert llm.openai_api_key.get_secret_value() == "sk-aisoc-local"+++def test_make_chat_model_refuses_an_alias_with_no_gateway(monkeypatch):+ monkeypatch.setenv("OPENAI_API_KEY", "sk-test")+ with pytest.raises(UnroutableModelError, match="aisoc-triage"):+ make_chat_model("triage")+++def test_byok_model_override_only_when_the_tenant_set_one(monkeypatch):+ # The second defect: OPENAI_MODEL reached make_chat_model through the BYOK+ # override and replaced the role alias with a model the gateway 400s. The+ # override mechanism itself is fine — a real tenant model still wins.+ monkeypatch.setenv("LLM_GATEWAY_URL", "http://litellm:4000/v1")+ monkeypatch.setenv("LITELLM_MASTER_KEY", "sk-aisoc-local")+ monkeypatch.setenv("OPENAI_MODEL", "gpt-4-turbo-preview")+ assert make_chat_model("triage").model_name == "aisoc-triage"+ with llm_override(model="tenant-model", base_url="http://tenant:1/v1", api_key="sk-tenant"):+ llm = make_chat_model("triage")+ assert llm.model_name == "tenant-model"+ assert str(llm.openai_api_base).rstrip("/") == "http://tenant:1/v1"+++# ── preflight_llm ────────────────────────────────────────────────────────────+++def test_preflight_clean_when_the_gateway_is_wired(monkeypatch):+ monkeypatch.setenv("LLM_GATEWAY_URL", "http://litellm:4000/v1")+ monkeypatch.setenv("OPENAI_MODEL", "aisoc-summary")+ assert preflight_llm() == []+++def test_preflight_reports_an_alias_with_no_gateway():+ warnings = preflight_llm()+ assert len(warnings) == 1+ assert "aisoc-triage" in warnings[0]+++def test_preflight_reports_a_concrete_pin_aimed_at_the_bundled_gateway(monkeypatch):+ # The reverse direction, which is the failure the commercial deployment hit:+ # every call 400'd with "Invalid model name" and degraded to stub output.+ monkeypatch.setenv("LLM_GATEWAY_URL", "http://litellm:4000/v1")+ monkeypatch.setenv("OPENAI_BASE_URL", "http://litellm:4000/v1")+ monkeypatch.setenv("AISOC_MODEL_PIN_TRIAGE", "gpt-4-turbo-preview")+ warnings = preflight_llm()+ assert any("gpt-4-turbo-preview" in w and "does not define" in w for w in warnings)+++def test_preflight_reports_a_byok_model_the_gateway_lacks(monkeypatch):+ monkeypatch.setenv("LLM_GATEWAY_URL", "http://litellm:4000/v1")+ monkeypatch.setenv("OPENAI_MODEL", "gpt-4-turbo-preview")+ warnings = preflight_llm()+ assert any("OPENAI_MODEL" in w and "gpt-4-turbo-preview" in w for w in warnings)+++def test_preflight_quiet_about_a_third_party_gateway(monkeypatch):+ # An operator pointing at their own vLLM with concrete model names is+ # correct; only the bundled gateway's contents are knowable from here.+ monkeypatch.setenv("OPENAI_BASE_URL", "http://my-vllm:8000/v1")+ monkeypatch.setenv("AISOC_MODEL_PIN_TRIAGE", "Qwen2.5-32B-Instruct")+ monkeypatch.setenv("OPENAI_MODEL", "Qwen2.5-32B-Instruct")+ assert preflight_llm() == []diff --git a/services/agents/tests/test_llm_hot_path.py b/services/agents/tests/test_llm_hot_path.pyindex 5961f63c1..5fa86c014 100644--- a/services/agents/tests/test_llm_hot_path.py+++ b/services/agents/tests/test_llm_hot_path.py@@ -50,8 +50,12 @@ def test_llm_override_routes_to_tenant_model(monkeypatch):with llm_override(api_key="tenant-byok-key", base_url="https://byok.example/v1", model="gpt-4o-tenant"):m = make_chat_model("triage")assert getattr(m, "model_name", None) == "gpt-4o-tenant"- # Outside the override, back to the alias (platform key from env, as in prod).+ # Outside the override, back to the alias — which needs the gateway that+ # docker-compose.yml supplies. A provider key alone is not enough to route+ # an alias, and building a client that will 404 is what used to look like+ # "the LLM is configured".monkeypatch.setenv("OPENAI_API_KEY", "sk-platform-test")+ monkeypatch.setenv("LLM_GATEWAY_URL", "http://litellm:4000/v1")m2 = make_chat_model("triage")assert getattr(m2, "model_name", None) == "aisoc-triage"diff --git a/services/api/app/_vendor/nl_query/translator.py b/services/api/app/_vendor/nl_query/translator.pyindex 5de992be3..98e4f6eb5 100644--- a/services/api/app/_vendor/nl_query/translator.py+++ b/services/api/app/_vendor/nl_query/translator.py@@ -930,11 +930,12 @@ async def enhance_with_llm({"role": "user", "content": prompt},]+ resolved_model = model or resolve_model_alias("nl")payload = await safe_chat_completions_request(api_key=api_key,- model=model or resolve_model_alias("nl"),+ model=resolved_model,messages=messages,- url=chat_completions_url(),+ url=chat_completions_url(resolved_model),timeout=timeout,response_format={"type": "json_object"},temperature=0,diff --git a/services/api/app/api/v1/endpoints/hunts.py b/services/api/app/api/v1/endpoints/hunts.pyindex 61cb6e24d..0deb48459 100644--- a/services/api/app/api/v1/endpoints/hunts.py+++ b/services/api/app/api/v1/endpoints/hunts.py@@ -39,7 +39,7 @@ from sqlalchemy import textfrom app.api.v1.deps import AuthUser, DBSessionfrom app.core.airgap import AirgapViolation, enforce_airgap_for_urlfrom app.services.llm_safety import LLMContractViolation, safe_chat_completions_request-from app.services.model_aliases import resolve_model_alias+from app.services.model_aliases import chat_completions_url, resolve_api_key, resolve_model_aliaslogger = logging.getLogger(__name__)@@ -137,16 +137,17 @@ No prose. Just JSON."""async def _generate_queries(hypothesis: str, mitre: str | None) -> dict[str, str] | None:- api_key = os.getenv("OPENAI_API_KEY") or os.getenv("LLM_API_KEY")+ model = os.getenv("LLM_MODEL") or resolve_model_alias("nl")+ # Resolved together with the route: when the call goes to the bundled+ # gateway the bearer is the gateway's master key, not a provider key.+ api_key = resolve_api_key(model)if not api_key:return None- base_url = os.getenv("LLM_BASE_URL", "https://api.openai.com/v1")- model = os.getenv("LLM_MODEL") or resolve_model_alias("nl")user_msg = f"HYPOTHESIS: {hypothesis}"if mitre:user_msg += f"\nMITRE TECHNIQUE: {mitre}"user_msg += "\nGenerate ES|QL, SPL, and KQL hunt queries."- completions_url = f"{base_url}/chat/completions"+ completions_url = chat_completions_url(model)enforce_airgap_for_url(completions_url)try:body = await safe_chat_completions_request( # T2.3diff --git a/services/api/app/api/v1/endpoints/knowledge_base.py b/services/api/app/api/v1/endpoints/knowledge_base.pyindex 1f1a6e133..8ccfdd78c 100644--- a/services/api/app/api/v1/endpoints/knowledge_base.py+++ b/services/api/app/api/v1/endpoints/knowledge_base.py@@ -29,7 +29,7 @@ from app.api.v1.deps import AuthUser, DBSessionfrom app.core.airgap import AirgapViolation, enforce_airgap_for_urlfrom app.services.kb_chunking import chunk_textfrom app.services.llm_safety import LLMContractViolation, safe_chat_completions_request-from app.services.model_aliases import resolve_model_alias+from app.services.model_aliases import chat_completions_url, resolve_api_key, resolve_model_aliaslogger = logging.getLogger(__name__)@@ -112,13 +112,14 @@ If the answer cannot be found in the context, say so clearly."""async def _synthesise(question: str, chunks: list[KBChunk]) -> str | None:- api_key = os.getenv("OPENAI_API_KEY") or os.getenv("LLM_API_KEY")+ model = os.getenv("LLM_MODEL") or resolve_model_alias("nl")+ # Resolved together with the route: when the call goes to the bundled+ # gateway the bearer is the gateway's master key, not a provider key.+ api_key = resolve_api_key(model)if not api_key:return None- base_url = os.getenv("LLM_BASE_URL", "https://api.openai.com/v1")- model = os.getenv("LLM_MODEL") or resolve_model_alias("nl")context = "\n\n".join(f"[{c.title}] chunk {c.chunk_index}:\n{c.content}" for c in chunks)- completions_url = f"{base_url}/chat/completions"+ completions_url = chat_completions_url(model)enforce_airgap_for_url(completions_url)try:# T2.3 — the retrieved KB chunks are the untrusted half here: adiff --git a/services/api/app/api/v1/endpoints/llm_status.py b/services/api/app/api/v1/endpoints/llm_status.pyindex 51f969a50..a758b90e0 100644--- a/services/api/app/api/v1/endpoints/llm_status.py+++ b/services/api/app/api/v1/endpoints/llm_status.py@@ -66,6 +66,7 @@ from app.core.airgap import is_host_allowed_for_airgapfrom app.core.config import settingsfrom app.models.llm_credential import TenantLlmCredentialfrom app.security.credential_vault import CredentialVaultError, get_vault+from app.services.model_aliases import gateway_url, is_gateway_alias, resolve_api_keyrouter = APIRouter(prefix="/llm", tags=["llm"])@@ -145,7 +146,15 @@ def _env_baseline() -> tuple[str, str, bool]:"""base_url = os.getenv("LLM_BASE_URL") or os.getenv("OPENAI_BASE_URL") or ""model = os.getenv("LLM_MODEL") or os.getenv("OPENAI_MODEL") or os.getenv("AISOC_LLM_MODEL") or ""- key_set = bool(os.getenv("OPENAI_API_KEY") or os.getenv("LLM_API_KEY"))+ if not base_url and gateway_url() and is_gateway_alias(model or "aisoc-summary"):+ # The indicator has to describe the route the explain path will+ # actually take, and that path now adopts the compose-provided gateway+ # for an alias-shaped model. Reporting "no LLM configured" while the+ # request goes to the gateway would be the same class of defect one+ # level up.+ base_url = gateway_url() or ""+ model = model or "aisoc-summary"+ key_set = bool(resolve_api_key(model))return base_url, model, key_setdiff --git a/services/api/app/api/v1/endpoints/nl_detection.py b/services/api/app/api/v1/endpoints/nl_detection.pyindex 1d801dfa7..323e7acc9 100644--- a/services/api/app/api/v1/endpoints/nl_detection.py+++ b/services/api/app/api/v1/endpoints/nl_detection.py@@ -14,7 +14,6 @@ functional in all deployment environments.from __future__ import annotationsimport json-import osimport uuidfrom datetime import UTC, datetimefrom typing import Annotated, Any, Literal@@ -29,6 +28,12 @@ from app.models.detection_proposal import DetectionRuleProposalfrom app.services.detection_eval import evaluate_candidate_rulefrom app.services.fixture_synth import derive_fixtures_from_sigmafrom app.services.llm_safety import LLMContractViolation, safe_chat_completions_request+from app.services.model_aliases import (+ UnroutableModelError,+ chat_completions_url,+ resolve_api_key,+ resolve_model_alias,+)logger = structlog.get_logger()@@ -122,9 +127,16 @@ FROM logs-*async def _llm_translate(request: NLDetectionRequest) -> dict[str, str | None]:"""Attempt LLM-based translation; fall back to templates on any error."""- api_key = os.environ.get("OPENAI_API_KEY", "").strip()+ # The detection builder asked ``OPENAI_MODEL`` for its model and posted to a+ # hardcoded api.openai.com — so it was the one LLM consumer that could not be+ # pointed anywhere, and the model it sent (`gpt-4-turbo-preview`, from+ # .env.example) is not one the bundled gateway defines. It is a+ # natural-language translation task, so it takes the `nl` role like every+ # other one.+ model = resolve_model_alias("nl")+ api_key = resolve_api_key(model) or ""if not api_key:- logger.debug("nl_detection.llm_unavailable", reason="no OPENAI_API_KEY")+ logger.debug("nl_detection.llm_unavailable", reason="no API key resolved for the nl role")return _template_fallback(request)platforms_str = ", ".join(request.target_platforms)@@ -136,14 +148,17 @@ async def _llm_translate(request: NLDetectionRequest) -> dict[str, str | None]:f"Return JSON with keys matching the platform names (sigma, kql, spl, esql).")- completions_url = "https://api.openai.com/v1/chat/completions"+ try:+ completions_url = chat_completions_url(model)+ except UnroutableModelError as exc:+ logger.warning("nl_detection.unroutable_model", model=model, reason=str(exc))+ return _template_fallback(request)try:enforce_airgap_for_url(completions_url)except AirgapViolation as exc:logger.info("nl_detection.airgap_block", url=completions_url, reason=str(exc))return _template_fallback(request)- model = os.environ.get("OPENAI_MODEL", "gpt-4o-mini")try:# T2.3 — the description is analyst-authored but routinely contains a# pasted log line, which is exactly what the contract refuses todiff --git a/services/api/app/api/v1/endpoints/phishing.py b/services/api/app/api/v1/endpoints/phishing.pyindex a336f34de..92606da71 100644--- a/services/api/app/api/v1/endpoints/phishing.py+++ b/services/api/app/api/v1/endpoints/phishing.py@@ -28,7 +28,7 @@ from sqlalchemy import textfrom app.api.v1.deps import AuthUser, DBSessionfrom app.core.airgap import AirgapViolation, enforce_airgap_for_urlfrom app.services.llm_safety import LLMContractViolation, safe_chat_completions_request-from app.services.model_aliases import resolve_model_alias+from app.services.model_aliases import chat_completions_url, resolve_api_key, resolve_model_aliaslogger = logging.getLogger(__name__)@@ -89,17 +89,18 @@ Analyse the submitted artifact and return ONLY valid JSON with:async def _triage(artifact_kind: str, content: str, urls: list[str]) -> TriageResult | None:- api_key = os.getenv("OPENAI_API_KEY") or os.getenv("LLM_API_KEY")+ model = os.getenv("LLM_MODEL") or resolve_model_alias("investigation")+ # Resolved together with the route: when the call goes to the bundled+ # gateway the bearer is the gateway's master key, not a provider key.+ api_key = resolve_api_key(model)if not api_key:return None- base_url = os.getenv("LLM_BASE_URL", "https://api.openai.com/v1")- model = os.getenv("LLM_MODEL") or resolve_model_alias("investigation")user_msg = f"ARTIFACT TYPE: {artifact_kind}\n"if urls:user_msg += f"URLS: {', '.join(urls[:10])}\n"if content:user_msg += f"CONTENT (first 2000 chars):\n{content[:2000]}"- completions_url = f"{base_url}/chat/completions"+ completions_url = chat_completions_url(model)# Air-gap enforcement: refuse the call rather than letting httpx fan out.# AirgapViolation propagates to the caller so the endpoint can surface 503.enforce_airgap_for_url(completions_url)diff --git a/services/api/app/api/v1/endpoints/translation.py b/services/api/app/api/v1/endpoints/translation.pyindex 77760b7bd..3814f3046 100644--- a/services/api/app/api/v1/endpoints/translation.py+++ b/services/api/app/api/v1/endpoints/translation.py@@ -24,7 +24,7 @@ from pydantic import BaseModel, Fieldfrom app.api.v1.deps import AuthUserfrom app.core.airgap import AirgapViolation, enforce_airgap_for_urlfrom app.services.llm_safety import LLMContractViolation, safe_chat_completions_request-from app.services.model_aliases import resolve_model_alias+from app.services.model_aliases import chat_completions_url, resolve_api_key, resolve_model_aliaslogger = logging.getLogger(__name__)@@ -112,12 +112,13 @@ def _user_prompt(req: TranslateRequest) -> str:async def _llm_translate(req: TranslateRequest) -> dict[str, Any] | None:- api_key = os.getenv("OPENAI_API_KEY") or os.getenv("LLM_API_KEY")+ model = os.getenv("LLM_MODEL") or resolve_model_alias("nl")+ # Resolved together with the route: when the call goes to the bundled+ # gateway the bearer is the gateway's master key, not a provider key.+ api_key = resolve_api_key(model)if not api_key:return None- base_url = os.getenv("LLM_BASE_URL", "https://api.openai.com/v1")- model = os.getenv("LLM_MODEL") or resolve_model_alias("nl")- completions_url = f"{base_url}/chat/completions"+ completions_url = chat_completions_url(model)# Air-gap check: refuses the call entirely (rather than silently# falling back to template-substitution) so misconfigurations are# loud. The except-Exception below would otherwise swallow this.diff --git a/services/api/app/services/llm_resolver.py b/services/api/app/services/llm_resolver.pyindex 29d2e4df4..2bd0c3c18 100644--- a/services/api/app/services/llm_resolver.py+++ b/services/api/app/services/llm_resolver.py@@ -45,12 +45,17 @@ from sqlalchemy.ext.asyncio import AsyncSessionfrom app.models.llm_credential import TenantLlmCredentialfrom app.security.credential_vault import CredentialVaultError, get_vault+from app.services.model_aliases import gateway_url, is_gateway_alias, resolve_api_keylogger = logging.getLogger(__name__)_DEFAULT_OPENAI_BASE = "https://api.openai.com"_DEFAULT_OPENAI_MODEL = "gpt-4o-mini"+#: See the agents-side resolver: the explain path's work is a summary, and its+#: alias is the default model when a gateway is configured, because the bundled+#: gateway does not define ``gpt-4o-mini``.+_EXPLAIN_ROLE_ALIAS = "aisoc-summary"@dataclass(frozen=True)@@ -80,6 +85,14 @@ class LlmConfig:api_key: str | Nonesource: strreason: str+ # Per-field provenance — mirrors the agents-side resolver. ``source``+ # collapses the whole config into one word, which is enough to log and not+ # enough to act on: a caller that binds this as a BYOK *override* needs to+ # know which fields the tenant actually set, or an env baseline silently+ # overrides a per-role model pin.+ model_from_tenant: bool = False+ base_url_from_tenant: bool = False+ api_key_from_tenant: bool = Falsedef _env_baseline() -> tuple[str, str, str | None]:@@ -94,7 +107,14 @@ def _env_baseline() -> tuple[str, str, str | None]:"""base_url = os.getenv("OPENAI_BASE_URL", "").strip() or os.getenv("LLM_BASE_URL", "").strip()model = os.getenv("OPENAI_MODEL", "").strip() or os.getenv("LLM_MODEL", "").strip() or os.getenv("AISOC_LLM_MODEL", "").strip()- api_key = os.getenv("OPENAI_API_KEY", "").strip() or os.getenv("LLM_API_KEY", "").strip()+ if not base_url and gateway_url() and is_gateway_alias(model or _EXPLAIN_ROLE_ALIAS):+ # Same rule as every other caller (``app.services.model_aliases``):+ # the compose-provided gateway is adopted for an alias and never for a+ # concrete model. Without this the explain path defaulted straight to+ # ``https://api.openai.com`` and took an alias there.+ base_url = gateway_url() or ""+ model = model or _EXPLAIN_ROLE_ALIAS+ api_key = resolve_api_key(model) or ""return base_url, model, (api_key or None)@@ -256,6 +276,9 @@ async def resolve_llm_config(db: AsyncSession, tenant_id: uuid.UUID) -> LlmConfiapi_key=None,source=source,reason="no API key configured (neither tenant BYOK nor env)",+ model_from_tenant=tenant_contributed_model,+ base_url_from_tenant=tenant_contributed_base_url,+ api_key_from_tenant=tenant_contributed_key,)blocked, reason = _airgap_blocks(base_url)@@ -269,6 +292,9 @@ async def resolve_llm_config(db: AsyncSession, tenant_id: uuid.UUID) -> LlmConfiapi_key=None,source=source,reason=reason,+ model_from_tenant=tenant_contributed_model,+ base_url_from_tenant=tenant_contributed_base_url,+ api_key_from_tenant=tenant_contributed_key,)return LlmConfig(@@ -278,4 +304,7 @@ async def resolve_llm_config(db: AsyncSession, tenant_id: uuid.UUID) -> LlmConfiapi_key=api_key,source=source,reason="",+ model_from_tenant=tenant_contributed_model,+ base_url_from_tenant=tenant_contributed_base_url,+ api_key_from_tenant=tenant_contributed_key,)diff --git a/services/api/app/services/model_aliases.py b/services/api/app/services/model_aliases.pyindex 9893a09e1..b92132d2a 100644--- a/services/api/app/services/model_aliases.py+++ b/services/api/app/services/model_aliases.py@@ -22,6 +22,24 @@ ROLES = frozenset({"triage", "recon", "investigation", "copilot", "summary", "reDEFAULT_OPENAI_CHAT_COMPLETIONS_URL = "https://api.openai.com/v1/chat/completions"+#: See :data:`services.agents.app.llm.routing.GATEWAY_ALIAS_PREFIX`.+GATEWAY_ALIAS_PREFIX = "aisoc-"++#: See :data:`services.agents.app.llm.routing.BYOK_MODEL_ENV_VARS`.+BYOK_MODEL_ENV_VARS = ("OPENAI_MODEL", "LLM_MODEL", "AISOC_LLM_MODEL")+++class UnroutableModelError(RuntimeError):+ """A gateway alias was requested with nowhere to send it.++ Mirror of :class:`services.agents.app.llm.factory.UnroutableModelError`.+ """+++def is_gateway_alias(model: str | None) -> bool:+ """Whether ``model`` is a logical alias that only the gateway can resolve."""+ return bool(model) and str(model).strip().startswith(GATEWAY_ALIAS_PREFIX)+def resolve_model_alias(role: str) -> str:"""Return the ``aisoc-<role>`` alias, honouring an ``AISOC_MODEL_PIN_<ROLE>`` override."""@@ -29,18 +47,77 @@ def resolve_model_alias(role: str) -> str:return override or f"aisoc-{role}"-def resolve_base_url() -> str | None:- """OpenAI-compatible base URL for live calls, or ``None`` for the default.+def _explicit_base_url() -> str | None:+ """The base URL an operator set by hand, if any."""+ return os.environ.get("OPENAI_BASE_URL", "").strip() or os.environ.get("LLM_BASE_URL", "").strip() or None+++def gateway_url() -> str | None:+ """The bundled LiteLLM gateway's in-network URL, as compose supplies it."""+ return os.environ.get("LLM_GATEWAY_URL", "").strip() or None- Mirrors :func:`services.agents.app.llm.factory.resolve_base_url`, including- its deliberate refusal to auto-adopt ``LLM_GATEWAY_URL``: routing through- the gateway is an explicit choice so it is never ambiguous whether the- bearer token is the gateway master key or a provider key.++def resolve_base_url(model: str | None = None) -> str | None:+ """OpenAI-compatible base URL for ``model``, or ``None`` for the default.++ Mirrors :func:`services.agents.app.llm.factory.resolve_base_url` exactly,+ including the rule that ``LLM_GATEWAY_URL`` — the only base-URL variable+ ``docker-compose.yml`` sets — is adopted for an alias-shaped model and left+ alone for a concrete pinned one."""- return os.environ.get("OPENAI_BASE_URL", "").strip() or os.environ.get("LLM_BASE_URL", "").strip() or None+ explicit = _explicit_base_url()+ if explicit:+ return explicit+ if model is None or is_gateway_alias(model):+ return gateway_url()+ return None+++def explicit_base_url() -> str | None:+ """The base URL an operator set by hand, if any."""+ return _explicit_base_url()+++def adopted_gateway(model: str | None = None) -> bool:+ """Whether the base URL for ``model`` came from ``LLM_GATEWAY_URL`` rather than an operator."""+ return _explicit_base_url() is None and resolve_base_url(model) is not None+++def at_bundled_gateway(base_url: str | None) -> bool:+ """Whether ``base_url`` is the gateway this repo ships a model list for.++ Mirrors :func:`services.agents.app.llm.routing.at_bundled_gateway`.+ """+ gateway = gateway_url()+ if not base_url or not gateway:+ return False+ return str(base_url).rstrip("/") == gateway.rstrip("/")+++def resolve_api_key(model: str | None = None) -> str | None:+ """Bearer token to send alongside :func:`resolve_base_url`'s answer.++ Mirrors :func:`services.agents.app.llm.factory.resolve_api_key`: when AiSOC+ picks the bundled gateway itself the bearer is ``LITELLM_MASTER_KEY``, never+ a provider key.+ """+ if adopted_gateway(model):+ return os.environ.get("LITELLM_MASTER_KEY", "").strip() or os.environ.get("OPENAI_API_KEY", "").strip() or None+ return os.environ.get("OPENAI_API_KEY", "").strip() or os.environ.get("LLM_API_KEY", "").strip() or None+++def assert_routable(model: str, base_url: str | None) -> None:+ """Raise :class:`UnroutableModelError` if ``model`` can never resolve at ``base_url``."""+ if is_gateway_alias(model) and not base_url:+ raise UnroutableModelError(+ f"model '{model}' is a LiteLLM gateway alias and no gateway is configured. "+ "Set LLM_GATEWAY_URL (docker-compose.yml already does) or OPENAI_BASE_URL to the "+ "gateway, or pin a concrete provider model via AISOC_MODEL_PIN_<ROLE>. "+ "Sending an alias to a provider default returns 404 and degrades silently."+ )-def chat_completions_url() -> str:+def chat_completions_url(model: str | None = None) -> str:"""Full chat-completions URL for the raw-HTTP path.Exists so the NL-query translator — one file vendored into both services —@@ -50,7 +127,9 @@ def chat_completions_url() -> str:``/nl-query`` silently never reached a model and always returned thedeterministic translation: safe by accident, and invisible."""- base = resolve_base_url()+ base = resolve_base_url(model)+ if model is not None:+ assert_routable(model, base)if base:return base.rstrip("/") + "/chat/completions"return DEFAULT_OPENAI_CHAT_COMPLETIONS_URLdiff --git a/services/api/tests/test_llm_status.py b/services/api/tests/test_llm_status.pyindex 610c64529..af7f3496d 100644--- a/services/api/tests/test_llm_status.py+++ b/services/api/tests/test_llm_status.py@@ -36,6 +36,11 @@ def _clear_llm_env(monkeypatch: pytest.MonkeyPatch):"OPENAI_API_KEY","OPENAI_MODEL","AISOC_LLM_MODEL",+ # The gateway pair the default deployment supplies. The indicator now+ # reports the route the explain path would actually take, so leaving+ # these set would make "nothing configured" describe a live gateway.+ "LLM_GATEWAY_URL",+ "LITELLM_MASTER_KEY",):monkeypatch.delenv(var, raising=False)monkeypatch.setattr(airgap_module.settings, "AISOC_AIRGAPPED", False)diff --git a/services/api/tests/test_model_aliases_routing.py b/services/api/tests/test_model_aliases_routing.pynew file mode 100644index 000000000..d42ddc0ac--- /dev/null+++ b/services/api/tests/test_model_aliases_routing.py@@ -0,0 +1,82 @@+"""The API service's half of the gateway routing rule.++``services/api`` is a separate package from ``services/agents`` and cannot+import its factory, so ``app.services.model_aliases`` mirrors+``app.llm.routing``. Two copies of a rule drift, and the drift is invisible+until a deployment behaves differently in one service than the other — so the+same behaviours are asserted on both sides, and+``scripts/check_llm_model_routing.py`` compares the two module surfaces.+"""++from __future__ import annotations++import pytest+from app.services.model_aliases import (+ ROLES,+ UnroutableModelError,+ chat_completions_url,+ is_gateway_alias,+ resolve_api_key,+ resolve_base_url,+ resolve_model_alias,+)++_DEFAULT_URL = "https://api.openai.com/v1/chat/completions"++[email protected](autouse=True)+def _clean_llm_env(monkeypatch):+ for var in (+ "OPENAI_BASE_URL",+ "LLM_BASE_URL",+ "LLM_GATEWAY_URL",+ "LITELLM_MASTER_KEY",+ "OPENAI_API_KEY",+ "LLM_API_KEY",+ *(f"AISOC_MODEL_PIN_{role.upper()}" for role in ROLES),+ ):+ monkeypatch.delenv(var, raising=False)+++def test_alias_shape_matches_the_roles_the_gateway_defines():+ for role in ROLES:+ assert resolve_model_alias(role) == f"aisoc-{role}"+ assert is_gateway_alias(resolve_model_alias(role))+++def test_base_url_adopts_the_compose_gateway_for_an_alias(monkeypatch):+ monkeypatch.setenv("LLM_GATEWAY_URL", "http://litellm:4000/v1")+ assert resolve_base_url("aisoc-nl") == "http://litellm:4000/v1"+++def test_base_url_leaves_a_concrete_model_at_its_provider(monkeypatch):+ monkeypatch.setenv("LLM_GATEWAY_URL", "http://litellm:4000/v1")+ assert resolve_base_url("gpt-4o-mini") is None+++def test_explicit_base_url_outranks_the_gateway(monkeypatch):+ monkeypatch.setenv("LLM_GATEWAY_URL", "http://litellm:4000/v1")+ monkeypatch.setenv("LLM_BASE_URL", "http://my-vllm:8000/v1")+ assert resolve_base_url("aisoc-nl") == "http://my-vllm:8000/v1"+++def test_api_key_pairs_with_the_route(monkeypatch):+ monkeypatch.setenv("LLM_GATEWAY_URL", "http://litellm:4000/v1")+ monkeypatch.setenv("LITELLM_MASTER_KEY", "sk-gateway")+ monkeypatch.setenv("OPENAI_API_KEY", "sk-provider")+ assert resolve_api_key("aisoc-nl") == "sk-gateway"+ assert resolve_api_key("gpt-4o-mini") == "sk-provider"+++def test_completions_url_refuses_an_alias_with_no_gateway():+ with pytest.raises(UnroutableModelError, match="aisoc-nl"):+ chat_completions_url("aisoc-nl")+++def test_completions_url_allows_a_concrete_model_with_no_gateway():+ assert chat_completions_url("gpt-4o-mini") == _DEFAULT_URL+++def test_completions_url_appends_the_path_once(monkeypatch):+ monkeypatch.setenv("LLM_GATEWAY_URL", "http://litellm:4000/v1/")+ assert chat_completions_url("aisoc-nl") == "http://litellm:4000/v1/chat/completions"