Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
fix(api,agents,mcp): four surfaces that reported success while doing nothing (#1280, #1275, #1276, #1274) (#1289)
Beenu Arorade54e6426ee7
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 4eef9a5ad..ff0d53cda 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,63 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]+### BREAKING++- **`PATCH /api/v1/alerts/{id}` now refuses a field it cannot apply.** The+ request model set no `extra=`, so pydantic v2's default `extra="ignore"`+ dropped anything outside `status`, `priority`, `tags`, `assignee` and+ `case_id` and still answered **200** with the unchanged row — a success+ response for a write that did not happen (#1280). The dropped fields were+ not nonsense: `disposition`, `severity`, `confidence` and `snoozed_until`+ are all real `alerts` columns and three of them are writable elsewhere,+ which is why callers sent them. A caller closing an alert with a+ disposition got a 200 and no disposition.++ The endpoint now answers **422**, naming the field *and* the route that+ does apply it — `POST /feedback/alert-override` for a disposition,+ `POST /alerts/{id}/escalate` for severity, `POST /alerts/{id}/snooze` for+ a snooze. The accepted surface is unchanged, including the `assignee`+ alias the queue releases alerts with; only previously-ignored fields are+ affected. The published spec gains `additionalProperties: false` on+ `AlertUpdateRequest`, and `alertsApi.update` in the console is typed to+ the five fields rather than `Partial<Alert>`, so a rejected call now fails+ to compile instead of failing at runtime.++### Fixed++- **The copilot stopped inventing investigations, and stopped blaming a+ missing API key for a model that timed out** (#1275). The template+ fallback returned one of five rotating paragraphs asserting findings+ about an estate it had never read — "this IP was seen in 3 other alerts",+ "the attacker dwell time appears short (< 2 hours)" — and ignored the+ question entirely, so a ransomware alert came back described as+ "credential-access activity ... LOLBin pattern". That text is gone. The+ two fallback conditions are also now distinguished: "no model is+ configured" and "the configured model did not answer" used to produce one+ message ending "Configure an LLM key", which is the wrong instruction on a+ deployment running the bundled local model, where the usual cause is+ contention with the auto-triage backlog. `CopilotChatResponse` carries an+ additive `template_reason`; `source` is unchanged.+- **Auto-triage records the prompt behind its verdict** (#1276). The path+ that handles every fused alert wrote a verdict and nothing to check it+ against: its hardcoded `INSERT` named neither `input_hash` nor+ `output_hash`, `duration_ms` was the literal `0`, and no prompt was kept+ anywhere — so the most common AI decision in the platform was the one that+ could not be replayed, while `aisoc_explain_step` is documented as+ returning "the prompt, response, and tools used". The model call is now a+ `llm_call` event at seq 0 carrying both hashes and its measured duration,+ with the literal prompt and response attached to the verdict event as+ artifacts, which is what `/investigations/{run}/explain` inlines.+ `graph_step` rows carry a measured duration instead of a hardcoded zero.+- **The MCP `aisoc_run_hunt` tool works on a default install** (#1274). It+ declared `tenant_id: z.string().uuid()`, and zod 4 enforces RFC 4122+ version and variant bits that `00000000-0000-0000-0000-000000000001` —+ the tenant every `make up` creates — does not set, so the tool was+ uncallable on a stock deployment. The parameter is removed rather than+ loosened: the tenant comes from the authenticated principal, and the API+ handler was already dropping the field, so the isolation had been an+ accident of schema strictness rather than a control.+### Fixed- **The autonomy settings page announced an autopilot posture no deploymentdiff --git a/apps/web/src/lib/api.ts b/apps/web/src/lib/api.tsindex ad4517452..2e953b90b 100644--- a/apps/web/src/lib/api.ts+++ b/apps/web/src/lib/api.ts@@ -1640,6 +1640,30 @@ export interface AlertFilters {sort?: 'newest' | 'priority';}+/**+ * The five fields `PATCH /api/v1/alerts/{id}` can actually apply.+ *+ * This was `Partial<Alert>`, which let the console send any of the row's+ * ~40 columns. The server ignored everything outside this set and still+ * answered 200, so `{ disposition }` or `{ severity }` compiled, ran and+ * changed nothing (issue #1280). The server now refuses them; narrowing+ * the type here makes the rejected call fail to compile instead, and+ * `AlertUpdateRequest` on the Python side is the one definition this+ * mirrors.+ *+ * Severity goes through `POST /alerts/{id}/escalate`, a disposition+ * through `POST /feedback/alert-override`, and a snooze through+ * `POST /alerts/{id}/snooze`.+ */+export interface AlertUpdate {+ status?: AlertStatus;+ priority?: number;+ tags?: string[];+ /** `null` releases the alert back to the pool. */+ assignee?: string | null;+ case_id?: string | null;+}+export const alertsApi = {list: async (filters: AlertFilters = {}) => {const raw = await request<{@@ -1682,7 +1706,7 @@ export const alertsApi = {return normalizeAlert(raw);},- update: async (id: string, data: Partial<Alert>) => {+ update: async (id: string, data: AlertUpdate) => {const raw = await request<unknown>(`/api/v1/alerts/${id}`, {method: 'PATCH',body: JSON.stringify(data),diff --git a/docs/openapi.yaml b/docs/openapi.yamlindex 16054a826..aaa9c8d41 100644--- a/docs/openapi.yaml+++ b/docs/openapi.yaml@@ -20976,6 +20976,7 @@ components:format: uuid- type: 'null'title: Case Id+ additionalProperties: falsetype: objecttitle: AlertUpdateRequestApiKeyOut:diff --git a/services/agents/app/agents/auto_triage_agent.py b/services/agents/app/agents/auto_triage_agent.pyindex db2b78a4c..9bbcdf10f 100644--- a/services/agents/app/agents/auto_triage_agent.py+++ b/services/agents/app/agents/auto_triage_agent.py@@ -432,14 +432,31 @@ async def run_auto_triage(state: InvestigationState) -> InvestigationState:# constructing outside would have failed the whole graph run over a# configuration problem the deterministic path handles fine.llm = make_chat_model("triage", temperature=0.0, max_tokens=512, json_output=True)+ system_prompt = prompt_text("triage.system") + "\n\n" + system_rule(nonce)response = await safe_ainvoke(llm,[- SystemMessage(content=prompt_text("triage.system") + "\n\n" + system_rule(nonce)),+ SystemMessage(content=system_prompt),HumanMessage(content=alert_context),],)raw_text = response.content+ # Recorded before parsing, so a verdict that fails to parse still+ # leaves behind what the model was asked and what it said. The+ # ledger carries this onto the run (issue #1276): auto-triage is+ # the path that handles every fused alert, and it was the one+ # path whose decisions could not be replayed.+ state.record_llm_exchange(+ agent="auto_triage",+ purpose="triage.classify",+ model=getattr(llm, "model_name", None) or getattr(llm, "model", None),+ prompt=[+ {"role": "system", "content": system_prompt},+ {"role": "user", "content": alert_context},+ ],+ response=str(raw_text),+ duration_ms=round((time.monotonic() - t0) * 1000),+ )result = _parse_llm_response(raw_text)except Exception as exc:# Issue #571: do NOT swallow + return a null-verdict RUNNING state.diff --git a/services/agents/app/api/copilot.py b/services/agents/app/api/copilot.pyindex a06acec1e..a3fe38375 100644--- a/services/agents/app/api/copilot.py+++ b/services/agents/app/api/copilot.py@@ -8,14 +8,14 @@ Endpoints (all under ``/api/v1/copilot``):POST /chat — one-shot chat (creates / continues conv.)POST /chat/stream — streaming NDJSON variant-Falls back to synthetic deterministic replies when ``OPENAI_API_KEY`` is-unset so the demo path never breaks.+When no model answers, the reply says so and names which of the two+reasons applies — nothing is configured, or what is configured did not+return. It does not describe the tenant's data, because nothing read it."""from __future__ import annotationsimport asyncio-import itertoolsimport jsonimport uuidfrom collections.abc import AsyncIterator@@ -71,6 +71,11 @@ class CopilotChatResponse(BaseModel):#: The console must label a ``template`` reply rather than presenting it as#: analysis of the user's environment.source: Literal["llm", "template"] = "llm"+ #: Why no model answered, when ``source`` is ``template``. Additive+ #: rather than a third ``source`` value, so existing clients branching+ #: on ``template`` keep working while a new one can tell "nothing is+ #: configured" from "what is configured did not answer" (issue #1275).+ template_reason: Literal["no_model_configured", "model_call_failed"] | None = Nonenotice: str | None = None#: Which factual claims in `reply` cite a record, and which cite#: nothing (parity 3.6). The console renders the label beside the@@ -113,39 +118,49 @@ def _tenant_of(principal: TenantPrincipal) -> uuid.UUID:raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=str(exc)) from exc-_SYNTHETIC_REPLIES = [- (- "I've analysed the alert context. The activity matches T1078 (Valid Accounts) combined"- " with T1021.002 (SMB/Windows Admin Shares) lateral movement. Recommend isolating the"- " host and reviewing recent authentication logs."- ),- (- "Based on the indicators, this looks like credential-access activity. The parent process"- " chain suggests a LOLBin pattern. Consider adding a detection rule for this specific"- " chain."- ),- (- "The entity risk score is elevated due to multiple failed authentications followed by a"- " successful login from an unusual geolocation. I recommend triggering a step-up MFA"- " challenge."- ),- (- "Correlation across the last 24 hours shows this IP was seen in 3 other alerts. The MITRE"- " mapping points to T1110 (Brute Force). Blocking the IP at the perimeter is the fastest"- " remediation."- ),- (- "I've reviewed the case timeline. The attacker dwell time appears short (< 2 hours),"- " suggesting this may be an automated credential-stuffing campaign rather than a targeted"- " intrusion."- ),-]--_reply_cycle = itertools.cycle(_SYNTHETIC_REPLIES)---def _synthetic_reply(user_msg: str) -> str:- return next(_reply_cycle)+#: Why no model answered. These are two different operator problems and+#: the old code could not tell them apart (issue #1275).+NO_MODEL = "no_model_configured"+CALL_FAILED = "model_call_failed"+++def _fallback_reply(reason: str, *, model: str | None = None, detail: str | None = None) -> str:+ """What to say when no model answered.++ This used to return one of five rotating paragraphs asserting findings+ about an estate it had never seen — "this IP was seen in 3 other+ alerts", "the attacker dwell time appears short (< 2 hours)", "multiple+ failed authentications followed by a successful login from an unusual+ geolocation". None of it was computed from anything. The reporter hit+ exactly that: a ransomware alert described back to them as+ "credential-access activity ... LOLBin pattern", because the function+ ignored its argument and returned the next item in the cycle.++ A canned paragraph that reads like analysis is worse than no answer,+ and the console appending a disclaimer underneath does not fix the+ body. So the reply is now the explanation, and it makes no claim about+ the tenant's data.+ """+ if reason == NO_MODEL:+ return (+ "I can't answer this: no language model is configured for this deployment.\n\n"+ "Nothing about your alerts, cases or detections was looked at, so treat this as "+ "an unanswered question rather than a finding.\n\n"+ "To enable the copilot, configure a model — either a hosted provider key or the "+ "bundled local model — and ask again."+ )++ named = f" ({model})" if model else ""+ because = f"\n\nThe call reported: {detail}" if detail else ""+ return (+ f"I can't answer this: the configured model{named} did not return a reply.{because}\n\n"+ "Nothing about your alerts, cases or detections was looked at, so treat this as an "+ "unanswered question rather than a finding.\n\n"+ "This is a model or gateway problem, not a missing credential — a model is configured. "+ "On a CPU-only deployment the usual cause is contention with the auto-triage backlog, "+ "which holds the same local model. Retrying usually works; if it does not, check the "+ "model service's own logs."+ )# ---------------------------------------------------------------------------@@ -157,17 +172,48 @@ def _title_from_message(msg: str) -> str:return msg[:60] + ("…" if len(msg) > 60 else "")+def _notice_for(reason: str | None) -> str | None:+ """The one-line banner the console puts above a template reply.++ One string served both fallback conditions and ended "Configure an LLM+ key to get a real investigation". On a deployment running the bundled+ local model that sentence names a credential the deployment does not+ use, so an operator whose model had merely timed out was sent to fix+ something that was not broken.+ """+ if reason == NO_MODEL:+ return "No language model is configured, so nothing answered this question. This is not analysis of your environment."+ if reason == CALL_FAILED:+ return (+ "A model is configured but the call did not return, so nothing answered this "+ "question. This is not analysis of your environment, and it is not a missing "+ "API key."+ )+ return None++async def _get_openai_reply(conversation: dict[str, Any],user_message: str,-) -> tuple[str, str]:- """Return ``(reply_text, source)`` where source is ``llm`` or ``template``.+) -> tuple[str, str, str | None]:+ """Return ``(reply_text, source, reason)``.++ ``source`` is ``llm`` or ``template``; ``reason`` is ``None`` on the+ model path and otherwise names *why* no model answered.The caller must surface ``template`` to the user. This function silentlyreturned a canned paragraph as a normal 200 whenever the key was missing orany exception fired, so an analyst read "this IP was seen in 3 otheralerts" as real analysis of their environment. The frontend had an honestfallback of its own that never fired, because the backend reported success.++ The two fallback conditions are now distinguished (issue #1275). They+ used to collapse into one message telling the operator to configure an+ API key — advice that is simply wrong when a local model is configured+ and timed out, which is the common case on CPU. This repository has the+ same lesson recorded from the RBA banner that named fusion while fusion+ was healthy: a diagnostic pointing at the wrong subsystem sends someone+ to fix something that is not broken, and is worse than a vague one."""from app.llm.factory import resolve_api_key, resolve_model_alias@@ -176,7 +222,7 @@ async def _get_openai_reply(# goes to the bundled gateway the bearer has to be the gateway's master key.api_key = resolve_api_key(model) or ""if not api_key:- return _synthetic_reply(user_message), "template"+ return _fallback_reply(NO_MODEL), "template", NO_MODELtry:from app.llm.contract import safe_chat_completions_request@@ -204,10 +250,14 @@ async def _get_openai_reply(url=chat_completions_url(model),max_tokens=512,)- return body["choices"][0]["message"]["content"], "llm"+ return body["choices"][0]["message"]["content"], "llm", Noneexcept Exception as exc:- logger.warning("copilot.openai_error", error=str(exc))- return _synthetic_reply(user_message), "template"+ # `str(exc)` is empty for several httpx timeout classes, which is+ # how the report ended up with `copilot.openai_error` carrying no+ # message at all. The class name always says something.+ detail = str(exc).strip() or type(exc).__name__+ logger.warning("copilot.openai_error", error=detail, model=model)+ return _fallback_reply(CALL_FAILED, model=model, detail=detail), "template", CALL_FAILED# ---------------------------------------------------------------------------@@ -266,7 +316,7 @@ async def chat(}history = {"messages": [*(existing.messages if existing else []), user_msg]}- reply_text, reply_source = await _get_openai_reply(history, req.message)+ reply_text, reply_source, reply_reason = await _get_openai_reply(history, req.message)assistant_msg: dict[str, Any] = {"id": str(uuid.uuid4()),@@ -295,14 +345,9 @@ async def chat(conversationId=stored.id,reply=CopilotMessage(**assistant_msg),source=reply_source,+ template_reason=reply_reason,grounding=grounding,- notice=(- "This reply came from a built-in template, not a language model. "- "It is generic guidance and is not analysis of your environment. "- "Configure an LLM key to get a real investigation."- if reply_source == "template"- else None- ),+ notice=_notice_for(reply_reason),)@@ -328,7 +373,7 @@ async def chat_stream(}history = {"messages": [*(existing.messages if existing else []), user_msg]}- reply_text, reply_source = await _get_openai_reply(history, req.message)+ reply_text, reply_source, reply_reason = await _get_openai_reply(history, req.message)msg_id = str(uuid.uuid4())assistant_msg: dict[str, Any] = {"id": msg_id,@@ -354,7 +399,18 @@ async def chat_stream(async def _stream() -> AsyncIterator[bytes]:# Provenance first: a consumer must be able to label the answer before# it starts rendering tokens, not after.- yield (json.dumps({"source": reply_source, "delta": "", "done": False}) + "\n").encode()+ yield (+ json.dumps(+ {+ "source": reply_source,+ "template_reason": reply_reason,+ "notice": _notice_for(reply_reason),+ "delta": "",+ "done": False,+ }+ )+ + "\n"+ ).encode()words = reply_text.split(" ")for i, word in enumerate(words):chunk = word + (" " if i < len(words) - 1 else "")diff --git a/services/agents/app/graph/runner.py b/services/agents/app/graph/runner.pyindex 4ef86f32e..07d024899 100644--- a/services/agents/app/graph/runner.py+++ b/services/agents/app/graph/runner.py@@ -20,6 +20,7 @@ from __future__ import annotationsimport asyncioimport os+import timefrom dataclasses import dataclassimport structlog@@ -97,6 +98,12 @@ async def _run(seq = seq_starttimed_out = Falsebudget_reason: str | None = None+ # Wall clock for the step about to be streamed. Every `graph_step` row+ # carried `duration_ms: 0` because the argument was simply never passed+ # (issue #1276), so the ledger reported that each node of an escalation+ # took no time — a confident zero for a measurement nobody made, which+ # is the shape this project treats as worse than an absent one.+ step_started = time.monotonic()try:async with asyncio.timeout(budget.max_seconds):async for step in graph.astream(state_dict):@@ -106,6 +113,9 @@ async def _run(seq += 1if isinstance(out, dict):merged.update(out)+ now = time.monotonic()+ elapsed_ms = round((now - step_started) * 1000)+ step_started = nowif tenant_uuid is not None:await ledger_module.record_event(run_id=state.run_id,@@ -115,6 +125,7 @@ async def _run(agent=str(node),summary=f"graph node '{node}' completed",payload={"node": str(node)},+ duration_ms=elapsed_ms,)# Checked **inside** the loop, once per streamed step, afterdiff --git a/services/agents/app/investigator/ledger.py b/services/agents/app/investigator/ledger.pyindex 50843b2ff..b4dd4a77f 100644--- a/services/agents/app/investigator/ledger.py+++ b/services/agents/app/investigator/ledger.py@@ -408,6 +408,23 @@ async def complete_run()+def _inserted_a_row(status: Any) -> bool:+ """True when an asyncpg command status reports a row was written.++ ``conn.execute`` returns the tag, e.g. ``"INSERT 0 1"`` for a write and+ ``"INSERT 0 0"`` when ``ON CONFLICT DO NOTHING`` suppressed it. That is+ the only signal distinguishing a first write from a replay, and the+ artifacts have no unique key of their own to lean on.+ """+ try:+ return int(str(status).rsplit(" ", 1)[-1]) > 0+ except (ValueError, IndexError):+ # An unrecognised tag means "do not know". Writing nothing is the+ # safe side: a missing transcript is visibly missing, a duplicated+ # one silently doubles every replayed run.+ return False++async def fail_stale_runs(*,older_than_seconds: float,@@ -511,6 +528,7 @@ async def persist_auto_triage(groundedness: float | None = None,ungrounded: bool | None = None,shadow: bool = False,+ llm_exchanges: list[dict[str, Any]] | None = None,) -> bool:"""Durably record an auto-triage outcome (issue #571) in ONE transaction:@@ -571,16 +589,56 @@ async def persist_auto_triage(json.dumps(raw_alert or {}),f"kafka:auto_triage:{tier}",)- await conn.execute(+ # The model call that produced the verdict, at seq 0 so it+ # precedes it (issue #1276). This path recorded no prompt at+ # all, so the decision taken on every fused alert was the one+ # decision in the platform that could not be replayed.+ #+ # seq 0 rather than renumbering: seq 1 is the verdict's key+ # under `ON CONFLICT (run_id, seq)`, the run id is+ # deterministic, and moving the verdict would let a replay+ # write a second one. Escalation graph steps start at seq 2+ # (`run_escalation(..., seq_start=1)`), so 0 is the free slot+ # and it is also the one the manual path puts its first+ # prompt in.+ for exchange in (llm_exchanges or [])[:1]:+ await conn.execute(+ """+ INSERT INTO investigation_events+ (id, run_id, tenant_id, seq, ts, kind, agent, summary,+ payload, input_hash, output_hash, duration_ms, created_at)+ VALUES+ ($1, $2, $3, 0, now(), 'llm_call', $4, $5, $6::jsonb, $7, $8, $9, now())+ ON CONFLICT (run_id, seq) DO NOTHING+ """,+ uuid.uuid4(),+ run_id,+ tenant_id,+ str(exchange.get("agent") or "auto_triage"),+ f"model call: {exchange.get('purpose') or 'triage'}"[:8000],+ json.dumps(exchange),+ exchange.get("input_hash"),+ exchange.get("output_hash"),+ int(exchange.get("duration_ms") or 0),+ )++ verdict_event_id = uuid.uuid4()+ # The hashes travel onto the verdict too. An auditor asking+ # "what produced this verdict" reads the verdict row, and a+ # null `input_hash` there is indistinguishable from a run that+ # never called a model — which is a real state on the+ # deterministic path and must stay tellable apart.+ first = (llm_exchanges or [{}])[0]+ written = await conn.execute("""INSERT INTO investigation_events(id, run_id, tenant_id, seq, ts, kind, agent, summary,- payload, duration_ms, created_at)+ payload, input_hash, output_hash, duration_ms, created_at)VALUES- ($1, $2, $3, 1, now(), 'triage_verdict', $4, $5, $6::jsonb, 0, now())+ ($1, $2, $3, 1, now(), 'triage_verdict', $4, $5, $6::jsonb, $7, $8, $9, now())ON CONFLICT (run_id, seq) DO NOTHING""",- uuid.uuid4(),+ verdict_event_id,run_id,tenant_id,f"auto_triage:{tier}",@@ -595,7 +653,53 @@ async def persist_auto_triage("auto_closed": auto_closed,}),+ first.get("input_hash"),+ first.get("output_hash"),+ int(first.get("duration_ms") or 0),)++ # The literal transcript, attached to the verdict event.+ # `GET /investigations/{run}/explain?step=1` inlines the+ # artifacts of the focal event, which is how the MCP+ # `aisoc_explain_step` tool keeps its documented promise to+ # return "the prompt, response, and tools used".+ #+ # Written inline rather than through `record_artifact`+ # because that acquires its own connection, and a transcript+ # committed separately from the verdict it explains can+ # survive a rollback of the verdict.+ #+ # Skipped entirely when the verdict row already existed.+ # `investigation_artifacts` has no natural key to conflict+ # on, so without this a replay of a deterministic run id+ # would append a second copy of the transcript on every+ # pass while the events it explains stayed at one.+ if not _inserted_a_row(written):+ llm_exchanges = []+ for exchange in llm_exchanges or []:+ for kind, text in (+ ("llm_prompt", exchange.get("prompt")),+ ("llm_response", exchange.get("response")),+ ):+ if not text:+ continue+ blob = str(text)+ await conn.execute(+ """+ INSERT INTO investigation_artifacts+ (id, run_id, event_id, tenant_id, kind, content,+ sha256, size_bytes, created_at)+ VALUES ($1, $2, $3, $4, $5, $6, $7, $8, now())+ """,+ uuid.uuid4(),+ run_id,+ verdict_event_id,+ tenant_id,+ kind,+ blob,+ hashlib.sha256(blob.encode("utf-8")).hexdigest(),+ len(blob.encode("utf-8")),+ )await conn.execute(# `total_cost_usd` is measured cost only. It used to carry# a list-price guess keyed on a gateway alias, so a localdiff --git a/services/agents/app/models/state.py b/services/agents/app/models/state.pyindex 0e85fa9ec..a2ca84500 100644--- a/services/agents/app/models/state.py+++ b/services/agents/app/models/state.py@@ -4,6 +4,8 @@ Agent state models for LangGraph workflows.from __future__ import annotations+import hashlib+import jsonfrom datetime import datetimefrom enum import Enumfrom typing import Any@@ -11,6 +13,14 @@ from uuid import UUID, uuid4from pydantic import BaseModel, Field+#: Ceiling on the literal prompt / response kept beside a hash. Matches+#: `InvestigatorState`'s, so the two paths truncate at the same point.+_LLM_LITERAL_LIMIT = 8000+++def _stable_hash(value: str) -> str:+ return hashlib.sha256(value.encode("utf-8")).hexdigest()+class AgentTask(str, Enum):TRIAGE = "triage"@@ -169,6 +179,19 @@ class InvestigationState(BaseModel):# pivoted five times. This is what the depth gate grades.investigation_depth: dict[str, Any] | None = None+ # The literal model calls this run made, in order (issue #1276).+ #+ # The auto-triage ledger recorded a verdict and nothing a reader could+ # check it against: no prompt, and `input_hash`/`output_hash` null on+ # every step. A verdict with no recorded prompt cannot be replayed, and+ # replay is what `aisoc_explain_step` is documented to provide.+ #+ # The convention is the manual path's, not a second one: the hash is+ # over the full text and is always present, while the literal copy is+ # truncated, because these payloads carry customer data. Written by+ # `record_llm_exchange`; read by `ledger.persist_auto_triage`.+ llm_exchanges: list[dict[str, Any]] = Field(default_factory=list)+# Metadataiteration_count: int = 0max_iterations: int = 10@@ -180,5 +203,42 @@ class InvestigationState(BaseModel):def add_finding(self, finding: str) -> None:self.findings.append(finding)+ def record_llm_exchange(+ self,+ *,+ agent: str,+ purpose: str,+ model: str | None,+ prompt: str | list[dict[str, Any]],+ response: str,+ duration_ms: int,+ ) -> dict[str, Any]:+ """Record one model call so the verdict it produced can be replayed.++ Mirrors ``InvestigatorState.log_llm_prompt`` /+ ``log_llm_response`` on the manual path: a SHA-256 over the full+ text always, a truncated literal for a human reader. The two+ differ only in shape — the manual path appends two audit entries,+ this one records a single call with both of its hashes, because+ the ledger row it becomes carries ``input_hash`` and+ ``output_hash`` on the same event.+ """+ prompt_text = prompt if isinstance(prompt, str) else json.dumps(prompt, sort_keys=True, default=str)+ exchange = {+ "agent": agent,+ "purpose": purpose,+ "model": model,+ "prompt": prompt_text[:_LLM_LITERAL_LIMIT],+ "prompt_truncated": len(prompt_text) > _LLM_LITERAL_LIMIT,+ "response": response[:_LLM_LITERAL_LIMIT],+ "response_truncated": len(response) > _LLM_LITERAL_LIMIT,+ "input_hash": _stable_hash(prompt_text),+ "output_hash": _stable_hash(response),+ "duration_ms": int(duration_ms),+ "timestamp": datetime.utcnow().isoformat(),+ }+ self.llm_exchanges.append(exchange)+ return exchange+def to_dict(self) -> dict:return self.model_dump(mode="json")diff --git a/services/agents/app/workers/fused_alert_consumer.py b/services/agents/app/workers/fused_alert_consumer.pyindex b9a0af14b..8d172d9df 100644--- a/services/agents/app/workers/fused_alert_consumer.py+++ b/services/agents/app/workers/fused_alert_consumer.py@@ -1289,6 +1289,10 @@ class FusedAlertTriageWorker:# only inside a findings string.groundedness=state.groundedness,ungrounded=(state.groundedness is not None and state.groundedness < _groundedness_floor()),+ # The prompt and response behind the verdict (issue #1276).+ # Empty on the deterministic path, which is the honest value+ # there — no model was asked, so there is nothing to replay.+ llm_exchanges=list(state.llm_exchanges),)@staticmethoddiff --git a/services/agents/tests/test_auto_triage_ledger_records_the_prompt.py b/services/agents/tests/test_auto_triage_ledger_records_the_prompt.pynew file mode 100644index 000000000..98abdbf90--- /dev/null+++ b/services/agents/tests/test_auto_triage_ledger_records_the_prompt.py@@ -0,0 +1,369 @@+"""Auto-triage records what it asked the model, not only what it decided.++Issue #1276. The launch claim is that investigations are replayable "with+prompts, tool calls and evidence captured", and the MCP tool+``aisoc_explain_step`` is documented as returning "the prompt, response,+and tools used". Both held on the manual path and neither held on the one+that handles every fused alert.++Three concrete gaps, all reproduced below:++* ``run_auto_triage`` discarded the prompt. ``log_llm_prompt`` exists and+ works, but its four callers are all manual agents — no worker called it,+ so the most common AI decision in the platform left nothing to replay.+* ``persist_auto_triage``'s hardcoded ``INSERT`` named neither+ ``input_hash`` nor ``output_hash``, so both landed NULL, and+ ``duration_ms`` was the literal ``0``.+* ``graph/runner.py`` called ``record_event`` without a duration, so every+ ``graph_step`` row also reported zero.++A zero duration and a null hash are not missing data in the harmless+sense: they read as measurements. "This step took no time" is a claim.+"""++from __future__ import annotations++import json+from typing import Any+from uuid import uuid4++import pytest+from app.models.state import InvestigationState+++def _state() -> InvestigationState:+ return InvestigationState(incident_id=uuid4(), tenant_id=uuid4())+++class TestTheExchangeIsRecorded:+ def test_a_model_call_leaves_a_replayable_record(self) -> None:+ state = _state()+ state.record_llm_exchange(+ agent="auto_triage",+ purpose="triage.classify",+ model="llama3.2:3b",+ prompt=[{"role": "system", "content": "classify this"}],+ response='{"verdict": "false_positive"}',+ duration_ms=1234,+ )++ assert len(state.llm_exchanges) == 1+ recorded = state.llm_exchanges[0]+ assert "classify this" in recorded["prompt"]+ assert recorded["response"] == '{"verdict": "false_positive"}'+ assert recorded["duration_ms"] == 1234+ assert recorded["model"] == "llama3.2:3b"++ def test_both_hashes_are_present_and_differ(self) -> None:+ """The two NULL columns from the report, from the other end."""+ state = _state()+ recorded = state.record_llm_exchange(+ agent="auto_triage",+ purpose="triage.classify",+ model=None,+ prompt="ask",+ response="answer",+ duration_ms=1,+ )++ assert len(recorded["input_hash"]) == 64+ assert len(recorded["output_hash"]) == 64+ assert recorded["input_hash"] != recorded["output_hash"]++ def test_the_hash_covers_the_whole_prompt_not_the_truncated_copy(self) -> None:+ """The convention the manual path uses, and the reason for it: the+ literal is bounded because it carries customer data, so a hash over+ the bounded copy would verify the wrong thing."""+ long_prompt = "x" * 9000+ a = _state().record_llm_exchange(agent="a", purpose="p", model=None, prompt=long_prompt, response="r", duration_ms=0)+ b = _state().record_llm_exchange(agent="a", purpose="p", model=None, prompt=long_prompt + "DIFFERENT", response="r", duration_ms=0)++ assert a["prompt"] == b["prompt"], "both literals truncate to the same 8000 chars"+ assert a["input_hash"] != b["input_hash"], "the hashes must still tell them apart"+ assert a["prompt_truncated"] is True++ def test_a_run_that_called_no_model_records_nothing(self) -> None:+ """The deterministic path must stay distinguishable from a model+ call whose prompt went missing. An empty list is the honest value."""+ assert _state().llm_exchanges == []++ def test_the_record_survives_serialisation_through_the_graph(self) -> None:+ """LangGraph round-trips the state as a dict."""+ state = _state()+ state.record_llm_exchange(agent="auto_triage", purpose="p", model=None, prompt="ask", response="answer", duration_ms=5)+ assert state.to_dict()["llm_exchanges"][0]["response"] == "answer"+++class TestTheWorkerPathActuallyCallsIt:+ """The mechanism existing is not the claim. ``log_llm_prompt`` existed,+ worked and was unit-tested the whole time — and its four callers were+ all manual agents, so nothing on the path that handles every fused+ alert ever reached it. Drive the real ``run_auto_triage``."""++ @pytest.mark.asyncio+ async def test_a_real_triage_run_records_its_prompt(self, monkeypatch: Any) -> None:+ from types import SimpleNamespace++ from app.agents import auto_triage_agent as ata++ payload = json.dumps({"verdict": "false_positive", "confidence": 0.93, "rationale": "scanner"})+ seen: list[Any] = []++ async def _fake_ainvoke(_llm: Any, messages: Any) -> Any:+ seen.append(messages)+ return SimpleNamespace(content=payload)++ monkeypatch.setattr(ata, "make_chat_model", lambda *a, **k: object())+ monkeypatch.setattr(ata, "safe_ainvoke", _fake_ainvoke)++ state = InvestigationState(+ incident_id=uuid4(),+ tenant_id=uuid4(),+ alert_summary="Port scan from 10.0.0.5",+ raw_alert={"severity": "medium", "hostname": "WEB-01"},+ )+ out = await ata.run_auto_triage(state)++ assert out.verdict == "false_positive"+ assert len(out.llm_exchanges) == 1, "the verdict was reached with no recorded prompt"++ recorded = out.llm_exchanges[0]+ # What was recorded is what was sent, not a reconstruction of it.+ sent = json.dumps([{"role": m.type, "content": m.content} for m in seen[0]], default=str)+ assert "WEB-01" in recorded["prompt"], "the alert context is missing from the record"+ assert "WEB-01" in sent+ assert recorded["response"] == payload+ assert recorded["duration_ms"] >= 0+ assert len(recorded["input_hash"]) == 64++ @pytest.mark.asyncio+ async def test_a_response_that_fails_to_parse_still_leaves_a_record(self, monkeypatch: Any) -> None:+ """The case where a transcript is worth most: a parse failure whose+ message is a character offset into text nobody kept."""+ from types import SimpleNamespace++ from app.agents import auto_triage_agent as ata++ async def _fake_ainvoke(_llm: Any, _messages: Any) -> Any:+ return SimpleNamespace(content="not json at all")++ monkeypatch.setattr(ata, "make_chat_model", lambda *a, **k: object())+ monkeypatch.setattr(ata, "safe_ainvoke", _fake_ainvoke)++ state = InvestigationState(incident_id=uuid4(), tenant_id=uuid4(), alert_summary="x")+ try:+ await ata.run_auto_triage(state)+ except Exception:+ # Swallowed on purpose: the subject here is what the agent+ # *recorded* before the unparseable reply took it down, asserted+ # below. Whether it raises or falls back is a different question+ # and a different test; letting either outcome through keeps this+ # one from failing for a reason it is not about.+ pass++ assert state.llm_exchanges, "nothing recorded for the call that failed to parse"+ assert state.llm_exchanges[0]["response"] == "not json at all"+++class TestTheLedgerWritesIt:+ """Drives the real ``persist_auto_triage`` against a connection that+ records what it was asked to execute. The assertion is on the SQL and+ the bound parameters, because the defect *was* the column list — a+ double that accepted any INSERT would reproduce the bug happily."""++ @staticmethod+ async def _run(exchanges: list[dict[str, Any]] | None) -> list[tuple[str, tuple]]:+ from app.investigator import ledger as ledger_module++ executed: list[tuple[str, tuple]] = []+ tenant = uuid4()++ class _Conn:+ async def execute(self, sql: str, *args: Any) -> str:+ executed.append((sql, args))+ return "INSERT 0 1"++ async def fetchval(self, *_a: Any, **_k: Any) -> Any:+ return tenant++ def transaction(self) -> Any:+ return _Txn()++ class _Txn:+ async def __aenter__(self) -> None:+ return None++ async def __aexit__(self, *_a: Any) -> bool:+ return False++ class _Acquire:+ async def __aenter__(self) -> _Conn:+ return _Conn()++ async def __aexit__(self, *_a: Any) -> bool:+ return False++ class _Pool:+ def acquire(self) -> _Acquire:+ return _Acquire()++ async def _pool() -> _Pool:+ return _Pool()++ async def _resolve(_conn: Any, _ref: str) -> Any:+ return tenant++ # Through `MonkeyPatch` rather than a hand-rolled save/assign/restore.+ # Assigning the module attribute directly is a rebind CodeQL can see,+ # and it makes `py/import-of-mutable-attribute` fire on+ # `test_ledger_tenant.py`, which imports `_resolve_tenant_id` by value+ # and would therefore keep the original — a real hazard, just not one+ # this test was trying to create.+ with pytest.MonkeyPatch.context() as mp:+ mp.setattr(ledger_module, "get_pool", _pool)+ mp.setattr(ledger_module, "_resolve_tenant_id", _resolve)+ await ledger_module.persist_auto_triage(+ run_id=uuid4(),+ alert_id=str(uuid4()),+ tenant_ref=str(tenant),+ alert_summary="s",+ raw_alert={},+ tier="L1",+ verdict="false_positive",+ confidence=0.9,+ rationale="because",+ llm_exchanges=exchanges,+ )+ return executed++ @pytest.mark.asyncio+ async def test_the_verdict_insert_names_both_hash_columns(self) -> None:+ executed = await self._run([])+ verdict_sql = [s for s, _ in executed if "triage_verdict" in s]+ assert verdict_sql, "no verdict row was written"+ assert "input_hash" in verdict_sql[0]+ assert "output_hash" in verdict_sql[0]++ @pytest.mark.asyncio+ async def test_the_verdict_no_longer_hardcodes_a_zero_duration(self) -> None:+ executed = await self._run([])+ verdict_sql = [s for s, _ in executed if "triage_verdict" in s][0]+ # The original literal was `..., $6::jsonb, 0, now())`.+ assert ", 0, now())" not in verdict_sql++ @pytest.mark.asyncio+ async def test_the_prompt_is_written_as_its_own_step(self) -> None:+ exchange = {+ "agent": "auto_triage",+ "purpose": "triage.classify",+ "prompt": "the literal prompt",+ "response": "the literal response",+ "input_hash": "a" * 64,+ "output_hash": "b" * 64,+ "duration_ms": 742,+ }+ executed = await self._run([exchange])++ llm_steps = [(s, a) for s, a in executed if "'llm_call'" in s]+ assert len(llm_steps) == 1, "the model call left no step of its own"+ sql, args = llm_steps[0]+ assert ", 0, now()," in sql or " 0, now(), 'llm_call'" in sql, "the prompt must precede the verdict"+ assert "a" * 64 in args+ assert "b" * 64 in args+ assert 742 in args+ assert any("the literal prompt" in json.dumps(a) for a in args if isinstance(a, str))++ @pytest.mark.asyncio+ async def test_the_transcript_is_attached_for_explain_step(self) -> None:+ """`/investigations/{run}/explain` inlines the artifacts of the+ focal event, which is how `aisoc_explain_step` returns a literal+ transcript rather than a hash."""+ executed = await self._run(+ [+ {+ "prompt": "the literal prompt",+ "response": "the literal response",+ "input_hash": "a" * 64,+ "output_hash": "b" * 64,+ "duration_ms": 1,+ }+ ]+ )++ artifacts = [a for s, a in executed if "investigation_artifacts" in s]+ kinds = {a[4] for a in artifacts}+ assert kinds == {"llm_prompt", "llm_response"}+ assert any("the literal prompt" in str(a) for a in artifacts[0])++ @pytest.mark.asyncio+ async def test_a_run_with_no_model_call_writes_no_transcript(self) -> None:+ """No invented artifact for the deterministic path."""+ executed = await self._run([])+ assert [s for s, _ in executed if "investigation_artifacts" in s] == []+ assert [s for s, _ in executed if "'llm_call'" in s] == []+++class TestReplayDoesNotDuplicateTheTranscript:+ def test_a_suppressed_insert_is_detected(self) -> None:+ """`investigation_artifacts` has no key to conflict on, so the+ artifact write is gated on whether the verdict row was new. The+ run id is deterministic, so this path is taken on every replay."""+ from app.investigator.ledger import _inserted_a_row++ assert _inserted_a_row("INSERT 0 1") is True+ assert _inserted_a_row("INSERT 0 0") is False++ def test_an_unreadable_tag_writes_nothing(self) -> None:+ """A missing transcript is visibly missing; a duplicated one+ silently doubles on every replay."""+ from app.investigator.ledger import _inserted_a_row++ assert _inserted_a_row(None) is False+ assert _inserted_a_row("weird") is False+++class TestGraphStepsCarryTheirDuration:+ """`record_event` defaults ``duration_ms`` to 0 and the call site never+ passed it, so every escalation step recorded as instantaneous. Driven+ through the real ``_run`` against a graph that takes measurable time —+ reading the source would pass on a call that passes a literal zero."""++ @pytest.mark.asyncio+ async def test_a_step_that_took_time_does_not_record_zero(self, monkeypatch: Any) -> None:+ import asyncio++ from app.graph import runner as runner_mod++ recorded: list[dict[str, Any]] = []++ async def _record_event(**kwargs: Any) -> None:+ recorded.append(kwargs)++ async def _resolve(_ref: str) -> Any:+ return uuid4()++ async def _start_run(**_kwargs: Any) -> Any:+ return uuid4()++ monkeypatch.setattr(runner_mod.ledger_module, "record_event", _record_event)+ monkeypatch.setattr(runner_mod.ledger_module, "resolve_tenant", _resolve)+ monkeypatch.setattr(runner_mod.ledger_module, "start_run", _start_run)++ class _Graph:+ async def astream(self, _state: dict[str, Any]) -> Any:+ await asyncio.sleep(0.05)+ yield {"triage": {}}+ await asyncio.sleep(0.05)+ yield {"respond": {}}++ await runner_mod._run(+ _Graph(),+ _state(),+ budget=runner_mod.default_budget(),+ persist=True,+ seq_start=1,+ )++ steps = [e for e in recorded if e.get("kind") == "graph_step"]+ assert len(steps) == 2, f"expected two graph steps, got {len(steps)}"+ assert all(s["duration_ms"] > 0 for s in steps), [s["duration_ms"] for s in steps]diff --git a/services/agents/tests/test_copilot_fallback_is_honest.py b/services/agents/tests/test_copilot_fallback_is_honest.pynew file mode 100644index 000000000..324bef534--- /dev/null+++ b/services/agents/tests/test_copilot_fallback_is_honest.py@@ -0,0 +1,167 @@+"""The copilot says what happened instead of inventing an investigation.++Issue #1275. Two defects on the same code path.++**The fallback text named the wrong cause.** One string served both+fallback conditions and ended "Configure an LLM key to get a real+investigation". That branch is reached on *any* provider failure,+including a timeout from the bundled local model, which needs no API key+at all — so an operator whose model had merely timed out was told to go+and configure a credential their deployment does not use. The same lesson+is already recorded here from the RBA banner that blamed fusion while+fusion was healthy.++**The canned body was fabricated security analysis.** ``_synthetic_reply``+ignored its argument and returned the next of five rotating paragraphs,+each asserting specific findings about an estate it had never read — "this+IP was seen in 3 other alerts", "attacker dwell time appears short+(< 2 hours)". The reporter watched a ransomware alert come back described+as "credential-access activity ... LOLBin pattern". The console appends a+disclaimer above it, which does not make the body true.+"""++from __future__ import annotations++from typing import Any++import pytest+from app.api import copilot+from app.llm import contract as llm_contract+from app.llm import factory as llm_factory+++class TestTheTwoReasonsAreDistinguished:+ """Driven through the real `_get_openai_reply`, because the defect was+ that both conditions arrived at one string — a test calling the two+ message builders directly would never have noticed."""++ @pytest.mark.asyncio+ async def test_no_model_configured_is_reported_as_such(self, monkeypatch: Any) -> None:+ monkeypatch.setattr(llm_factory, "resolve_api_key", lambda _m: "")++ _text, source, reason = await copilot._get_openai_reply({"messages": []}, "what happened?")++ assert source == "template"+ assert reason == copilot.NO_MODEL++ @pytest.mark.asyncio+ async def test_a_configured_model_that_fails_is_not_called_a_missing_key(self, monkeypatch: Any) -> None:+ """The exact situation from the report: a model *is* configured and+ the call timed out."""+ monkeypatch.setattr(llm_factory, "resolve_api_key", lambda _m: "sk-present")+ monkeypatch.setattr(llm_factory, "chat_completions_url", lambda _m=None: "http://local/v1/chat/completions")++ async def _boom(**_kwargs: Any) -> Any:+ raise TimeoutError()++ monkeypatch.setattr(llm_contract, "safe_chat_completions_request", _boom)++ text, source, reason = await copilot._get_openai_reply({"messages": []}, "what happened?")++ assert source == "template"+ assert reason == copilot.CALL_FAILED+ assert "not a missing credential" in text+ assert "not a missing API key" in (copilot._notice_for(reason) or "")++ @pytest.mark.asyncio+ async def test_an_empty_exception_message_still_names_the_failure(self, monkeypatch: Any) -> None:+ """`str(exc)` is empty for several httpx timeout classes, which is+ why the report saw `copilot.openai_error` with no message."""+ monkeypatch.setattr(llm_factory, "resolve_api_key", lambda _m: "sk-present")+ monkeypatch.setattr(llm_factory, "chat_completions_url", lambda _m=None: "http://local/v1/chat/completions")++ async def _boom(**_kwargs: Any) -> Any:+ raise TimeoutError()++ monkeypatch.setattr(llm_contract, "safe_chat_completions_request", _boom)++ text, _source, _reason = await copilot._get_openai_reply({"messages": []}, "q")+ assert "TimeoutError" in text++ def test_the_two_notices_differ(self) -> None:+ assert copilot._notice_for(copilot.NO_MODEL) != copilot._notice_for(copilot.CALL_FAILED)++ def test_only_the_unconfigured_notice_mentions_configuring_one(self) -> None:+ """The whole defect in one assertion."""+ failed = copilot._notice_for(copilot.CALL_FAILED) or ""+ assert "not a missing API key" in failed+ assert "A model is configured" in failed++ def test_a_model_answer_carries_no_notice(self) -> None:+ assert copilot._notice_for(None) is None+++class TestTheFallbackBodyAssertsNothingAboutTheEstate:+ """Every one of these strings was in a reply the product shipped as a+ 200, under a `source` field most callers never read."""++ @pytest.mark.parametrize(+ "fabricated",+ [+ "seen in 3 other alerts",+ "dwell time",+ "LOLBin",+ "unusual geolocation",+ "T1078",+ "T1021.002",+ "T1110",+ "isolating the host",+ "step-up MFA",+ "credential-stuffing",+ ],+ )+ def test_no_fabricated_finding_survives_in_any_fallback(self, fabricated: str) -> None:+ bodies = [+ copilot._fallback_reply(copilot.NO_MODEL),+ copilot._fallback_reply(copilot.CALL_FAILED, model="llama3.2:3b", detail="timeout"),+ ]+ for body in bodies:+ assert fabricated.lower() not in body.lower()++ def test_the_module_no_longer_carries_the_canned_corpus(self) -> None:+ """The paragraphs are gone, not merely unreferenced. A constant left+ in place is one edit away from being used again."""+ assert not hasattr(copilot, "_SYNTHETIC_REPLIES")+ assert not hasattr(copilot, "_synthetic_reply")++ def test_the_reply_states_plainly_that_nothing_was_read(self) -> None:+ for body in (+ copilot._fallback_reply(copilot.NO_MODEL),+ copilot._fallback_reply(copilot.CALL_FAILED),+ ):+ assert "unanswered question rather than a finding" in body++ def test_the_failed_call_names_the_model_and_the_error(self) -> None:+ body = copilot._fallback_reply(copilot.CALL_FAILED, model="llama3.2:3b", detail="ReadTimeout")+ assert "llama3.2:3b" in body+ assert "ReadTimeout" in body++ def test_it_points_at_the_contention_the_reporter_actually_hit(self) -> None:+ """CPU-only deployments share one local model between interactive+ copilot and the auto-triage backlog. Naming it is the difference+ between a retry and an afternoon of debugging."""+ body = copilot._fallback_reply(copilot.CALL_FAILED)+ assert "auto-triage" in body++ def test_the_same_question_does_not_get_a_different_canned_answer(self) -> None:+ """`_synthetic_reply` cycled, so asking twice produced two+ unrelated 'analyses' of one alert."""+ first = copilot._fallback_reply(copilot.NO_MODEL)+ second = copilot._fallback_reply(copilot.NO_MODEL)+ assert first == second+++class TestTheWireContractCarriesTheReason:+ def test_the_response_model_declares_the_reason(self) -> None:+ fields = copilot.CopilotChatResponse.model_fields+ assert "template_reason" in fields++ def test_it_is_optional_so_existing_clients_are_unaffected(self) -> None:+ """Additive, not a third `source` value — the console branches on+ `source === 'template'` and must keep working."""+ built = copilot.CopilotChatResponse(+ conversationId="c",+ reply=copilot.CopilotMessage(role="assistant", content="x"),+ )+ assert built.template_reason is None+ assert built.source == "llm"diff --git a/services/agents/tests/test_response_provenance.py b/services/agents/tests/test_response_provenance.pyindex 5ce4cbce2..c6ac0f989 100644--- a/services/agents/tests/test_response_provenance.py+++ b/services/agents/tests/test_response_provenance.py@@ -45,9 +45,13 @@ class TestCopilotProvenance:@pytest.mark.asyncioasync def test_no_api_key_is_reported_as_template(self, monkeypatch: pytest.MonkeyPatch):monkeypatch.delenv("OPENAI_API_KEY", raising=False)- text, source = await copilot_mod._get_openai_reply({"messages": []}, "what happened?")+ text, source, reason = await copilot_mod._get_openai_reply({"messages": []}, "what happened?")assert source == "template"assert text, "the fallback should still say something useful"+ # The reason is carried alongside the source (issue #1275): "nothing+ # is configured" and "what is configured did not answer" used to be+ # the same answer, and only one of them is fixed by a credential.+ assert reason == copilot_mod.NO_MODEL@pytest.mark.asyncioasync def test_llm_failure_is_reported_as_template(self, monkeypatch: pytest.MonkeyPatch):@@ -58,8 +62,9 @@ class TestCopilotProvenance:raise RuntimeError("provider unreachable")monkeypatch.setattr("app.llm.contract.safe_chat_completions_request", _boom, raising=False)- _text, source = await copilot_mod._get_openai_reply({"messages": []}, "hello")+ _text, source, reason = await copilot_mod._get_openai_reply({"messages": []}, "hello")assert source == "template"+ assert reason == copilot_mod.CALL_FAILEDdef test_response_model_defaults_to_llm_but_can_carry_template(self):"""The field must exist on the wire, not just in the handler."""diff --git a/services/api/app/api/v1/endpoints/alerts.py b/services/api/app/api/v1/endpoints/alerts.pyindex af397ce7f..02c06d517 100644--- a/services/api/app/api/v1/endpoints/alerts.py+++ b/services/api/app/api/v1/endpoints/alerts.py@@ -8,7 +8,7 @@ from datetime import UTC, datetimefrom typing import Annotated, Any, Literalfrom fastapi import APIRouter, Depends, Header, HTTPException, Query, Request, status-from pydantic import BaseModel, ConfigDict, Field+from pydantic import BaseModel, ConfigDict, Field, model_validatorfrom sqlalchemy import and_, func, select, updatefrom sqlalchemy.exc import IntegrityError@@ -232,8 +232,26 @@ class AlertListResponse(BaseModel):facets: AlertFacetCounts = Field(default_factory=AlertFacetCounts)+#: Fields this endpoint cannot apply, mapped to the route that can.+#:+#: Each key is a real ``Alert`` column, which is why they arrive here and+#: why dropping them silently was expensive: the caller is asking for+#: something the platform does support, just not through this verb.+_ALERT_UPDATE_ELSEWHERE = {+ "disposition": "POST /api/v1/feedback/alert-override",+ "corrected_verdict": "POST /api/v1/feedback/alert-override",+ "verdict": "POST /api/v1/feedback/alert-override",+ "severity": "POST /api/v1/alerts/{alert_id}/escalate",+ "snoozed_until": "POST /api/v1/alerts/{alert_id}/snooze",+}++class AlertUpdateRequest(BaseModel):- model_config = ConfigDict(populate_by_name=True)+ #: ``extra="forbid"``, because the default is ``ignore`` and this is a+ #: write endpoint. An ignored field on a read is a tolerated unknown;+ #: on a write it is a 200 reporting a change that did not happen, and+ #: nothing in the response body distinguishes the two.+ model_config = ConfigDict(populate_by_name=True, extra="forbid")status: str | None = Nonepriority: int | None = None@@ -243,6 +261,29 @@ class AlertUpdateRequest(BaseModel):assigned_to_id: uuid.UUID | None = Field(default=None, alias="assignee")case_id: uuid.UUID | None = None+ @model_validator(mode="before")+ @classmethod+ def _name_the_route_that_applies_it(cls, data: Any) -> Any:+ """Refuse an unapplicable field by name, and say where it does apply.++ ``extra="forbid"`` alone answers "Extra inputs are not permitted",+ which tells an operator their call was wrong without telling them+ what is right. Three of these fields have a route that works.+ """+ if not isinstance(data, dict):+ return data++ accepted = {"status", "priority", "tags", "assigned_to_id", "assignee", "case_id"}+ unknown = [str(key) for key in data if str(key) not in accepted]+ if not unknown:+ return data++ parts = []+ for key in sorted(unknown):+ route = _ALERT_UPDATE_ELSEWHERE.get(key)+ parts.append(f"{key} (use {route})" if route else key)+ raise ValueError(f"PATCH /alerts cannot apply: {', '.join(parts)}. It accepts only: {', '.join(sorted(accepted))}.")+class AlertStatsResponse(BaseModel):total: intdiff --git a/services/api/tests/test_alert_update_refuses_unknown_fields.py b/services/api/tests/test_alert_update_refuses_unknown_fields.pynew file mode 100644index 000000000..2d498f5d3--- /dev/null+++ b/services/api/tests/test_alert_update_refuses_unknown_fields.py@@ -0,0 +1,114 @@+"""``PATCH /alerts/{id}`` refuses a field it cannot apply (issue #1280).++``AlertUpdateRequest`` set no ``extra=``, so pydantic v2's default+``extra="ignore"`` applied and every key outside the declared five was+dropped before the handler ran. The response was a 200 carrying the+unchanged row, which is indistinguishable from a successful write.++The fields that reach this endpoint are not nonsense — ``disposition``,+``severity``, ``confidence`` and ``snoozed_until`` are all real ``Alert``+columns, three of them writable through a different route. That is what+made the silent drop expensive: a caller closing an alert with a+disposition got a 200 and no disposition, and nothing in the response+distinguished "applied" from "ignored".+"""++from __future__ import annotations++import uuid++import pytest+from app.api.v1.endpoints.alerts import AlertUpdateRequest+from pydantic import ValidationError+++def _message(exc: ValidationError) -> str:+ return str(exc)+++class TestAnUnapplicableFieldIsRefused:+ def test_the_reported_payload_is_refused_rather_than_half_applied(self) -> None:+ """The exact body from the report: status lands, disposition does not."""+ with pytest.raises(ValidationError) as caught:+ AlertUpdateRequest(status="resolved", disposition="false_positive")++ assert "disposition" in _message(caught.value)++ @pytest.mark.parametrize(+ "field",+ ["disposition", "severity", "confidence", "resolution", "snoozed_until"],+ )+ def test_every_silently_dropped_field_is_refused(self, field: str) -> None:+ """The report named one; the drop list was wider than one."""+ with pytest.raises(ValidationError) as caught:+ AlertUpdateRequest(**{field: "x"})++ assert field in _message(caught.value)++ def test_a_typo_in_a_real_field_is_refused_rather_than_ignored(self) -> None:+ """`statuss` used to be a 200 that changed nothing."""+ with pytest.raises(ValidationError):+ AlertUpdateRequest(statuss="resolved")+++class TestTheRefusalNamesSomewhereToGo:+ """A 422 that only says "not permitted" tells an operator their call+ was wrong without telling them what is right, and three of these+ fields have a route that does apply them."""++ @pytest.mark.parametrize(+ ("field", "route"),+ [+ ("disposition", "/feedback/alert-override"),+ ("severity", "/escalate"),+ ("snoozed_until", "/snooze"),+ ],+ )+ def test_a_field_with_a_home_elsewhere_names_that_route(self, field: str, route: str) -> None:+ with pytest.raises(ValidationError) as caught:+ AlertUpdateRequest(**{field: "x"})++ assert route in _message(caught.value)++ def test_a_field_with_no_home_lists_what_is_accepted(self) -> None:+ """No route is invented for a field that has none."""+ with pytest.raises(ValidationError) as caught:+ AlertUpdateRequest(nonsense=1)++ message = _message(caught.value)+ assert "status" in message+ assert "assignee" in message+++class TestTheAcceptedSurfaceIsUnchanged:+ """`extra="forbid"` is a behaviour change on a published endpoint, so+ the controls matter as much as the refusal. Every shape the console+ and the SDK already send must still be accepted."""++ def test_the_five_declared_fields_are_accepted(self) -> None:+ parsed = AlertUpdateRequest(+ status="triaged",+ priority=80,+ tags=["a"],+ assigned_to_id=None,+ case_id=None,+ )+ assert parsed.status == "triaged"+ assert parsed.priority == 80++ def test_the_assignee_alias_is_not_extra(self) -> None:+ """A declared alias is not an unknown field. The queue releases an+ alert with `{"assignee": null}` and that must keep working."""+ parsed = AlertUpdateRequest(assignee=None)+ assert "assigned_to_id" in parsed.model_fields_set++ def test_assigning_by_alias_still_binds_the_value(self) -> None:+ who = uuid.uuid4()+ assert AlertUpdateRequest(assignee=who).assigned_to_id == who++ def test_the_canonical_field_name_still_works(self) -> None:+ who = uuid.uuid4()+ assert AlertUpdateRequest(assigned_to_id=who).assigned_to_id == who++ def test_an_empty_body_is_still_a_no_op_rather_than_an_error(self) -> None:+ assert AlertUpdateRequest().model_fields_set == set()diff --git a/services/mcp/src/tools/hunt.ts b/services/mcp/src/tools/hunt.tsindex c15eb59e0..938ce07db 100644--- a/services/mcp/src/tools/hunt.ts+++ b/services/mcp/src/tools/hunt.ts@@ -23,6 +23,21 @@** **It is read-only.** A hunt searches; it does not act. The annotation* says so and no write path exists here.+ *+ * **It takes no tenant.** It used to declare+ * `tenant_id: z.string().uuid()` and forward it in the body (issue+ * #1274). Two things were wrong with that, and the smaller one is what+ * got noticed: zod 4's `.uuid()` enforces RFC 4122 version and variant+ * bits, which the tenant every default install is bootstrapped into+ * (`00000000-0000-0000-0000-000000000001`) does not set — so the tool+ * could not be called at all on a stock deployment.+ *+ * The larger one is that a tenant supplied by a *model* is not a+ * boundary. `AgentHuntRequest` declares only `hypothesis`, so pydantic+ * was silently dropping the field: the isolation was an accident of+ * schema strictness rather than a decision. Relaxing the validator to+ * `z.guid()` would have made the bad input pass and left that in place.+ * The tenant comes from the credential, here as everywhere else.*/import { z } from "zod";@@ -46,7 +61,6 @@ const RunHuntSchema = z.describe("What you suspect, in plain language. For example: 'a service account signed in from a country it has never used before'.",),- tenant_id: z.string().uuid().describe("The tenant whose recorded events to search."),}).strict();@@ -69,7 +83,6 @@ export const runHuntTool: ToolDefinition<typeof RunHuntSchema> = {async handle(ctx, args) {const reply = await ctx.client.post<HuntReply>("/api/v1/agents/hunt", {hypothesis: args.hypothesis,- tenant_id: args.tenant_id,});const matches = reply.matches ?? [];diff --git a/services/mcp/tests/tenant-is-not-an-argument.test.ts b/services/mcp/tests/tenant-is-not-an-argument.test.tsnew file mode 100644index 000000000..0c9f879b6--- /dev/null+++ b/services/mcp/tests/tenant-is-not-an-argument.test.ts@@ -0,0 +1,159 @@+/**+ * The tenant is never a tool argument (issue #1274).+ *+ * `aisoc_run_hunt` declared `tenant_id: z.string().uuid()` and forwarded+ * it in the request body. Two separate things were wrong with that.+ *+ * The one the report hit: zod 4's `.uuid()` checks RFC 4122 version and+ * variant bits, and the tenant every default install is bootstrapped into+ * (`00000000-0000-0000-0000-000000000001`) sets neither. So the tool was+ * unusable on a stock deployment — the model could not produce a value the+ * schema would accept.+ *+ * The one that matters more: a tenant a *model* supplies is not a control.+ * `AgentHuntRequest` declares only `hypothesis` and the handler's docstring+ * says the tenant "comes from the authenticated principal and is never read+ * from the body", so today's protection is an accident of pydantic dropping+ * an undeclared key — not a boundary. Loosening the validator to `z.guid()`+ * would have made the bad input pass and left the shape in place.+ *+ * The assertion is therefore structural and repo-wide, not a fix to one+ * tool: no tool on this server takes a tenant from its caller.+ */+import { readFileSync, readdirSync } from "node:fs";+import { join } from "node:path";+import { fileURLToPath } from "node:url";++import { describe, expect, it } from "vitest";+import { z } from "zod";++import { ALL_TOOLS } from "../src/tools/index.js";+import { runHuntTool } from "../src/tools/hunt.js";++const SRC = fileURLToPath(new URL("../src", import.meta.url));++/** `bootstrap_admin.DEFAULT_TENANT_ID` — what a stock `make up` creates. */+const DEFAULT_TENANT = "00000000-0000-0000-0000-000000000001";++function everySourceFile(dir: string): string[] {+ const out: string[] = [];+ for (const entry of readdirSync(dir, { withFileTypes: true })) {+ const path = join(dir, entry.name);+ if (entry.isDirectory()) out.push(...everySourceFile(path));+ else if (entry.name.endsWith(".ts")) out.push(path);+ }+ return out;+}++interface RecordedCall {+ path: string;+ body?: unknown;+}++function recordingClient(calls: RecordedCall[], reply: unknown) {+ return {+ async get(path: string) {+ calls.push({ path });+ return reply;+ },+ async post(path: string, body: unknown) {+ calls.push({ path, body });+ return reply;+ },+ };+}++const log = { debug() {}, info() {}, warn() {}, error() {} };++describe("the reported symptom", () => {+ it("confirms zod rejects the default tenant that z.guid() accepts", () => {+ // Not an assertion about our code — it pins the upstream behaviour the+ // report rests on, so a zod bump that changes it shows up here rather+ // than as a mystery. The version-and-variant bits are all zero.+ expect(z.string().uuid().safeParse(DEFAULT_TENANT).success).toBe(false);+ expect(z.guid().safeParse(DEFAULT_TENANT).success).toBe(true);++ // A conventional v4 uuid passes both, which is why this went unnoticed:+ // it works on any tenant except the one every first run lands in.+ const v4 = "550e8400-e29b-41d4-a716-446655440000";+ expect(z.string().uuid().safeParse(v4).success).toBe(true);+ });+});++describe("no tool takes a tenant from its caller", () => {+ it("declares no tenant field in any tool's input schema", () => {+ const offenders: string[] = [];+ for (const tool of ALL_TOOLS) {+ const schema = tool.metadata.inputSchema as {+ properties?: Record<string, unknown>;+ };+ for (const key of Object.keys(schema.properties ?? {})) {+ if (/tenant/i.test(key)) offenders.push(`${tool.metadata.name}.${key}`);+ }+ }+ expect(offenders).toEqual([]);+ });++ it("sends no tenant in any request body built from tool arguments", async () => {+ // The schema and the wire are two different claims. A tool could drop+ // the parameter and still stamp a tenant into the body from somewhere.+ const calls: RecordedCall[] = [];+ await runHuntTool.handle(+ {+ client: recordingClient(calls, {+ hypothesis: "h",+ checked: true,+ matches: [],+ refusals: [],+ unavailable_reason: null,+ }),+ log,+ } as never,+ { hypothesis: "a service account signed in from a new country" } as never,+ );++ expect(calls).toHaveLength(1);+ expect(calls[0].path).toBe("/api/v1/agents/hunt");+ expect(Object.keys(calls[0].body as object)).toEqual(["hypothesis"]);+ });++ it("names no tenant_id key anywhere a request body is built", () => {+ // Structural, so a tool added later is covered without editing this+ // test. Prose and response-shape types legitimately mention the word,+ // so match an object key being assigned, which is what a body literal+ // and a zod schema field both look like.+ const offenders: string[] = [];+ for (const file of everySourceFile(SRC)) {+ const body = readFileSync(file, "utf8");+ for (const line of body.split("\n")) {+ // Skip comments — `lake.ts` explains at length that the predicate is+ // injected server-side, which is the behaviour this test wants.+ const code = line.replace(/^\s*(\/\/|\*|\/\*).*$/, "");+ if (/\btenant_id\s*:/.test(code) && !/\btenant_id\s*:\s*string\s*\|\s*null/.test(code)) {+ offenders.push(`${file.replace(SRC, "src")}: ${line.trim()}`);+ }+ }+ }+ expect(offenders).toEqual([]);+ });+});++describe("the hunt tool still works", () => {+ it("accepts a hypothesis alone", () => {+ expect(runHuntTool.schema.safeParse({ hypothesis: "x" }).success).toBe(true);+ });++ it("refuses a caller-supplied tenant rather than ignoring it", () => {+ // `.strict()` is deliberate. Silently dropping the key is how the API+ // side ended up with a protection nobody had decided on.+ const parsed = runHuntTool.schema.safeParse({+ hypothesis: "x",+ tenant_id: DEFAULT_TENANT,+ });+ expect(parsed.success).toBe(false);+ });++ it("still requires a non-empty hypothesis", () => {+ expect(runHuntTool.schema.safeParse({ hypothesis: "" }).success).toBe(false);+ });+});