Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
feat(retro-hunt): check KEV entries against a tenant's own vulnerability data (#931)
Beenu Arora90864f5820f2
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex e329f472a..27d75f7c0 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -348,6 +348,42 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0minted before this change fail closed rather than outliving the revocationmeant to end them. The refresh path checks it too, and matters more there: arefresh token outlives an access token by days.+- **A CISA KEV entry is now checked against the tenant's own vulnerability+ findings** (gap-closure Phase 8.2).++ A CVE takes a different path from every other indicator, and the reason is+ the point. A hash or an address appears in event telemetry, so "have we seen+ this" is a question for the event lake. A CVE never appears there, so+ sweeping the lake for `CVE-2024-3400` would return zero on every tenant+ forever while looking exactly like a sweep that worked. The router sends a+ vulnerability to the exposure check instead, and records that as a decision+ rather than leaving it as an omission.++ Exposure means an **unremediated finding in the tenant's own vulnerability+ data** whose CVE matches. Inferring it by matching the catalogue's vendor and+ product strings against an asset's operating system field is deliberately not+ done: that produces a plausible-looking answer built on string similarity,+ and a case task an analyst has to disprove costs more than no task, because+ the second one they disprove is the last one they read.++ A tenant with no vulnerability data is told exposure **could not be checked**,+ never that they are unaffected. `checked` and `exposed_asset_count` are+ separate fields and `exposed` requires both, so a caller reading the result+ cannot render an unscanned tenant as clean. The task body also says the count+ is a floor rather than a total, because assets with no scan coverage cannot+ appear in it.++ A match additionally sets `is_exploited` on the matching findings. CISA is a+ better source for that field than a scanner that has not caught up, and it is+ the one write on this path that is not a case task.++ Dedup shares the `retro_hunt_sightings` ledger under indicator type `cve`, so+ the catalogue republishing its whole contents on every fetch opens one task+ rather than one a day. It is not charged against the sweep budget: two+ indexed Postgres queries are not a warehouse scan, and charging them against+ a budget sized for the latter would starve the cheaper check that has the+ clearer action attached to it.+- **The `NEW_IOC` events the threat-intel pipeline has always emitted now havea consumer** (gap-closure Phase 8.1).diff --git a/GAP_CLOSURE_PROGRESS.md b/GAP_CLOSURE_PROGRESS.mdindex d3c80beb1..af577e33c 100644--- a/GAP_CLOSURE_PROGRESS.md+++ b/GAP_CLOSURE_PROGRESS.md@@ -430,7 +430,12 @@ suites' job, and anything at all against a live vendor MCP server.- Budget is per tenant, hourly and daily, checked before the sweep so an exhausted budget costs one UPDATE; skipped sweeps are counted so an empty budget is distinguishable from a quiet feed. ClickHouse enforces a bytes-to-read and execution-time ceiling on every query.- Mapping proven, not asserted: `scripts/check_ioc_lake_mapping.py` (static, across three service trees) plus `tests/isolation/test_retro_hunt_live.py` (real writer, real DDL, real generator, live ClickHouse). The first live test was **vacuous** — injecting a dropped `is_ip` left all 11 tests passing, because the `iocs` array matched through the OR — so it now probes each column alone. Measuring that also corrected the reason `toIPv6` is used: ClickHouse coerces the literal, so it is explicitness rather than the difference between matching and not.- Migration `070_retro_hunts.sql` (069 was the latest on disk, not the 063 the plan captured).-- [ ] **8.2 KEV exposure** checked against asset and vulnerability data, opening a case task when exposed.+- [x] **8.2 KEV exposure** checked against asset and vulnerability data, opening a case task when exposed.+ - A CVE takes a different path from every other indicator, because it never appears in event telemetry: a lake sweep for `CVE-2024-3400` returns zero on every tenant forever while looking exactly like a sweep that worked. `intel_types.route_feed_type` routes it to `kev_exposure` and records that as a decision.+ - Exposure is an unremediated `asset_vulnerabilities` row whose CVE matches. Vendor/product string matching against an asset's OS field is **deliberately not done**: a task an analyst has to disprove is worse than no task.+ - No vulnerability data reports **could not be checked**, never "not exposed". `checked` and `exposed_asset_count` are separate and `exposed` requires both.+ - A match also sets `is_exploited` on the matching findings, since CISA is a better source for that field than a scanner that has not caught up.+ - Dedup shares `retro_hunt_sightings` under type `cve`, so the catalogue republishing its whole contents on every fetch does not open a task a day. Not charged against the sweep budget: two indexed Postgres queries, not a warehouse scan.- [ ] **8.3 Hunting agent** turning a hypothesis into a plan, read-only queries and evidence-backed findings; new `aisoc-hunt` role alias.- [x] **8.4 Hunt library** grown from 5 to at least 50, each with positive and negative synthetic scenarios, and `hunts/README.md` corrected where it cites a script that does not exist.- **68 hunts**, 63 new, across the ATT&CK tactics and the log sources AiSOC ships connectors for. Executable means replayed against a scenario and observed to fire; it is not a claim a hunt fires on a given deployment's telemetry, and `hunts/README.md` states the distinction.diff --git a/ROADMAP.md b/ROADMAP.mdindex 953aea17e..ea8ce0ccf 100644--- a/ROADMAP.md+++ b/ROADMAP.md@@ -44,7 +44,7 @@ A proof-first, security-first program to make every README claim gate-backed, cl- **Phase D (breadth):** D1 eight connectors (QRadar/Exabeam/Securonix/Devo/Netskope/Windows-Sysmon/Zeek-Suricata/syslog-CEF) · D2 AI/LLM-usage audit connector + `llm-*` detections + hot/cold lake tiering · D3 live-vendor mock-server smoke.- **Phase E (prove it):** E1 CI-gated benchmark scoreboard tied to a deterministic live-agent MITRE-accuracy run.-The claim-to-gate matrix stands at **222 rows — 214 GATED / 8 PARTIAL / 0 NO GATE** — **every product claim is backed by a failing test** and the ratchet (`MAX_NO_GATE=0`) forbids any regression. The 8 remaining PARTIAL rows are honest, named deferrals; none was relabelled without building the gate it names. Count the table rows with `python3 scripts/check_claim_gate_matrix.py` rather than trusting a figure quoted in prose — this line has gone stale before. The six lettered deferrals are scoped in [`docs/audit/DEFERRED_SUBPHASES.md`](docs/audit/DEFERRED_SUBPHASES.md).+The claim-to-gate matrix stands at **224 rows — 216 GATED / 8 PARTIAL / 0 NO GATE** — **every product claim is backed by a failing test** and the ratchet (`MAX_NO_GATE=0`) forbids any regression. The 8 remaining PARTIAL rows are honest, named deferrals; none was relabelled without building the gate it names. Count the table rows with `python3 scripts/check_claim_gate_matrix.py` rather than trusting a figure quoted in prose — this line has gone stale before. The six lettered deferrals are scoped in [`docs/audit/DEFERRED_SUBPHASES.md`](docs/audit/DEFERRED_SUBPHASES.md).## v4.0 — Shippeddiff --git a/apps/docs/docs/compliance/evidence-pack.md b/apps/docs/docs/compliance/evidence-pack.mdindex 07832f854..be342ccf7 100644--- a/apps/docs/docs/compliance/evidence-pack.md+++ b/apps/docs/docs/compliance/evidence-pack.md@@ -33,7 +33,7 @@ and says so.| Prompt-injection resistance | Three adversarial corpora: 32 payloads scanned in isolation and 71 incident pairs whose payloads are placed in attacker-controllable fields, both against a clean twin with a recall floor and a false-positive ceiling, plus 34 held-out pairs authored after the guard was frozen and carrying no floor at all, on which the guard measures 7.1% | `services/agents/tests/adversarial/` || Claim-to-gate traceability | Every product claim mapped to the CI job that fails when it stops being true | `docs/audit/CLAIM_TO_GATE_MATRIX.md` |-The last row is the one worth reading first. It is the honest index: 214 rows+The last row is the one worth reading first. It is the honest index: 216 rows`GATED`, 8 `PARTIAL` with the specific gap named, and a ratchet that refusesto let a row sit at `NO GATE`.diff --git a/apps/docs/docs/concepts/retro-hunts.md b/apps/docs/docs/concepts/retro-hunts.mdindex b152d5317..476791926 100644--- a/apps/docs/docs/concepts/retro-hunts.md+++ b/apps/docs/docs/concepts/retro-hunts.md@@ -95,8 +95,40 @@ exactly the deployments that could not answer. URLs are swept through connectedSIEMs, which do carry a URL field.**CVEs are not swept against telemetry.** A CVE identifier does not appear in-event data, so searching for one would match nothing anywhere. KEV entries are-checked against asset and vulnerability inventory instead.+event data, so searching for one would match nothing anywhere. KEV entries take+a different path, described next.++## KEV exposure++When the CISA catalogue publishes a vulnerability as actively exploited, AiSOC+asks a different question: **do we run the affected thing?**++Exposure means the tenant's own vulnerability findings contain an unremediated+entry whose CVE matches. That is narrow on purpose. AiSOC does **not** infer+exposure by matching the catalogue's vendor and product strings against an+asset's operating system field: that produces a plausible-looking answer built+on string similarity, and a case task an analyst has to disprove costs more+than no task at all, because the second one they disprove is the last one they+read.++A match does three things:++1. opens a case with a task naming the affected assets, most critical first;+2. sets `is_exploited` on the matching findings, because CISA is a better+ source for that field than most scanners and many lag it;+3. records the CVE in the same dedup ledger an IOC sighting uses, so the+ catalogue republishing its whole contents on every fetch does not open a+ task a day.++A tenant with no vulnerability data is told **exposure could not be checked**,+not that they are unaffected. The task body also says the count is a floor+rather than a total, because assets with no scan coverage cannot appear in it.++The exposure check is not charged against the sweep budget. It is two indexed+queries against the tenant's own Postgres rows rather than a warehouse scan,+and charging it against a budget sized for the latter would starve the cheaper+check that has the clearer action attached to it. The per-tenant opt-in still+applies.All three hash types read the same column. The lake writer stores the firstfingerprint an event carries whatever algorithm produced it, so an MD5 lands indiff --git a/docs/audit/CLAIM_TO_GATE_MATRIX.md b/docs/audit/CLAIM_TO_GATE_MATRIX.mdindex ebefac75c..51d4bf99f 100644--- a/docs/audit/CLAIM_TO_GATE_MATRIX.md+++ b/docs/audit/CLAIM_TO_GATE_MATRIX.md@@ -230,6 +230,8 @@ Statuses: `GATED` (a CI job fails when the claim stops being true) · `PARTIAL`| A retro-hunt sweeps lake columns the lake writer actually fills | `plans/aisoc_gap_closure_plan.plan.md` (8.1); `apps/docs/docs/concepts/retro-hunts.md` | `ci.yml :: python-lint` (`check_ioc_lake_mapping.py` reads `lake_writer._COLUMNS`, the `raw_events` DDL and the three threat-intel clients, and fails when a swept column is not one the writer writes, a declared OCSF path is not one `event_to_row` reads, an `is_array`/`is_ip` flag disagrees with the column type, or a feed type any client emits routes to unknown) + `isolation-live.yml` (`test_retro_hunt_live.py` drives a real OCSF event through the real writer into a live ClickHouse and probes **each mapped column on its own**, because an OR hides a broken branch behind a working one) | GATED | Proven against injected faults rather than only against the tree as committed: a column renamed to one the writer does not fill, a dropped `is_ip`, and a topic disagreement each reproduce a finding. The first version of the live test passed with `is_ip` removed, because `iocs` carries the address as a plain string and masked the typed column; per-column probing is what closed that || One published indicator opens at most one alert per tenant | `plans/aisoc_gap_closure_plan.plan.md` (8.1) | `ci.yml :: api tests` (`test_retro_hunt.py` asserts the generated sweep is an aggregate with no selectable event column, so a thousand matches cannot become a thousand rows) + `isolation-live.yml` (`test_retro_hunt_live.py` inserts 1,000 matching events and asserts the sweep returns exactly one row counting all of them, with the distinct-value evidence sets still capped) | GATED | Two independent dedup layers, and the row asserts both: the `retro_hunt_sightings` UNIQUE constraint stops the alert being attempted on a later sweep, and the `alerts` per-tenant partial unique index on `idempotency_key` stops a duplicate landing if the branching is ever wrong || A retro-hunt that could not check a surface says so rather than reporting zero sightings | `plans/aisoc_gap_closure_plan.plan.md` (8.1) | `ci.yml :: api tests` (`test_retro_hunt.py` — `checked` and `sightings` are separate fields and `matched` requires both; an unreachable lake, an exceeded budget, an unreadable result and a SIEM source with no field mapping each produce a distinct named gap, and a sweep that matched on one surface while another failed still carries the gap) | GATED | The same rule `agent_tools.siem_search` draws for a backend that errored. Not gated: whether an operator reads the gap. The alert body places it under a heading rather than in a field a renderer could drop |+| A KEV entry is checked against a tenant's own vulnerability findings, not inferred from software names | `plans/aisoc_gap_closure_plan.plan.md` (8.2); `apps/docs/docs/concepts/retro-hunts.md` (KEV exposure) | `ci.yml :: api tests` (`test_kev_exposure.py` — a CVE routes to the exposure path and never to a telemetry sweep, asserted in both directions so no type can claim both; a value without CVE shape is refused rather than queried, because a scanner's `cve_id` column returns zero rows for an advisory title and that zero would read as "not exposed") | GATED | Exposure is drawn from `asset_vulnerabilities` rows. Matching the catalogue's vendor and product strings against an asset's OS field is deliberately not done: it would produce a plausible answer built on string similarity, and a task an analyst has to disprove is worse than no task |+| A tenant with no vulnerability data is told exposure could not be checked, not that they are unaffected | `apps/docs/docs/concepts/retro-hunts.md` (KEV exposure) | `ci.yml :: api tests` (`test_kev_exposure.py` — `checked` and `exposed_asset_count` are separate fields and `exposed` requires both, so a caller reading the dataclass cannot render an unscanned tenant as clean; the wording is asserted against the module constant rather than a copy written in the test) | GATED | The case-task body additionally states the count is a floor rather than a total, because assets with no scan coverage cannot appear in it. Not gated: whether an analyst reads that sentence |## Summary@@ -237,7 +239,7 @@ Statuses: `GATED` (a CI job fails when the claim stops being true) · `PARTIAL`## Summary-- GATED: 214+- GATED: 216- PARTIAL: 8- NO GATE: 0 (**every claim is backed by a failing test.** The last NO GATE — the weekly benchmark scoreboard running live against `main` — closed in Phase E1: `scripts/check_scoreboard.py` ties the published scoreboard to a deterministic per-PR live-agent MITRE-accuracy run, while the funded weekly `wet-eval.yml` appends the LLM-tier rows. The remaining PARTIAL rows are honest, named deferrals — each states the specific gap and what closes it — not unproven claims.)diff --git a/services/api/app/services/retro_hunt/kev_exposure.py b/services/api/app/services/retro_hunt/kev_exposure.pynew file mode 100644index 000000000..f93ea9341--- /dev/null+++ b/services/api/app/services/retro_hunt/kev_exposure.py@@ -0,0 +1,374 @@+"""When CISA says a vulnerability is being exploited, check whether we have it.++Gap-closure Phase 8.2.++The CISA Known Exploited Vulnerabilities catalogue is the one threat-intel+feed in this deployment that needs no API key and ships on by default, and it+flows through the same pipeline as every other indicator: an entry becomes a+``NEW_IOC`` event with ``type: "vulnerability"``. Until Phase 8.1 nothing+consumed those events at all. This module is what happens to the ones that+carry a CVE.++Why a CVE takes a different path from every other indicator+-----------------------------------------------------------++A hash, an address or a domain is something that appears in event telemetry,+so the question "have we seen this" is a question for the event lake. A CVE+is not. It never appears in a process-creation event or a firewall log, and+sweeping the lake for the string "CVE-2024-3400" would return zero on every+tenant forever while looking exactly like a sweep that worked. The question a+CVE asks is "do we run the affected thing", and the only surfaces that can+answer it are the asset inventory and whatever vulnerability scanner feeds it.++So ``intel_types.route_feed_type`` routes a vulnerability here instead, and+records that as a decision rather than as an omission.++What "exposed" means, and what it deliberately does not+--------------------------------------------------------++Exposed means the tenant's own vulnerability data has an **unremediated**+finding whose CVE matches. That is a narrow definition on purpose.++It does not mean "an asset runs software CISA named". Matching a KEV entry's+vendor and product strings against an asset's OS field would produce a+plausible-looking answer built on string similarity, and a case task that+claims a host is exposed when it is not is worse than no task: it costs an+analyst the time to disprove it, and the second one they disprove is the last+one they read. If a tenant has no vulnerability data, this reports that it+could not check rather than reporting no exposure, which is the same rule the+lake sweep follows.++The KEV catalogue is authoritative about exploitation, so a matching finding+also has ``is_exploited`` set. That is a real change to the tenant's data and+it is the one write here that is not a case task: a scanner that has not+caught up on exploitation status is the ordinary case, and CISA is a better+source for that field than most scanners.+"""++from __future__ import annotations++import logging+import re+import uuid+from dataclasses import dataclass, field+from datetime import UTC, datetime++from sqlalchemy import select, update+from sqlalchemy.ext.asyncio import AsyncSession++from app.models.asset import Asset, AssetVulnerability+from app.models.case import Case, CaseTask+from app.models.retro_hunt import RetroHuntSettings, RetroHuntSighting++logger = logging.getLogger(__name__)++#: The indicator type CVE sightings are recorded under in+#: ``retro_hunt_sightings``. Not one of the Phase 4 searchable types, because+#: nothing searches telemetry for it; it shares the table so one indicator+#: opens one case task per tenant under the same UNIQUE constraint that stops+#: an IOC opening a second alert.+CVE_INDICATOR_TYPE = "cve"++#: Assets above this count are summarised rather than listed. A case task is+#: read by a person, and a task body naming nine hundred hosts is not.+_MAX_LISTED_ASSETS = 25++_CVE = re.compile(r"^CVE-\d{4}-\d{4,}$", re.IGNORECASE)++#: What a tenant with no vulnerability data is told.+#:+#: A module constant rather than an inline string because the last sentence is+#: the load-bearing one and it has to be assertable. "No unremediated findings"+#: and "nothing has ever scanned you" both render as zero exposed assets, and+#: only one of them is reassuring.+NO_VULN_DATA_REASON = (+ "This tenant has no vulnerability data in AiSOC, so exposure could NOT be checked. "+ "Connect a vulnerability scanner or import an asset inventory. "+ "This is not a statement that the tenant is unaffected."+)+++@dataclass+class ExposureResult:+ """What the tenant's own data said about one KEV entry."""++ cve_id: str+ #: False when the tenant has no vulnerability data at all. A caller must+ #: not read ``exposed_assets == []`` without checking this: "we scanned and+ #: you are clean" and "nothing has ever scanned you" are different answers+ #: and only one of them is reassuring.+ checked: bool+ exposed_asset_names: list[str] = field(default_factory=list)+ exposed_asset_count: int = 0+ findings_updated: int = 0+ case_id: uuid.UUID | None = None+ task_id: uuid.UUID | None = None+ deduplicated: bool = False+ unavailable_reason: str | None = None++ @property+ def exposed(self) -> bool:+ return self.checked and self.exposed_asset_count > 0+++def is_cve(value: str) -> bool:+ """Whether a feed's value has the shape of a CVE identifier.++ Checked rather than trusted. A feed publishing ``type: vulnerability``+ with a free-text advisory title in the value would otherwise produce a+ query for a string no scanner ever records, and the zero rows would read+ as "not exposed".+ """+ return bool(_CVE.match(str(value or "").strip()))+++async def _tenant_has_vulnerability_data(db: AsyncSession, tenant_id: uuid.UUID) -> bool:+ """Whether anything has ever written a vulnerability finding for a tenant.++ This is the difference between a clean answer and no answer, so it is a+ separate query rather than an inference from an empty result set.+ """+ row = await db.execute(select(AssetVulnerability.id).where(AssetVulnerability.tenant_id == tenant_id).limit(1))+ return row.scalar_one_or_none() is not None+++async def check_exposure(+ db: AsyncSession,+ *,+ tenant_id: uuid.UUID,+ cve_id: str,+ feed_source: str,+ intel_first_seen_at: datetime | None = None,+ now: datetime | None = None,+) -> ExposureResult:+ """Check one tenant's vulnerability data for one KEV entry.++ Opens at most one case task per (tenant, CVE), enforced by the same+ ``retro_hunt_sightings`` UNIQUE constraint that keeps an IOC from opening+ a second alert. The caller owns the transaction.+ """+ now = now or datetime.now(UTC)+ cve = str(cve_id).strip().upper()++ if not is_cve(cve):+ return ExposureResult(+ cve_id=cve,+ checked=False,+ unavailable_reason=(+ f"{cve_id!r} does not have the shape of a CVE identifier, so the tenant's vulnerability data was NOT checked."+ ),+ )++ if not await _tenant_has_vulnerability_data(db, tenant_id):+ return ExposureResult(+ cve_id=cve,+ checked=False,+ unavailable_reason=NO_VULN_DATA_REASON,+ )++ rows = (+ await db.execute(+ select(AssetVulnerability, Asset)+ .join(Asset, Asset.id == AssetVulnerability.asset_id)+ .where(+ AssetVulnerability.tenant_id == tenant_id,+ Asset.tenant_id == tenant_id,+ AssetVulnerability.cve_id.ilike(cve),+ AssetVulnerability.remediated_at.is_(None),+ )+ )+ ).all()++ if not rows:+ return ExposureResult(cve_id=cve, checked=True)++ # Order by asset criticality so a truncated list shows the ones that+ # matter. `critical` first, then high, then the rest.+ order = {"critical": 0, "high": 1, "medium": 2, "low": 3}+ assets = sorted({(a.name, a.criticality) for _v, a in rows}, key=lambda pair: (order.get(str(pair[1]).lower(), 9), pair[0]))+ names = [name for name, _crit in assets]++ # CISA is authoritative on exploitation, and most scanners lag it. This is+ # the one write here that is not a case task.+ updated = await db.execute(+ update(AssetVulnerability)+ .where(+ AssetVulnerability.tenant_id == tenant_id,+ AssetVulnerability.cve_id.ilike(cve),+ AssetVulnerability.remediated_at.is_(None),+ AssetVulnerability.is_exploited.is_(False),+ )+ .values(is_exploited=True)+ )++ existing = (+ await db.execute(+ select(RetroHuntSighting).where(+ RetroHuntSighting.tenant_id == tenant_id,+ RetroHuntSighting.indicator_type == CVE_INDICATOR_TYPE,+ RetroHuntSighting.indicator_value == cve,+ )+ )+ ).scalar_one_or_none()++ if existing is not None:+ existing.last_matched_at = now+ existing.sightings = len(assets)+ existing.times_seen += 1+ existing.updated_at = now+ return ExposureResult(+ cve_id=cve,+ checked=True,+ exposed_asset_names=names[:_MAX_LISTED_ASSETS],+ exposed_asset_count=len(assets),+ findings_updated=int(updated.rowcount or 0),+ deduplicated=True,+ unavailable_reason=None,+ )++ case, task = await _open_case_task(+ db,+ tenant_id=tenant_id,+ cve=cve,+ feed_source=feed_source,+ assets=assets,+ intel_first_seen_at=intel_first_seen_at,+ now=now,+ )++ db.add(+ RetroHuntSighting(+ tenant_id=tenant_id,+ indicator_type=CVE_INDICATOR_TYPE,+ indicator_value=cve,+ feed_source=feed_source,+ intel_first_seen_at=intel_first_seen_at,+ first_matched_at=now,+ last_matched_at=now,+ sightings=len(assets),+ matched_surfaces=[{"surface": "vulnerability_inventory", "assets": len(assets)}],+ times_seen=1,+ created_at=now,+ updated_at=now,+ )+ )+ logger.info("retro_hunt.kev_exposure tenant=%s assets=%d", tenant_id, len(assets))++ return ExposureResult(+ cve_id=cve,+ checked=True,+ exposed_asset_names=names[:_MAX_LISTED_ASSETS],+ exposed_asset_count=len(assets),+ findings_updated=int(updated.rowcount or 0),+ case_id=case.id,+ task_id=task.id,+ )+++async def _open_case_task(+ db: AsyncSession,+ *,+ tenant_id: uuid.UUID,+ cve: str,+ feed_source: str,+ assets: list[tuple[str, str]],+ intel_first_seen_at: datetime | None,+ now: datetime,+) -> tuple[Case, CaseTask]:+ """Open the case and the one task this CVE gets for this tenant."""+ listed = assets[:_MAX_LISTED_ASSETS]+ overflow = len(assets) - len(listed)++ lines = [+ f"{feed_source} published {cve} as a known exploited vulnerability.",+ "",+ "Provenance",+ f" Feed: {feed_source}",+ (+ f" Added to the catalogue: {intel_first_seen_at.date().isoformat()}"+ if intel_first_seen_at+ else " Added to the catalogue: not published by this feed"+ ),+ "",+ f"Exposure in your own data: {len(assets)} unremediated finding(s)",+ ]+ lines += [f" {name} ({criticality})" for name, criticality in listed]+ if overflow:+ lines.append(f" and {overflow} more asset(s), listed on the vulnerability page")+ lines += [+ "",+ "This is drawn from your vulnerability findings, not inferred from software names.",+ "Assets with no scan coverage will not appear here, so this is a floor rather than a total.",+ ]++ case = Case(+ tenant_id=tenant_id,+ case_number=f"KEV-{cve.replace('CVE-', '')}-{int(now.timestamp())}",+ title=f"Known exploited vulnerability {cve} affects {len(assets)} asset(s)",+ description="\n".join(lines),+ case_type="vulnerability_exposure",+ priority="high" if any(c in {"critical", "high"} for _n, c in assets) else "medium",+ severity="high",+ status="open",+ tags=["kev", f"cve:{cve}", f"feed:{feed_source}"],+ )+ db.add(case)+ await db.flush()++ task = CaseTask(+ case_id=case.id,+ tenant_id=tenant_id,+ title=f"Patch or mitigate {cve} on {len(assets)} asset(s)",+ description=(+ f"CISA lists {cve} as actively exploited. Your vulnerability data shows "+ f"{len(assets)} unremediated finding(s). Patch, mitigate, or record why the "+ f"affected assets are not reachable."+ ),+ status="pending",+ )+ db.add(task)+ await db.flush()+ return case, task+++async def handle_kev_entry(+ db: AsyncSession,+ *,+ settings_row: RetroHuntSettings,+ cve_id: str,+ feed_source: str,+ intel_first_seen_at: datetime | None = None,+ now: datetime | None = None,+) -> ExposureResult:+ """Entry point from the intel consumer for one tenant and one CVE.++ Gated on the same per-tenant opt-in as a lake sweep. Deliberately not on+ the sweep *budget*: an exposure check is two indexed queries against the+ tenant's own Postgres rows rather than a warehouse scan, so charging it+ against a budget sized for the latter would starve the cheap check to+ protect against the expensive one.+ """+ if not settings_row.enabled:+ return ExposureResult(+ cve_id=str(cve_id),+ checked=False,+ unavailable_reason="This tenant has not opted in to retro-hunts.",+ )+ return await check_exposure(+ db,+ tenant_id=settings_row.tenant_id,+ cve_id=cve_id,+ feed_source=feed_source,+ intel_first_seen_at=intel_first_seen_at,+ now=now,+ )+++__all__ = [+ "CVE_INDICATOR_TYPE",+ "ExposureResult",+ "NO_VULN_DATA_REASON",+ "check_exposure",+ "handle_kev_entry",+ "is_cve",+]diff --git a/services/api/app/workers/retro_hunt_consumer.py b/services/api/app/workers/retro_hunt_consumer.pyindex d0c8208a2..1416bce3a 100644--- a/services/api/app/workers/retro_hunt_consumer.py+++ b/services/api/app/workers/retro_hunt_consumer.py@@ -49,6 +49,7 @@ from app.core.config import settingsfrom app.db.cross_tenant import assert_cross_tenant_sessionfrom app.db.database import AsyncSessionLocalfrom app.services.retro_hunt.intel_types import route_feed_type+from app.services.retro_hunt.kev_exposure import handle_kev_entryfrom app.services.retro_hunt.service import (IntelIndicator,TenantSweepReport,@@ -116,6 +117,87 @@ def _first_seen(data: dict[str, Any], envelope_ts: Any) -> datetime | None:return None+def cve_from_event(payload: dict[str, Any]) -> tuple[str, str, datetime | None] | None:+ """Pull a CVE out of a ``NEW_IOC`` that routes to the exposure check.++ Returns ``(cve_id, feed_source, first_seen)`` or ``None``. A CVE takes a+ different path from every other indicator because it never appears in+ event telemetry: sweeping the lake for one would return zero on every+ tenant while looking exactly like a sweep that worked. See+ :mod:`app.services.retro_hunt.kev_exposure`.+ """+ data = payload.get("data") or {}+ value = str(data.get("value") or "").strip()+ if not value:+ return None+ if not route_feed_type(str(data.get("type") or "")).is_vulnerability:+ return None+ source = str(payload.get("source") or data.get("source") or "unknown feed")+ return value, source, _first_seen(data, payload.get("timestamp"))+++async def handle_kev_indicator(cve_id: str, feed_source: str, first_seen: datetime | None) -> list[TenantSweepReport]:+ """Fan one KEV entry out to every tenant that opted in.++ Same session and tenant-binding shape as :func:`handle_indicator`: the+ read of who opted in is genuinely cross-tenant, and every write after it+ rebinds the tenant so the case and task inserts are RLS-enforced.+ """+ reports: list[TenantSweepReport] = []+ async with AsyncSessionLocal() as db:+ await assert_cross_tenant_session(db, "retro-hunt KEV fan-out")+ tenants = await opted_in_tenants(db)+ if not tenants:+ return reports++ for settings_row in tenants:+ try:+ await db.execute(+ text("SELECT set_config('app.current_tenant_id', :t, true)"),+ {"t": str(settings_row.tenant_id)},+ )+ result = await handle_kev_entry(+ db,+ settings_row=settings_row,+ cve_id=cve_id,+ feed_source=feed_source,+ intel_first_seen_at=first_seen,+ )+ except Exception as exc: # noqa: BLE001 - one tenant must not stop the rest+ logger.exception(+ "retro_hunt.kev_check_failed tenant=%s err=%s",+ settings_row.tenant_id,+ type(exc).__name__,+ )+ reports.append(+ TenantSweepReport(+ tenant_id=settings_row.tenant_id,+ outcome="failed",+ detail=type(exc).__name__,+ )+ )+ continue++ reports.append(+ TenantSweepReport(+ tenant_id=settings_row.tenant_id,+ outcome="swept" if result.checked else "skipped_unsupported",+ matched=result.exposed,+ deduplicated=result.deduplicated,+ detail=(+ result.unavailable_reason+ or (+ f"{result.exposed_asset_count} exposed asset(s); case task opened."+ if result.exposed+ else "No unremediated findings for this CVE."+ )+ ),+ )+ )+ await db.commit()+ return reports++def indicator_from_event(payload: dict[str, Any]) -> IntelIndicator | None:"""Turn one decoded ``NEW_IOC`` into a typed indicator, or refuse it.@@ -247,6 +329,14 @@ async def run_forever() -> None:# subscription: one bad message is not a reason to stop# consuming a topic that is otherwise fine.continue+ kev = cve_from_event(payload)+ if kev is not None:+ try:+ await handle_kev_indicator(*kev)+ except Exception as exc: # noqa: BLE001 - keep consuming+ logger.exception("retro_hunt.kev_fan_out_failed err=%s", type(exc).__name__)+ continue+indicator = indicator_from_event(payload)if indicator is None:routing = route_feed_type(str((payload.get("data") or {}).get("type") or ""))diff --git a/services/api/tests/test_kev_exposure.py b/services/api/tests/test_kev_exposure.pynew file mode 100644index 000000000..b6f1b9201--- /dev/null+++ b/services/api/tests/test_kev_exposure.py@@ -0,0 +1,274 @@+"""Gap-closure Phase 8.2: KEV exposure.++Three groups, each covering a way this reports something it does not know.++**A CVE never reaches the telemetry sweep.** Routing is the whole design+decision: a CVE does not appear in event data, so sweeping the lake for one+returns zero on every tenant forever while looking exactly like a sweep that+worked.++**No vulnerability data is not "no exposure".** The single most damaging thing+this surface could do is tell a tenant with no scanner that they are+unaffected. ``checked`` and ``exposed_asset_count`` are separate for that+reason and ``exposed`` requires both.++**Exposure is drawn from findings, not inferred from software names.** Matching+a KEV entry's vendor and product strings against an asset's OS field would+produce a plausible answer built on string similarity, and a case task an+analyst has to disprove is worse than no task at all.+"""++from __future__ import annotations++import uuid+from dataclasses import dataclass+from datetime import datetime++import pytest+from app.services.retro_hunt.intel_types import route_feed_type+from app.services.retro_hunt.kev_exposure import (+ CVE_INDICATOR_TYPE,+ NO_VULN_DATA_REASON,+ ExposureResult,+ is_cve,+)+from app.services.retro_hunt.sweep import DEFAULT_LOOKBACK_DAYS++TENANT = uuid.UUID("11111111-1111-1111-1111-111111111111")+++# ------------------------------------------------------------------ routing++[email protected]("feed_type", ["vulnerability", "Vulnerability", "cve", "CVE"])+def test_a_cve_routes_to_exposure_not_to_a_telemetry_sweep(feed_type: str) -> None:+ """The decision this phase turns on.++ A lake sweep for "CVE-2024-3400" matches nothing on every deployment and+ would report a confident zero. Routing it elsewhere is what stops that.+ """+ routing = route_feed_type(feed_type)+ assert routing.is_vulnerability is True+ assert routing.indicator_type is None+ assert routing.reason and "not against event telemetry" in routing.reason+++def test_the_indicator_path_and_the_cve_path_cannot_both_claim_one_event() -> None:+ """A type routes to exactly one of the two paths, never both."""+ for feed_type in ("vulnerability", "ipv4-addr", "sha256", "domain-name", "url"):+ routing = route_feed_type(feed_type)+ assert not (routing.is_vulnerability and routing.indicator_type), feed_type+++# ------------------------------------------------------------- value shapes++[email protected]("value", ["CVE-2024-3400", "cve-2021-44228", "CVE-2019-0708", "CVE-2024-123456"])+def test_real_cve_identifiers_are_accepted(value: str) -> None:+ assert is_cve(value) is True+++ "value",+ [+ "",+ " ",+ "Apache HTTP Server path traversal",+ "CVE-24-3400",+ "CVE-2024-",+ "2024-3400",+ "GHSA-xxxx-yyyy-zzzz",+ ],+)+def test_a_value_that_is_not_a_cve_is_refused_rather_than_queried(value: str) -> None:+ """A feed publishing an advisory title under ``type: vulnerability``.++ Querying a scanner's ``cve_id`` column for free text returns zero rows,+ and that zero would be reported as "not exposed".+ """+ assert is_cve(value) is False+++def test_a_refused_value_reports_a_reason_rather_than_a_clean_result() -> None:+ result = ExposureResult(+ cve_id="not a cve",+ checked=False,+ unavailable_reason="does not have the shape of a CVE identifier",+ )+ assert result.exposed is False+ assert result.checked is False+ assert result.unavailable_reason+++# ------------------------------------------- the distinction that matters+++def test_no_vulnerability_data_is_not_no_exposure() -> None:+ """The most damaging thing this surface could report.++ ``exposed`` requires ``checked``, so a tenant with no scanner cannot be+ rendered as clean by any caller that reads the dataclass rather than the+ count.+ """+ result = ExposureResult(cve_id="CVE-2024-3400", checked=False, unavailable_reason="unused")+ assert result.exposed is False+ assert result.exposed_asset_count == 0++ # Asserted against the wording the module actually emits rather than a+ # copy written here. A test that restates a message passes while the+ # message says something else, which is the shape of defect this phase+ # keeps finding.+ assert "could NOT be checked" in NO_VULN_DATA_REASON+ assert "not a statement that the tenant is unaffected" in NO_VULN_DATA_REASON+++def test_a_genuine_clean_answer_is_distinguishable_from_an_unchecked_one() -> None:+ """Scanned and clean, versus never scanned. Different facts."""+ clean = ExposureResult(cve_id="CVE-2024-3400", checked=True)+ unchecked = ExposureResult(cve_id="CVE-2024-3400", checked=False, unavailable_reason="no scanner")+ assert clean.exposed is False+ assert unchecked.exposed is False+ # Both are "not exposed" by the boolean, and only one of them is+ # reassuring. The fields that carry the difference must not be equal.+ assert clean.checked != unchecked.checked+ assert (clean.unavailable_reason is None) != (unchecked.unavailable_reason is None)+++def test_an_exposed_result_carries_the_assets_it_is_asserting_about() -> None:+ result = ExposureResult(+ cve_id="CVE-2024-3400",+ checked=True,+ exposed_asset_names=["fw-edge-01", "fw-edge-02"],+ exposed_asset_count=2,+ findings_updated=2,+ )+ assert result.exposed is True+ assert result.exposed_asset_names+++def test_the_asset_list_is_bounded_while_the_count_is_not() -> None:+ """A task body naming nine hundred hosts is not read by anyone.++ The count stays exact so the task never understates the problem; only the+ enumeration is capped.+ """+ from app.services.retro_hunt.kev_exposure import _MAX_LISTED_ASSETS++ result = ExposureResult(+ cve_id="CVE-2024-3400",+ checked=True,+ exposed_asset_names=[f"host-{i}" for i in range(_MAX_LISTED_ASSETS)],+ exposed_asset_count=900,+ )+ assert len(result.exposed_asset_names) == _MAX_LISTED_ASSETS+ assert result.exposed_asset_count == 900+++# ------------------------------------------------------------------- dedup+++def test_cve_sightings_share_the_ioc_dedup_table_under_their_own_type() -> None:+ """One CVE opens one case task per tenant, by the same mechanism.++ The type is not one of the Phase 4 searchable types, because nothing+ searches telemetry for it. It shares the table so the UNIQUE constraint+ that stops an IOC alerting twice also stops a KEV entry opening a task a+ day, which it would otherwise do: the catalogue republishes its whole+ contents on every fetch.+ """+ assert CVE_INDICATOR_TYPE == "cve"+ from app.services.retro_hunt.ioc_fields import SWEEPABLE_TYPES++ assert CVE_INDICATOR_TYPE not in SWEEPABLE_TYPES+++def test_the_exposure_check_is_not_charged_against_the_sweep_budget() -> None:+ """Two indexed Postgres queries, not a warehouse scan.++ Recorded as a test because the alternative is a plausible mistake: reusing+ the budget would let a busy day of IOCs starve the cheap check that has+ the clearest action attached to it.+ """+ import inspect++ from app.services.retro_hunt import kev_exposure++ source = inspect.getsource(kev_exposure)+ assert "_consume_budget" not in source+ assert "max_sweeps_per_hour" not in source+ # The opt-in still applies; only the budget does not.+ assert "settings_row.enabled" in source+++def test_the_lake_sweep_default_window_is_untouched_by_this_phase() -> None:+ """A guard against the exposure path quietly changing sweep behaviour."""+ assert DEFAULT_LOOKBACK_DAYS == 30+++# --------------------------------------------------- the consumer's routing+++@dataclass+class _Payload:+ data: dict+ source: str = "cisa-kev"++ def as_dict(self) -> dict:+ return {"event_type": "NEW_IOC", "source": self.source, "timestamp": "2026-09-20T00:00:00Z", "data": self.data}+++def test_the_consumer_sends_a_kev_entry_to_exposure_and_not_to_a_sweep() -> None:+ from app.workers.retro_hunt_consumer import cve_from_event, indicator_from_event++ payload = _Payload({"type": "vulnerability", "value": "CVE-2024-3400", "date_added": "2026-09-18"}).as_dict()+ kev = cve_from_event(payload)+ assert kev is not None+ cve, source, first_seen = kev+ assert cve == "CVE-2024-3400"+ assert source == "cisa-kev"+ assert isinstance(first_seen, datetime)+ assert first_seen.tzinfo is not None+ # And the indicator path refuses the same event, so the two cannot both+ # act on it.+ assert indicator_from_event(payload) is None+++def test_the_consumer_sends_an_ordinary_indicator_to_the_sweep_and_not_to_exposure() -> None:+ from app.workers.retro_hunt_consumer import cve_from_event, indicator_from_event++ payload = _Payload({"type": "ipv4-addr", "value": "203.0.113.9"}, source="otx").as_dict()+ assert cve_from_event(payload) is None+ indicator = indicator_from_event(payload)+ assert indicator is not None+ assert indicator.indicator_type == "ip"+++def test_a_feed_that_published_no_date_does_not_get_one_invented() -> None:+ """``first_seen`` falls back to the envelope, never to ``now``."""+ from app.workers.retro_hunt_consumer import cve_from_event++ payload = {"event_type": "NEW_IOC", "source": "cisa-kev", "data": {"type": "vulnerability", "value": "CVE-2024-3400"}}+ kev = cve_from_event(payload)+ assert kev is not None+ assert kev[2] is None+++def test_an_empty_value_is_not_routed_anywhere(monkeypatch: pytest.MonkeyPatch) -> None:+ from app.workers.retro_hunt_consumer import cve_from_event++ assert cve_from_event(_Payload({"type": "vulnerability", "value": ""}).as_dict()) is None+++def test_the_case_task_body_states_that_unscanned_assets_are_absent() -> None:+ """A floor, not a total, and it says so.++ An analyst reading "3 assets affected" against an estate with 40% scanner+ coverage will act on the 3 and conclude the rest are fine.+ """+ import inspect++ from app.services.retro_hunt import kev_exposure++ source = inspect.getsource(kev_exposure._open_case_task)+ assert "floor rather than a total" in source+ assert "not inferred from software names" in source