Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

feat(retro-hunt): check KEV entries against a tenant's own vulnerability data (#931)

Beenu Arora90864f5820f2
diff --git a/CHANGELOG.md b/CHANGELOG.md
index e329f472a..27d75f7c0 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -348,6 +348,42 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
minted before this change fail closed rather than outliving the revocation
meant to end them. The refresh path checks it too, and matters more there: a
refresh token outlives an access token by days.
+- **A CISA KEV entry is now checked against the tenant's own vulnerability
+ findings** (gap-closure Phase 8.2).
+
+ A CVE takes a different path from every other indicator, and the reason is
+ the point. A hash or an address appears in event telemetry, so "have we seen
+ this" is a question for the event lake. A CVE never appears there, so
+ sweeping the lake for `CVE-2024-3400` would return zero on every tenant
+ forever while looking exactly like a sweep that worked. The router sends a
+ vulnerability to the exposure check instead, and records that as a decision
+ rather than leaving it as an omission.
+
+ Exposure means an **unremediated finding in the tenant's own vulnerability
+ data** whose CVE matches. Inferring it by matching the catalogue's vendor and
+ product strings against an asset's operating system field is deliberately not
+ done: that produces a plausible-looking answer built on string similarity,
+ and a case task an analyst has to disprove costs more than no task, because
+ the second one they disprove is the last one they read.
+
+ A tenant with no vulnerability data is told exposure **could not be checked**,
+ never that they are unaffected. `checked` and `exposed_asset_count` are
+ separate fields and `exposed` requires both, so a caller reading the result
+ cannot render an unscanned tenant as clean. The task body also says the count
+ is a floor rather than a total, because assets with no scan coverage cannot
+ appear in it.
+
+ A match additionally sets `is_exploited` on the matching findings. CISA is a
+ better source for that field than a scanner that has not caught up, and it is
+ the one write on this path that is not a case task.
+
+ Dedup shares the `retro_hunt_sightings` ledger under indicator type `cve`, so
+ the catalogue republishing its whole contents on every fetch opens one task
+ rather than one a day. It is not charged against the sweep budget: two
+ indexed Postgres queries are not a warehouse scan, and charging them against
+ a budget sized for the latter would starve the cheaper check that has the
+ clearer action attached to it.
+
- **The `NEW_IOC` events the threat-intel pipeline has always emitted now have
a consumer** (gap-closure Phase 8.1).
diff --git a/GAP_CLOSURE_PROGRESS.md b/GAP_CLOSURE_PROGRESS.md
index d3c80beb1..af577e33c 100644
--- a/GAP_CLOSURE_PROGRESS.md
+++ b/GAP_CLOSURE_PROGRESS.md
@@ -430,7 +430,12 @@ suites' job, and anything at all against a live vendor MCP server.
- Budget is per tenant, hourly and daily, checked before the sweep so an exhausted budget costs one UPDATE; skipped sweeps are counted so an empty budget is distinguishable from a quiet feed. ClickHouse enforces a bytes-to-read and execution-time ceiling on every query.
- Mapping proven, not asserted: `scripts/check_ioc_lake_mapping.py` (static, across three service trees) plus `tests/isolation/test_retro_hunt_live.py` (real writer, real DDL, real generator, live ClickHouse). The first live test was **vacuous** — injecting a dropped `is_ip` left all 11 tests passing, because the `iocs` array matched through the OR — so it now probes each column alone. Measuring that also corrected the reason `toIPv6` is used: ClickHouse coerces the literal, so it is explicitness rather than the difference between matching and not.
- Migration `070_retro_hunts.sql` (069 was the latest on disk, not the 063 the plan captured).
-- [ ] **8.2 KEV exposure** checked against asset and vulnerability data, opening a case task when exposed.
+- [x] **8.2 KEV exposure** checked against asset and vulnerability data, opening a case task when exposed.
+ - A CVE takes a different path from every other indicator, because it never appears in event telemetry: a lake sweep for `CVE-2024-3400` returns zero on every tenant forever while looking exactly like a sweep that worked. `intel_types.route_feed_type` routes it to `kev_exposure` and records that as a decision.
+ - Exposure is an unremediated `asset_vulnerabilities` row whose CVE matches. Vendor/product string matching against an asset's OS field is **deliberately not done**: a task an analyst has to disprove is worse than no task.
+ - No vulnerability data reports **could not be checked**, never "not exposed". `checked` and `exposed_asset_count` are separate and `exposed` requires both.
+ - A match also sets `is_exploited` on the matching findings, since CISA is a better source for that field than a scanner that has not caught up.
+ - Dedup shares `retro_hunt_sightings` under type `cve`, so the catalogue republishing its whole contents on every fetch does not open a task a day. Not charged against the sweep budget: two indexed Postgres queries, not a warehouse scan.
- [ ] **8.3 Hunting agent** turning a hypothesis into a plan, read-only queries and evidence-backed findings; new `aisoc-hunt` role alias.
- [x] **8.4 Hunt library** grown from 5 to at least 50, each with positive and negative synthetic scenarios, and `hunts/README.md` corrected where it cites a script that does not exist.
- **68 hunts**, 63 new, across the ATT&CK tactics and the log sources AiSOC ships connectors for. Executable means replayed against a scenario and observed to fire; it is not a claim a hunt fires on a given deployment's telemetry, and `hunts/README.md` states the distinction.
diff --git a/ROADMAP.md b/ROADMAP.md
index 953aea17e..ea8ce0ccf 100644
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -44,7 +44,7 @@ A proof-first, security-first program to make every README claim gate-backed, cl
- **Phase D (breadth):** D1 eight connectors (QRadar/Exabeam/Securonix/Devo/Netskope/Windows-Sysmon/Zeek-Suricata/syslog-CEF) · D2 AI/LLM-usage audit connector + `llm-*` detections + hot/cold lake tiering · D3 live-vendor mock-server smoke.
- **Phase E (prove it):** E1 CI-gated benchmark scoreboard tied to a deterministic live-agent MITRE-accuracy run.
-The claim-to-gate matrix stands at **222 rows — 214 GATED / 8 PARTIAL / 0 NO GATE** — **every product claim is backed by a failing test** and the ratchet (`MAX_NO_GATE=0`) forbids any regression. The 8 remaining PARTIAL rows are honest, named deferrals; none was relabelled without building the gate it names. Count the table rows with `python3 scripts/check_claim_gate_matrix.py` rather than trusting a figure quoted in prose — this line has gone stale before. The six lettered deferrals are scoped in [`docs/audit/DEFERRED_SUBPHASES.md`](docs/audit/DEFERRED_SUBPHASES.md).
+The claim-to-gate matrix stands at **224 rows — 216 GATED / 8 PARTIAL / 0 NO GATE** — **every product claim is backed by a failing test** and the ratchet (`MAX_NO_GATE=0`) forbids any regression. The 8 remaining PARTIAL rows are honest, named deferrals; none was relabelled without building the gate it names. Count the table rows with `python3 scripts/check_claim_gate_matrix.py` rather than trusting a figure quoted in prose — this line has gone stale before. The six lettered deferrals are scoped in [`docs/audit/DEFERRED_SUBPHASES.md`](docs/audit/DEFERRED_SUBPHASES.md).
## v4.0 — Shipped
diff --git a/apps/docs/docs/compliance/evidence-pack.md b/apps/docs/docs/compliance/evidence-pack.md
index 07832f854..be342ccf7 100644
--- a/apps/docs/docs/compliance/evidence-pack.md
+++ b/apps/docs/docs/compliance/evidence-pack.md
@@ -33,7 +33,7 @@ and says so.
| Prompt-injection resistance | Three adversarial corpora: 32 payloads scanned in isolation and 71 incident pairs whose payloads are placed in attacker-controllable fields, both against a clean twin with a recall floor and a false-positive ceiling, plus 34 held-out pairs authored after the guard was frozen and carrying no floor at all, on which the guard measures 7.1% | `services/agents/tests/adversarial/` |
| Claim-to-gate traceability | Every product claim mapped to the CI job that fails when it stops being true | `docs/audit/CLAIM_TO_GATE_MATRIX.md` |
-The last row is the one worth reading first. It is the honest index: 214 rows
+The last row is the one worth reading first. It is the honest index: 216 rows
`GATED`, 8 `PARTIAL` with the specific gap named, and a ratchet that refuses
to let a row sit at `NO GATE`.
diff --git a/apps/docs/docs/concepts/retro-hunts.md b/apps/docs/docs/concepts/retro-hunts.md
index b152d5317..476791926 100644
--- a/apps/docs/docs/concepts/retro-hunts.md
+++ b/apps/docs/docs/concepts/retro-hunts.md
@@ -95,8 +95,40 @@ exactly the deployments that could not answer. URLs are swept through connected
SIEMs, which do carry a URL field.
**CVEs are not swept against telemetry.** A CVE identifier does not appear in
-event data, so searching for one would match nothing anywhere. KEV entries are
-checked against asset and vulnerability inventory instead.
+event data, so searching for one would match nothing anywhere. KEV entries take
+a different path, described next.
+
+## KEV exposure
+
+When the CISA catalogue publishes a vulnerability as actively exploited, AiSOC
+asks a different question: **do we run the affected thing?**
+
+Exposure means the tenant's own vulnerability findings contain an unremediated
+entry whose CVE matches. That is narrow on purpose. AiSOC does **not** infer
+exposure by matching the catalogue's vendor and product strings against an
+asset's operating system field: that produces a plausible-looking answer built
+on string similarity, and a case task an analyst has to disprove costs more
+than no task at all, because the second one they disprove is the last one they
+read.
+
+A match does three things:
+
+1. opens a case with a task naming the affected assets, most critical first;
+2. sets `is_exploited` on the matching findings, because CISA is a better
+ source for that field than most scanners and many lag it;
+3. records the CVE in the same dedup ledger an IOC sighting uses, so the
+ catalogue republishing its whole contents on every fetch does not open a
+ task a day.
+
+A tenant with no vulnerability data is told **exposure could not be checked**,
+not that they are unaffected. The task body also says the count is a floor
+rather than a total, because assets with no scan coverage cannot appear in it.
+
+The exposure check is not charged against the sweep budget. It is two indexed
+queries against the tenant's own Postgres rows rather than a warehouse scan,
+and charging it against a budget sized for the latter would starve the cheaper
+check that has the clearer action attached to it. The per-tenant opt-in still
+applies.
All three hash types read the same column. The lake writer stores the first
fingerprint an event carries whatever algorithm produced it, so an MD5 lands in
diff --git a/docs/audit/CLAIM_TO_GATE_MATRIX.md b/docs/audit/CLAIM_TO_GATE_MATRIX.md
index ebefac75c..51d4bf99f 100644
--- a/docs/audit/CLAIM_TO_GATE_MATRIX.md
+++ b/docs/audit/CLAIM_TO_GATE_MATRIX.md
@@ -230,6 +230,8 @@ Statuses: `GATED` (a CI job fails when the claim stops being true) · `PARTIAL`
| A retro-hunt sweeps lake columns the lake writer actually fills | `plans/aisoc_gap_closure_plan.plan.md` (8.1); `apps/docs/docs/concepts/retro-hunts.md` | `ci.yml :: python-lint` (`check_ioc_lake_mapping.py` reads `lake_writer._COLUMNS`, the `raw_events` DDL and the three threat-intel clients, and fails when a swept column is not one the writer writes, a declared OCSF path is not one `event_to_row` reads, an `is_array`/`is_ip` flag disagrees with the column type, or a feed type any client emits routes to unknown) + `isolation-live.yml` (`test_retro_hunt_live.py` drives a real OCSF event through the real writer into a live ClickHouse and probes **each mapped column on its own**, because an OR hides a broken branch behind a working one) | GATED | Proven against injected faults rather than only against the tree as committed: a column renamed to one the writer does not fill, a dropped `is_ip`, and a topic disagreement each reproduce a finding. The first version of the live test passed with `is_ip` removed, because `iocs` carries the address as a plain string and masked the typed column; per-column probing is what closed that |
| One published indicator opens at most one alert per tenant | `plans/aisoc_gap_closure_plan.plan.md` (8.1) | `ci.yml :: api tests` (`test_retro_hunt.py` asserts the generated sweep is an aggregate with no selectable event column, so a thousand matches cannot become a thousand rows) + `isolation-live.yml` (`test_retro_hunt_live.py` inserts 1,000 matching events and asserts the sweep returns exactly one row counting all of them, with the distinct-value evidence sets still capped) | GATED | Two independent dedup layers, and the row asserts both: the `retro_hunt_sightings` UNIQUE constraint stops the alert being attempted on a later sweep, and the `alerts` per-tenant partial unique index on `idempotency_key` stops a duplicate landing if the branching is ever wrong |
| A retro-hunt that could not check a surface says so rather than reporting zero sightings | `plans/aisoc_gap_closure_plan.plan.md` (8.1) | `ci.yml :: api tests` (`test_retro_hunt.py` — `checked` and `sightings` are separate fields and `matched` requires both; an unreachable lake, an exceeded budget, an unreadable result and a SIEM source with no field mapping each produce a distinct named gap, and a sweep that matched on one surface while another failed still carries the gap) | GATED | The same rule `agent_tools.siem_search` draws for a backend that errored. Not gated: whether an operator reads the gap. The alert body places it under a heading rather than in a field a renderer could drop |
+| A KEV entry is checked against a tenant's own vulnerability findings, not inferred from software names | `plans/aisoc_gap_closure_plan.plan.md` (8.2); `apps/docs/docs/concepts/retro-hunts.md` (KEV exposure) | `ci.yml :: api tests` (`test_kev_exposure.py` — a CVE routes to the exposure path and never to a telemetry sweep, asserted in both directions so no type can claim both; a value without CVE shape is refused rather than queried, because a scanner's `cve_id` column returns zero rows for an advisory title and that zero would read as "not exposed") | GATED | Exposure is drawn from `asset_vulnerabilities` rows. Matching the catalogue's vendor and product strings against an asset's OS field is deliberately not done: it would produce a plausible answer built on string similarity, and a task an analyst has to disprove is worse than no task |
+| A tenant with no vulnerability data is told exposure could not be checked, not that they are unaffected | `apps/docs/docs/concepts/retro-hunts.md` (KEV exposure) | `ci.yml :: api tests` (`test_kev_exposure.py` — `checked` and `exposed_asset_count` are separate fields and `exposed` requires both, so a caller reading the dataclass cannot render an unscanned tenant as clean; the wording is asserted against the module constant rather than a copy written in the test) | GATED | The case-task body additionally states the count is a floor rather than a total, because assets with no scan coverage cannot appear in it. Not gated: whether an analyst reads that sentence |
## Summary
@@ -237,7 +239,7 @@ Statuses: `GATED` (a CI job fails when the claim stops being true) · `PARTIAL`
## Summary
-- GATED: 214
+- GATED: 216
- PARTIAL: 8
- NO GATE: 0 (**every claim is backed by a failing test.** The last NO GATE — the weekly benchmark scoreboard running live against `main` — closed in Phase E1: `scripts/check_scoreboard.py` ties the published scoreboard to a deterministic per-PR live-agent MITRE-accuracy run, while the funded weekly `wet-eval.yml` appends the LLM-tier rows. The remaining PARTIAL rows are honest, named deferrals — each states the specific gap and what closes it — not unproven claims.)
diff --git a/services/api/app/services/retro_hunt/kev_exposure.py b/services/api/app/services/retro_hunt/kev_exposure.py
new file mode 100644
index 000000000..f93ea9341
--- /dev/null
+++ b/services/api/app/services/retro_hunt/kev_exposure.py
@@ -0,0 +1,374 @@
+"""When CISA says a vulnerability is being exploited, check whether we have it.
+
+Gap-closure Phase 8.2.
+
+The CISA Known Exploited Vulnerabilities catalogue is the one threat-intel
+feed in this deployment that needs no API key and ships on by default, and it
+flows through the same pipeline as every other indicator: an entry becomes a
+``NEW_IOC`` event with ``type: "vulnerability"``. Until Phase 8.1 nothing
+consumed those events at all. This module is what happens to the ones that
+carry a CVE.
+
+Why a CVE takes a different path from every other indicator
+-----------------------------------------------------------
+
+A hash, an address or a domain is something that appears in event telemetry,
+so the question "have we seen this" is a question for the event lake. A CVE
+is not. It never appears in a process-creation event or a firewall log, and
+sweeping the lake for the string "CVE-2024-3400" would return zero on every
+tenant forever while looking exactly like a sweep that worked. The question a
+CVE asks is "do we run the affected thing", and the only surfaces that can
+answer it are the asset inventory and whatever vulnerability scanner feeds it.
+
+So ``intel_types.route_feed_type`` routes a vulnerability here instead, and
+records that as a decision rather than as an omission.
+
+What "exposed" means, and what it deliberately does not
+--------------------------------------------------------
+
+Exposed means the tenant's own vulnerability data has an **unremediated**
+finding whose CVE matches. That is a narrow definition on purpose.
+
+It does not mean "an asset runs software CISA named". Matching a KEV entry's
+vendor and product strings against an asset's OS field would produce a
+plausible-looking answer built on string similarity, and a case task that
+claims a host is exposed when it is not is worse than no task: it costs an
+analyst the time to disprove it, and the second one they disprove is the last
+one they read. If a tenant has no vulnerability data, this reports that it
+could not check rather than reporting no exposure, which is the same rule the
+lake sweep follows.
+
+The KEV catalogue is authoritative about exploitation, so a matching finding
+also has ``is_exploited`` set. That is a real change to the tenant's data and
+it is the one write here that is not a case task: a scanner that has not
+caught up on exploitation status is the ordinary case, and CISA is a better
+source for that field than most scanners.
+"""
+
+from __future__ import annotations
+
+import logging
+import re
+import uuid
+from dataclasses import dataclass, field
+from datetime import UTC, datetime
+
+from sqlalchemy import select, update
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.models.asset import Asset, AssetVulnerability
+from app.models.case import Case, CaseTask
+from app.models.retro_hunt import RetroHuntSettings, RetroHuntSighting
+
+logger = logging.getLogger(__name__)
+
+#: The indicator type CVE sightings are recorded under in
+#: ``retro_hunt_sightings``. Not one of the Phase 4 searchable types, because
+#: nothing searches telemetry for it; it shares the table so one indicator
+#: opens one case task per tenant under the same UNIQUE constraint that stops
+#: an IOC opening a second alert.
+CVE_INDICATOR_TYPE = "cve"
+
+#: Assets above this count are summarised rather than listed. A case task is
+#: read by a person, and a task body naming nine hundred hosts is not.
+_MAX_LISTED_ASSETS = 25
+
+_CVE = re.compile(r"^CVE-\d{4}-\d{4,}$", re.IGNORECASE)
+
+#: What a tenant with no vulnerability data is told.
+#:
+#: A module constant rather than an inline string because the last sentence is
+#: the load-bearing one and it has to be assertable. "No unremediated findings"
+#: and "nothing has ever scanned you" both render as zero exposed assets, and
+#: only one of them is reassuring.
+NO_VULN_DATA_REASON = (
+ "This tenant has no vulnerability data in AiSOC, so exposure could NOT be checked. "
+ "Connect a vulnerability scanner or import an asset inventory. "
+ "This is not a statement that the tenant is unaffected."
+)
+
+
+@dataclass
+class ExposureResult:
+ """What the tenant's own data said about one KEV entry."""
+
+ cve_id: str
+ #: False when the tenant has no vulnerability data at all. A caller must
+ #: not read ``exposed_assets == []`` without checking this: "we scanned and
+ #: you are clean" and "nothing has ever scanned you" are different answers
+ #: and only one of them is reassuring.
+ checked: bool
+ exposed_asset_names: list[str] = field(default_factory=list)
+ exposed_asset_count: int = 0
+ findings_updated: int = 0
+ case_id: uuid.UUID | None = None
+ task_id: uuid.UUID | None = None
+ deduplicated: bool = False
+ unavailable_reason: str | None = None
+
+ @property
+ def exposed(self) -> bool:
+ return self.checked and self.exposed_asset_count > 0
+
+
+def is_cve(value: str) -> bool:
+ """Whether a feed's value has the shape of a CVE identifier.
+
+ Checked rather than trusted. A feed publishing ``type: vulnerability``
+ with a free-text advisory title in the value would otherwise produce a
+ query for a string no scanner ever records, and the zero rows would read
+ as "not exposed".
+ """
+ return bool(_CVE.match(str(value or "").strip()))
+
+
+async def _tenant_has_vulnerability_data(db: AsyncSession, tenant_id: uuid.UUID) -> bool:
+ """Whether anything has ever written a vulnerability finding for a tenant.
+
+ This is the difference between a clean answer and no answer, so it is a
+ separate query rather than an inference from an empty result set.
+ """
+ row = await db.execute(select(AssetVulnerability.id).where(AssetVulnerability.tenant_id == tenant_id).limit(1))
+ return row.scalar_one_or_none() is not None
+
+
+async def check_exposure(
+ db: AsyncSession,
+ *,
+ tenant_id: uuid.UUID,
+ cve_id: str,
+ feed_source: str,
+ intel_first_seen_at: datetime | None = None,
+ now: datetime | None = None,
+) -> ExposureResult:
+ """Check one tenant's vulnerability data for one KEV entry.
+
+ Opens at most one case task per (tenant, CVE), enforced by the same
+ ``retro_hunt_sightings`` UNIQUE constraint that keeps an IOC from opening
+ a second alert. The caller owns the transaction.
+ """
+ now = now or datetime.now(UTC)
+ cve = str(cve_id).strip().upper()
+
+ if not is_cve(cve):
+ return ExposureResult(
+ cve_id=cve,
+ checked=False,
+ unavailable_reason=(
+ f"{cve_id!r} does not have the shape of a CVE identifier, so the tenant's vulnerability data was NOT checked."
+ ),
+ )
+
+ if not await _tenant_has_vulnerability_data(db, tenant_id):
+ return ExposureResult(
+ cve_id=cve,
+ checked=False,
+ unavailable_reason=NO_VULN_DATA_REASON,
+ )
+
+ rows = (
+ await db.execute(
+ select(AssetVulnerability, Asset)
+ .join(Asset, Asset.id == AssetVulnerability.asset_id)
+ .where(
+ AssetVulnerability.tenant_id == tenant_id,
+ Asset.tenant_id == tenant_id,
+ AssetVulnerability.cve_id.ilike(cve),
+ AssetVulnerability.remediated_at.is_(None),
+ )
+ )
+ ).all()
+
+ if not rows:
+ return ExposureResult(cve_id=cve, checked=True)
+
+ # Order by asset criticality so a truncated list shows the ones that
+ # matter. `critical` first, then high, then the rest.
+ order = {"critical": 0, "high": 1, "medium": 2, "low": 3}
+ assets = sorted({(a.name, a.criticality) for _v, a in rows}, key=lambda pair: (order.get(str(pair[1]).lower(), 9), pair[0]))
+ names = [name for name, _crit in assets]
+
+ # CISA is authoritative on exploitation, and most scanners lag it. This is
+ # the one write here that is not a case task.
+ updated = await db.execute(
+ update(AssetVulnerability)
+ .where(
+ AssetVulnerability.tenant_id == tenant_id,
+ AssetVulnerability.cve_id.ilike(cve),
+ AssetVulnerability.remediated_at.is_(None),
+ AssetVulnerability.is_exploited.is_(False),
+ )
+ .values(is_exploited=True)
+ )
+
+ existing = (
+ await db.execute(
+ select(RetroHuntSighting).where(
+ RetroHuntSighting.tenant_id == tenant_id,
+ RetroHuntSighting.indicator_type == CVE_INDICATOR_TYPE,
+ RetroHuntSighting.indicator_value == cve,
+ )
+ )
+ ).scalar_one_or_none()
+
+ if existing is not None:
+ existing.last_matched_at = now
+ existing.sightings = len(assets)
+ existing.times_seen += 1
+ existing.updated_at = now
+ return ExposureResult(
+ cve_id=cve,
+ checked=True,
+ exposed_asset_names=names[:_MAX_LISTED_ASSETS],
+ exposed_asset_count=len(assets),
+ findings_updated=int(updated.rowcount or 0),
+ deduplicated=True,
+ unavailable_reason=None,
+ )
+
+ case, task = await _open_case_task(
+ db,
+ tenant_id=tenant_id,
+ cve=cve,
+ feed_source=feed_source,
+ assets=assets,
+ intel_first_seen_at=intel_first_seen_at,
+ now=now,
+ )
+
+ db.add(
+ RetroHuntSighting(
+ tenant_id=tenant_id,
+ indicator_type=CVE_INDICATOR_TYPE,
+ indicator_value=cve,
+ feed_source=feed_source,
+ intel_first_seen_at=intel_first_seen_at,
+ first_matched_at=now,
+ last_matched_at=now,
+ sightings=len(assets),
+ matched_surfaces=[{"surface": "vulnerability_inventory", "assets": len(assets)}],
+ times_seen=1,
+ created_at=now,
+ updated_at=now,
+ )
+ )
+ logger.info("retro_hunt.kev_exposure tenant=%s assets=%d", tenant_id, len(assets))
+
+ return ExposureResult(
+ cve_id=cve,
+ checked=True,
+ exposed_asset_names=names[:_MAX_LISTED_ASSETS],
+ exposed_asset_count=len(assets),
+ findings_updated=int(updated.rowcount or 0),
+ case_id=case.id,
+ task_id=task.id,
+ )
+
+
+async def _open_case_task(
+ db: AsyncSession,
+ *,
+ tenant_id: uuid.UUID,
+ cve: str,
+ feed_source: str,
+ assets: list[tuple[str, str]],
+ intel_first_seen_at: datetime | None,
+ now: datetime,
+) -> tuple[Case, CaseTask]:
+ """Open the case and the one task this CVE gets for this tenant."""
+ listed = assets[:_MAX_LISTED_ASSETS]
+ overflow = len(assets) - len(listed)
+
+ lines = [
+ f"{feed_source} published {cve} as a known exploited vulnerability.",
+ "",
+ "Provenance",
+ f" Feed: {feed_source}",
+ (
+ f" Added to the catalogue: {intel_first_seen_at.date().isoformat()}"
+ if intel_first_seen_at
+ else " Added to the catalogue: not published by this feed"
+ ),
+ "",
+ f"Exposure in your own data: {len(assets)} unremediated finding(s)",
+ ]
+ lines += [f" {name} ({criticality})" for name, criticality in listed]
+ if overflow:
+ lines.append(f" and {overflow} more asset(s), listed on the vulnerability page")
+ lines += [
+ "",
+ "This is drawn from your vulnerability findings, not inferred from software names.",
+ "Assets with no scan coverage will not appear here, so this is a floor rather than a total.",
+ ]
+
+ case = Case(
+ tenant_id=tenant_id,
+ case_number=f"KEV-{cve.replace('CVE-', '')}-{int(now.timestamp())}",
+ title=f"Known exploited vulnerability {cve} affects {len(assets)} asset(s)",
+ description="\n".join(lines),
+ case_type="vulnerability_exposure",
+ priority="high" if any(c in {"critical", "high"} for _n, c in assets) else "medium",
+ severity="high",
+ status="open",
+ tags=["kev", f"cve:{cve}", f"feed:{feed_source}"],
+ )
+ db.add(case)
+ await db.flush()
+
+ task = CaseTask(
+ case_id=case.id,
+ tenant_id=tenant_id,
+ title=f"Patch or mitigate {cve} on {len(assets)} asset(s)",
+ description=(
+ f"CISA lists {cve} as actively exploited. Your vulnerability data shows "
+ f"{len(assets)} unremediated finding(s). Patch, mitigate, or record why the "
+ f"affected assets are not reachable."
+ ),
+ status="pending",
+ )
+ db.add(task)
+ await db.flush()
+ return case, task
+
+
+async def handle_kev_entry(
+ db: AsyncSession,
+ *,
+ settings_row: RetroHuntSettings,
+ cve_id: str,
+ feed_source: str,
+ intel_first_seen_at: datetime | None = None,
+ now: datetime | None = None,
+) -> ExposureResult:
+ """Entry point from the intel consumer for one tenant and one CVE.
+
+ Gated on the same per-tenant opt-in as a lake sweep. Deliberately not on
+ the sweep *budget*: an exposure check is two indexed queries against the
+ tenant's own Postgres rows rather than a warehouse scan, so charging it
+ against a budget sized for the latter would starve the cheap check to
+ protect against the expensive one.
+ """
+ if not settings_row.enabled:
+ return ExposureResult(
+ cve_id=str(cve_id),
+ checked=False,
+ unavailable_reason="This tenant has not opted in to retro-hunts.",
+ )
+ return await check_exposure(
+ db,
+ tenant_id=settings_row.tenant_id,
+ cve_id=cve_id,
+ feed_source=feed_source,
+ intel_first_seen_at=intel_first_seen_at,
+ now=now,
+ )
+
+
+__all__ = [
+ "CVE_INDICATOR_TYPE",
+ "ExposureResult",
+ "NO_VULN_DATA_REASON",
+ "check_exposure",
+ "handle_kev_entry",
+ "is_cve",
+]
diff --git a/services/api/app/workers/retro_hunt_consumer.py b/services/api/app/workers/retro_hunt_consumer.py
index d0c8208a2..1416bce3a 100644
--- a/services/api/app/workers/retro_hunt_consumer.py
+++ b/services/api/app/workers/retro_hunt_consumer.py
@@ -49,6 +49,7 @@ from app.core.config import settings
from app.db.cross_tenant import assert_cross_tenant_session
from app.db.database import AsyncSessionLocal
from app.services.retro_hunt.intel_types import route_feed_type
+from app.services.retro_hunt.kev_exposure import handle_kev_entry
from app.services.retro_hunt.service import (
IntelIndicator,
TenantSweepReport,
@@ -116,6 +117,87 @@ def _first_seen(data: dict[str, Any], envelope_ts: Any) -> datetime | None:
return None
+def cve_from_event(payload: dict[str, Any]) -> tuple[str, str, datetime | None] | None:
+ """Pull a CVE out of a ``NEW_IOC`` that routes to the exposure check.
+
+ Returns ``(cve_id, feed_source, first_seen)`` or ``None``. A CVE takes a
+ different path from every other indicator because it never appears in
+ event telemetry: sweeping the lake for one would return zero on every
+ tenant while looking exactly like a sweep that worked. See
+ :mod:`app.services.retro_hunt.kev_exposure`.
+ """
+ data = payload.get("data") or {}
+ value = str(data.get("value") or "").strip()
+ if not value:
+ return None
+ if not route_feed_type(str(data.get("type") or "")).is_vulnerability:
+ return None
+ source = str(payload.get("source") or data.get("source") or "unknown feed")
+ return value, source, _first_seen(data, payload.get("timestamp"))
+
+
+async def handle_kev_indicator(cve_id: str, feed_source: str, first_seen: datetime | None) -> list[TenantSweepReport]:
+ """Fan one KEV entry out to every tenant that opted in.
+
+ Same session and tenant-binding shape as :func:`handle_indicator`: the
+ read of who opted in is genuinely cross-tenant, and every write after it
+ rebinds the tenant so the case and task inserts are RLS-enforced.
+ """
+ reports: list[TenantSweepReport] = []
+ async with AsyncSessionLocal() as db:
+ await assert_cross_tenant_session(db, "retro-hunt KEV fan-out")
+ tenants = await opted_in_tenants(db)
+ if not tenants:
+ return reports
+
+ for settings_row in tenants:
+ try:
+ await db.execute(
+ text("SELECT set_config('app.current_tenant_id', :t, true)"),
+ {"t": str(settings_row.tenant_id)},
+ )
+ result = await handle_kev_entry(
+ db,
+ settings_row=settings_row,
+ cve_id=cve_id,
+ feed_source=feed_source,
+ intel_first_seen_at=first_seen,
+ )
+ except Exception as exc: # noqa: BLE001 - one tenant must not stop the rest
+ logger.exception(
+ "retro_hunt.kev_check_failed tenant=%s err=%s",
+ settings_row.tenant_id,
+ type(exc).__name__,
+ )
+ reports.append(
+ TenantSweepReport(
+ tenant_id=settings_row.tenant_id,
+ outcome="failed",
+ detail=type(exc).__name__,
+ )
+ )
+ continue
+
+ reports.append(
+ TenantSweepReport(
+ tenant_id=settings_row.tenant_id,
+ outcome="swept" if result.checked else "skipped_unsupported",
+ matched=result.exposed,
+ deduplicated=result.deduplicated,
+ detail=(
+ result.unavailable_reason
+ or (
+ f"{result.exposed_asset_count} exposed asset(s); case task opened."
+ if result.exposed
+ else "No unremediated findings for this CVE."
+ )
+ ),
+ )
+ )
+ await db.commit()
+ return reports
+
+
def indicator_from_event(payload: dict[str, Any]) -> IntelIndicator | None:
"""Turn one decoded ``NEW_IOC`` into a typed indicator, or refuse it.
@@ -247,6 +329,14 @@ async def run_forever() -> None:
# subscription: one bad message is not a reason to stop
# consuming a topic that is otherwise fine.
continue
+ kev = cve_from_event(payload)
+ if kev is not None:
+ try:
+ await handle_kev_indicator(*kev)
+ except Exception as exc: # noqa: BLE001 - keep consuming
+ logger.exception("retro_hunt.kev_fan_out_failed err=%s", type(exc).__name__)
+ continue
+
indicator = indicator_from_event(payload)
if indicator is None:
routing = route_feed_type(str((payload.get("data") or {}).get("type") or ""))
diff --git a/services/api/tests/test_kev_exposure.py b/services/api/tests/test_kev_exposure.py
new file mode 100644
index 000000000..b6f1b9201
--- /dev/null
+++ b/services/api/tests/test_kev_exposure.py
@@ -0,0 +1,274 @@
+"""Gap-closure Phase 8.2: KEV exposure.
+
+Three groups, each covering a way this reports something it does not know.
+
+**A CVE never reaches the telemetry sweep.** Routing is the whole design
+decision: a CVE does not appear in event data, so sweeping the lake for one
+returns zero on every tenant forever while looking exactly like a sweep that
+worked.
+
+**No vulnerability data is not "no exposure".** The single most damaging thing
+this surface could do is tell a tenant with no scanner that they are
+unaffected. ``checked`` and ``exposed_asset_count`` are separate for that
+reason and ``exposed`` requires both.
+
+**Exposure is drawn from findings, not inferred from software names.** Matching
+a KEV entry's vendor and product strings against an asset's OS field would
+produce a plausible answer built on string similarity, and a case task an
+analyst has to disprove is worse than no task at all.
+"""
+
+from __future__ import annotations
+
+import uuid
+from dataclasses import dataclass
+from datetime import datetime
+
+import pytest
+from app.services.retro_hunt.intel_types import route_feed_type
+from app.services.retro_hunt.kev_exposure import (
+ CVE_INDICATOR_TYPE,
+ NO_VULN_DATA_REASON,
+ ExposureResult,
+ is_cve,
+)
+from app.services.retro_hunt.sweep import DEFAULT_LOOKBACK_DAYS
+
+TENANT = uuid.UUID("11111111-1111-1111-1111-111111111111")
+
+
+# ------------------------------------------------------------------ routing
+
+
[email protected]("feed_type", ["vulnerability", "Vulnerability", "cve", "CVE"])
+def test_a_cve_routes_to_exposure_not_to_a_telemetry_sweep(feed_type: str) -> None:
+ """The decision this phase turns on.
+
+ A lake sweep for "CVE-2024-3400" matches nothing on every deployment and
+ would report a confident zero. Routing it elsewhere is what stops that.
+ """
+ routing = route_feed_type(feed_type)
+ assert routing.is_vulnerability is True
+ assert routing.indicator_type is None
+ assert routing.reason and "not against event telemetry" in routing.reason
+
+
+def test_the_indicator_path_and_the_cve_path_cannot_both_claim_one_event() -> None:
+ """A type routes to exactly one of the two paths, never both."""
+ for feed_type in ("vulnerability", "ipv4-addr", "sha256", "domain-name", "url"):
+ routing = route_feed_type(feed_type)
+ assert not (routing.is_vulnerability and routing.indicator_type), feed_type
+
+
+# ------------------------------------------------------------- value shapes
+
+
[email protected]("value", ["CVE-2024-3400", "cve-2021-44228", "CVE-2019-0708", "CVE-2024-123456"])
+def test_real_cve_identifiers_are_accepted(value: str) -> None:
+ assert is_cve(value) is True
+
+
+ "value",
+ [
+ "",
+ " ",
+ "Apache HTTP Server path traversal",
+ "CVE-24-3400",
+ "CVE-2024-",
+ "2024-3400",
+ "GHSA-xxxx-yyyy-zzzz",
+ ],
+)
+def test_a_value_that_is_not_a_cve_is_refused_rather_than_queried(value: str) -> None:
+ """A feed publishing an advisory title under ``type: vulnerability``.
+
+ Querying a scanner's ``cve_id`` column for free text returns zero rows,
+ and that zero would be reported as "not exposed".
+ """
+ assert is_cve(value) is False
+
+
+def test_a_refused_value_reports_a_reason_rather_than_a_clean_result() -> None:
+ result = ExposureResult(
+ cve_id="not a cve",
+ checked=False,
+ unavailable_reason="does not have the shape of a CVE identifier",
+ )
+ assert result.exposed is False
+ assert result.checked is False
+ assert result.unavailable_reason
+
+
+# ------------------------------------------- the distinction that matters
+
+
+def test_no_vulnerability_data_is_not_no_exposure() -> None:
+ """The most damaging thing this surface could report.
+
+ ``exposed`` requires ``checked``, so a tenant with no scanner cannot be
+ rendered as clean by any caller that reads the dataclass rather than the
+ count.
+ """
+ result = ExposureResult(cve_id="CVE-2024-3400", checked=False, unavailable_reason="unused")
+ assert result.exposed is False
+ assert result.exposed_asset_count == 0
+
+ # Asserted against the wording the module actually emits rather than a
+ # copy written here. A test that restates a message passes while the
+ # message says something else, which is the shape of defect this phase
+ # keeps finding.
+ assert "could NOT be checked" in NO_VULN_DATA_REASON
+ assert "not a statement that the tenant is unaffected" in NO_VULN_DATA_REASON
+
+
+def test_a_genuine_clean_answer_is_distinguishable_from_an_unchecked_one() -> None:
+ """Scanned and clean, versus never scanned. Different facts."""
+ clean = ExposureResult(cve_id="CVE-2024-3400", checked=True)
+ unchecked = ExposureResult(cve_id="CVE-2024-3400", checked=False, unavailable_reason="no scanner")
+ assert clean.exposed is False
+ assert unchecked.exposed is False
+ # Both are "not exposed" by the boolean, and only one of them is
+ # reassuring. The fields that carry the difference must not be equal.
+ assert clean.checked != unchecked.checked
+ assert (clean.unavailable_reason is None) != (unchecked.unavailable_reason is None)
+
+
+def test_an_exposed_result_carries_the_assets_it_is_asserting_about() -> None:
+ result = ExposureResult(
+ cve_id="CVE-2024-3400",
+ checked=True,
+ exposed_asset_names=["fw-edge-01", "fw-edge-02"],
+ exposed_asset_count=2,
+ findings_updated=2,
+ )
+ assert result.exposed is True
+ assert result.exposed_asset_names
+
+
+def test_the_asset_list_is_bounded_while_the_count_is_not() -> None:
+ """A task body naming nine hundred hosts is not read by anyone.
+
+ The count stays exact so the task never understates the problem; only the
+ enumeration is capped.
+ """
+ from app.services.retro_hunt.kev_exposure import _MAX_LISTED_ASSETS
+
+ result = ExposureResult(
+ cve_id="CVE-2024-3400",
+ checked=True,
+ exposed_asset_names=[f"host-{i}" for i in range(_MAX_LISTED_ASSETS)],
+ exposed_asset_count=900,
+ )
+ assert len(result.exposed_asset_names) == _MAX_LISTED_ASSETS
+ assert result.exposed_asset_count == 900
+
+
+# ------------------------------------------------------------------- dedup
+
+
+def test_cve_sightings_share_the_ioc_dedup_table_under_their_own_type() -> None:
+ """One CVE opens one case task per tenant, by the same mechanism.
+
+ The type is not one of the Phase 4 searchable types, because nothing
+ searches telemetry for it. It shares the table so the UNIQUE constraint
+ that stops an IOC alerting twice also stops a KEV entry opening a task a
+ day, which it would otherwise do: the catalogue republishes its whole
+ contents on every fetch.
+ """
+ assert CVE_INDICATOR_TYPE == "cve"
+ from app.services.retro_hunt.ioc_fields import SWEEPABLE_TYPES
+
+ assert CVE_INDICATOR_TYPE not in SWEEPABLE_TYPES
+
+
+def test_the_exposure_check_is_not_charged_against_the_sweep_budget() -> None:
+ """Two indexed Postgres queries, not a warehouse scan.
+
+ Recorded as a test because the alternative is a plausible mistake: reusing
+ the budget would let a busy day of IOCs starve the cheap check that has
+ the clearest action attached to it.
+ """
+ import inspect
+
+ from app.services.retro_hunt import kev_exposure
+
+ source = inspect.getsource(kev_exposure)
+ assert "_consume_budget" not in source
+ assert "max_sweeps_per_hour" not in source
+ # The opt-in still applies; only the budget does not.
+ assert "settings_row.enabled" in source
+
+
+def test_the_lake_sweep_default_window_is_untouched_by_this_phase() -> None:
+ """A guard against the exposure path quietly changing sweep behaviour."""
+ assert DEFAULT_LOOKBACK_DAYS == 30
+
+
+# --------------------------------------------------- the consumer's routing
+
+
+@dataclass
+class _Payload:
+ data: dict
+ source: str = "cisa-kev"
+
+ def as_dict(self) -> dict:
+ return {"event_type": "NEW_IOC", "source": self.source, "timestamp": "2026-09-20T00:00:00Z", "data": self.data}
+
+
+def test_the_consumer_sends_a_kev_entry_to_exposure_and_not_to_a_sweep() -> None:
+ from app.workers.retro_hunt_consumer import cve_from_event, indicator_from_event
+
+ payload = _Payload({"type": "vulnerability", "value": "CVE-2024-3400", "date_added": "2026-09-18"}).as_dict()
+ kev = cve_from_event(payload)
+ assert kev is not None
+ cve, source, first_seen = kev
+ assert cve == "CVE-2024-3400"
+ assert source == "cisa-kev"
+ assert isinstance(first_seen, datetime)
+ assert first_seen.tzinfo is not None
+ # And the indicator path refuses the same event, so the two cannot both
+ # act on it.
+ assert indicator_from_event(payload) is None
+
+
+def test_the_consumer_sends_an_ordinary_indicator_to_the_sweep_and_not_to_exposure() -> None:
+ from app.workers.retro_hunt_consumer import cve_from_event, indicator_from_event
+
+ payload = _Payload({"type": "ipv4-addr", "value": "203.0.113.9"}, source="otx").as_dict()
+ assert cve_from_event(payload) is None
+ indicator = indicator_from_event(payload)
+ assert indicator is not None
+ assert indicator.indicator_type == "ip"
+
+
+def test_a_feed_that_published_no_date_does_not_get_one_invented() -> None:
+ """``first_seen`` falls back to the envelope, never to ``now``."""
+ from app.workers.retro_hunt_consumer import cve_from_event
+
+ payload = {"event_type": "NEW_IOC", "source": "cisa-kev", "data": {"type": "vulnerability", "value": "CVE-2024-3400"}}
+ kev = cve_from_event(payload)
+ assert kev is not None
+ assert kev[2] is None
+
+
+def test_an_empty_value_is_not_routed_anywhere(monkeypatch: pytest.MonkeyPatch) -> None:
+ from app.workers.retro_hunt_consumer import cve_from_event
+
+ assert cve_from_event(_Payload({"type": "vulnerability", "value": ""}).as_dict()) is None
+
+
+def test_the_case_task_body_states_that_unscanned_assets_are_absent() -> None:
+ """A floor, not a total, and it says so.
+
+ An analyst reading "3 assets affected" against an estate with 40% scanner
+ coverage will act on the 3 and conclude the rest are fine.
+ """
+ import inspect
+
+ from app.services.retro_hunt import kev_exposure
+
+ source = inspect.getsource(kev_exposure._open_case_task)
+ assert "floor rather than a total" in source
+ assert "not inferred from software names" in source