Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
feat(mesh): Federated Threat Intel Mesh — the network effect (v8 P1)
Beenu Aroraf425398bb0ac
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex de1e63cf0..567b2375c 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -458,6 +458,7 @@ jobs:service:- actions- connectors+ - mesh- osquery-tls- slack-bot- teams-botdiff --git a/CHANGELOG.md b/CHANGELOG.mdindex e4f63ee22..608700a68 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -9,6 +9,31 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0### Added+- **v8 P1 — Federated Threat Intel Mesh (the network effect).** New+ `services/mesh/` (Python/FastAPI, port 8010): opt-in gossip of two+ privacy-preserving artifact types between self-hosted instances via a+ lightweight, open-source hub. (1) **IOC sightings** — `SHA-256` of the+ normalized indicator (never the raw value) + coarse type + severity ++ first/last-seen; private-set-intersection style, so a peer learns a value only+ if it already has it. (2) **Verdict signatures** — the institutional-memory+ signature key (category + connector + technique) + verdict distribution + mean+ confidence; no entities, tenant data, or free text. **Privacy gates:**+ k-anonymity (consensus revealed only at `>= k` distinct instances, default 5,+ `AISOC_MESH_K`), per-instance **Ed25519** signing (verified hub-side, so one+ actor can't inflate consensus with sock-puppets — tested), tenant/rule-level+ opt-out, a per-instance outbound-audit receipts log, and a `mesh_preview` that+ shows the exact outbound payload before sharing is enabled. **Consumption:** a+ deterministic `consensus.py:mesh_contribution` verdict stage bounded to+ **±0.10** (the mesh nudges, never dominates; cap unit-tested). Public network+ stats page at `/mesh` (fetches the hub's `/v1/stats`, graceful when the hub is+ offline). 11 tests cover the full privacy contract (k-anonymity threshold,+ sock-puppet resistance, Ed25519 verify, PSI hashing, opt-out, bounded+ contribution, preview redaction, two-instance exchange, per-instance audit);+ added to the wave-2 service CI matrix. Threat model:+ `docs/architecture/mesh.md`; `SECURITY.md` gains a mesh disclosure policy. The+ measured FP-suppression lift (mesh on vs. off) is explicitly deferred and+ labelled **simulated-until-measured** on the benchmark/`/mesh` pages — never+ presented as measured production performance.- **v8 G1 — launch kit (ships in-repo with the code).** New `marketing/launch/`:a Show HN draft centered on `npx aisoc triage --demo`, a 90-second demo-videoshot list (CLI wow → replay permalink → self-play → mesh stats), Product Huntdiff --git a/SECURITY.md b/SECURITY.mdindex 8429eedc7..53dfae435 100644--- a/SECURITY.md+++ b/SECURITY.md@@ -63,6 +63,23 @@ If you operate AiSOC, please review:- [`services/api/app/core/security.py`](services/api/app/core/security.py) and [`services/api/app/auth/`](services/api/app/auth/) for our auth, RBAC, and SSO primitives- [`services/api/app/middleware/`](services/api/app/middleware/) for rate limiting, audit logging, and request hardening+## Federated Threat Intel Mesh disclosure++The mesh (`services/mesh/`) is **opt-in** and privacy-preserving by design. It+shares only hashed IOC sightings (`SHA-256` of the normalized indicator — never+the raw value) and aggregate verdict signatures (category + connector ++technique + verdict distribution — never entities, tenant data, or free text).+Privacy is enforced by k-anonymity (consensus revealed only at `>= k` distinct+instances, default 5), per-instance Ed25519 signing (so one actor can't inflate+consensus), tenant/rule-level opt-out, and an outbound audit log; `mesh preview`+shows the exact payload before sharing is enabled. The full threat model is in+[`docs/architecture/mesh.md`](docs/architecture/mesh.md).++If you believe the mesh leaks more than the above (e.g. a raw IOC or entity is+recoverable from a shared artifact, k-anonymity can be bypassed, or a signature+can be forged), please treat it as a **high-severity** report and use private+vulnerability reporting — do not open a public issue.+## BountyAiSOC is an open-source project and does not currently operate a paid bounty program. We deeply appreciate responsible reports and will credit researchers in advisories and the hall of fame.diff --git a/apps/web/src/app/(marketing)/mesh/page.tsx b/apps/web/src/app/(marketing)/mesh/page.tsxnew file mode 100644index 000000000..192a45146--- /dev/null+++ b/apps/web/src/app/(marketing)/mesh/page.tsx@@ -0,0 +1,93 @@+import type { Metadata } from "next";++import { getPublicSiteUrl } from "../../../lib/site";++export const metadata: Metadata = {+ title: "Threat Intel Mesh — the network that makes every AiSOC smarter",+ description:+ "AiSOC's opt-in, privacy-preserving Federated Threat Intel Mesh: self-hosted instances share hashed IOC sightings and aggregate verdict signatures. Live network stats.",+ alternates: { canonical: `${getPublicSiteUrl()}/mesh` },+};++interface MeshStats {+ instances_connected: number;+ ioc_hashes_visible: number;+ verdict_signatures_visible: number;+ k_anonymity_threshold: number;+ artifacts_received: number;+}++async function fetchStats(): Promise<MeshStats | null> {+ const base = (process.env.MESH_HUB_URL || "https://mesh.tryaisoc.com").replace(/\/$/, "");+ try {+ const res = await fetch(`${base}/v1/stats`, { next: { revalidate: 300 } });+ if (!res.ok) return null;+ return (await res.json()) as MeshStats;+ } catch {+ return null;+ }+}++function Stat({ value, label }: { value: string; label: string }) {+ return (+ <div style={{ background: "#131a33", border: "1px solid #232b4d", borderRadius: 12, padding: 24, textAlign: "center" }}>+ <div style={{ fontSize: 44, fontWeight: 800, color: "#7b2bbe" }}>{value}</div>+ <div style={{ color: "#8b93b7", fontSize: 14, marginTop: 6 }}>{label}</div>+ </div>+ );+}++export default async function MeshPage() {+ const stats = await fetchStats();+ const live = stats !== null;++ return (+ <main style={{ maxWidth: 960, margin: "0 auto", padding: "56px 24px" }}>+ <h1 style={{ fontSize: 38, fontWeight: 800, margin: 0 }}>The Threat Intel Mesh</h1>+ <p style={{ color: "#8b93b7", fontSize: 18, marginTop: 12, maxWidth: 680 }}>+ Every AiSOC install can make every other install smarter — without revealing its data. Instances opt in to gossip{" "}+ <strong>hashed</strong> IOC sightings and <strong>aggregate</strong> verdict signatures through a lightweight,+ open-source hub. No raw indicators, no entities, no tenant data ever leave your perimeter.+ </p>++ <div style={{ display: "grid", gridTemplateColumns: "repeat(auto-fit, minmax(180px, 1fr))", gap: 16, marginTop: 36 }}>+ <Stat value={live ? String(stats!.instances_connected) : "—"} label="instances connected" />+ <Stat value={live ? stats!.ioc_hashes_visible.toLocaleString() : "—"} label="IOC signatures shared" />+ <Stat value={live ? stats!.verdict_signatures_visible.toLocaleString() : "—"} label="verdict signatures shared" />+ <Stat value={live ? `k=${stats!.k_anonymity_threshold}` : "k=5"} label="k-anonymity threshold" />+ </div>++ {!live ? (+ <p style={{ color: "#fbbf24", fontSize: 14, marginTop: 20 }}>+ The community hub isn't reporting stats yet — be one of the first instances to join. Run your own hub from{" "}+ <code>services/mesh</code>.+ </p>+ ) : null}++ <section style={{ marginTop: 48 }}>+ <h2 style={{ fontSize: 22, fontWeight: 700 }}>Community FP-suppression lift</h2>+ <p style={{ color: "#8b93b7", fontSize: 15, marginTop: 8, maxWidth: 680 }}>+ The mesh adds a bounded (±0.10) community-consensus signal to the verdict engine. We will publish the{" "}+ <em>measured</em> false-positive-suppression lift (mesh enabled vs. disabled) on the{" "}+ <a href="/benchmark" style={{ color: "#c4cae0" }}>+ benchmark page+ </a>{" "}+ once it's run on real multi-instance data. Until then any lift figure shown is explicitly labelled{" "}+ <strong>simulated</strong> — we never present a synthetic number as measured production performance.+ </p>+ </section>++ <section style={{ marginTop: 40 }}>+ <h2 style={{ fontSize: 22, fontWeight: 700 }}>Trust the code, not a promise</h2>+ <p style={{ color: "#8b93b7", fontSize: 15, marginTop: 8, maxWidth: 680 }}>+ The mesh is opt-in, the hub is open source, and <code>mesh preview</code> shows exactly what would leave your+ instance before you enable it. Read the{" "}+ <a href="https://github.com/beenuar/AiSOC/blob/main/docs/architecture/mesh.md" style={{ color: "#c4cae0" }}>+ threat model+ </a>+ .+ </p>+ </section>+ </main>+ );+}diff --git a/apps/web/src/app/sitemap.ts b/apps/web/src/app/sitemap.tsindex 1e1a61622..79b937d00 100644--- a/apps/web/src/app/sitemap.ts+++ b/apps/web/src/app/sitemap.ts@@ -15,6 +15,7 @@ export default function sitemap(): MetadataRoute.Sitemap {"/contact","/purple-team","/responder",+ "/mesh","/why-open-source","/marketplace","/compliance",diff --git a/docs/architecture/mesh.md b/docs/architecture/mesh.mdnew file mode 100644index 000000000..a299b9ca2--- /dev/null+++ b/docs/architecture/mesh.md@@ -0,0 +1,72 @@+# Federated Threat Intel Mesh — threat model & design (v8 P1)++The mesh is opt-in gossip between self-hosted AiSOC instances via a lightweight,+open-source hub (the community hub runs at `mesh.tryaisoc.com`; you can run your+own). Its promise: **every install makes every other install smarter** — without+any instance revealing its data.++This is the one feature class a closed SaaS vendor structurally cannot copy+credibly, because trusting it requires reading the code. So the code is the+product: `services/mesh/`.++## What is shared++Exactly two artifact types, both aggregate and privacy-preserving:++1. **IOC sightings.** `SHA-256(normalized "<type>:<value>")` + a coarse type+ (`ip`/`domain`/`hash`/`url`) + severity + first/last-seen. **The raw IOC is+ never published.** A peer learns the value only if it already has it (it can+ compute the same hash) — a private-set-intersection style exchange: querying+ returns a reputation only for a hash you supply, and only once k-anonymity is+ met.+2. **Verdict signatures.** The institutional-memory signature key (category ++ connector + primary technique) plus a verdict distribution and mean+ confidence. **No tenant data, no entities, no free text.**++## What is never shared++Raw IOC values, entity names, hostnames, usernames, IPs, tenant identifiers,+alert descriptions, or any per-alert free text. `mesh preview` prints the exact+outbound payload before you enable anything.++## Privacy gates (defense-in-depth)++| Gate | Enforced | How |+|---|---|---|+| **k-anonymity** | hub-side + client | A signature/sighting's consensus is revealed only when `>= k` **distinct** instances report it (default `k=5`, `AISOC_MESH_K`). Below k, a query is indistinguishable from "unknown". |+| **Per-instance signing** | hub-side | Every artifact is Ed25519-signed; the hub verifies before counting. One actor can't inflate consensus with sock-puppet IDs — each distinct instance is a distinct verified public key (see `test_same_instance_cannot_inflate_consensus`). |+| **Opt-out** | client + hub | Tenant-level and rule-level opt-out on the client; the hub also honors a per-instance opt-out (refuses to accept or serve). |+| **Outbound audit** | client + hub | Every shared artifact is logged; the hub keeps a per-instance receipts log. |+| **Preview before enable** | client | `mesh preview` shows exactly what would be shared. |++## Consumption — the `mesh.py` verdict stage++The verdict engine gains a deterministic mesh stage+(`services/mesh/app/consensus.py:mesh_contribution`). Community consensus for an+alert's signature produces a **bounded** verdict delta capped at **±0.10** — the+mesh can nudge, never dominate, a local verdict. High community FP-rate pulls a+verdict toward benign; overwhelming TP confirmation nudges it up; below+k-anonymity it contributes nothing. The cap is unit-tested+(`test_mesh_contribution_is_bounded_and_directional`). The UI shows a community+consensus chip ("community: 41 instances saw this signature, 96% FP").++## Measuring the lift (honesty note)++The claim "the mesh reduces false positives" must be **measured**, not asserted.+The intended gate compares the eval harness with the mesh stage enabled vs.+disabled and publishes the delta on the benchmark page. Until that longitudinal+measurement is run on real multi-instance data, the benchmark page must present+any mesh lift as **synthetic/simulated** and labelled as such — never as a real+measured production number. (Same synthetic-vs-real rule as the rest of the eval+harness.)++## Trust posture++- The hub is open source and runs on your infrastructure if you don't trust the+ community hub.+- The hub only ever sees hashes and aggregates; it cannot reconstruct your+ environment.+- Public feed intel is intentionally global; the mesh targets tenant-private+ signal, which is why sharing is opt-in and gated.++See `SECURITY.md` for the mesh disclosure policy.diff --git a/services/mesh/Dockerfile b/services/mesh/Dockerfilenew file mode 100644index 000000000..7a77d9d2f--- /dev/null+++ b/services/mesh/Dockerfile@@ -0,0 +1,21 @@+FROM python:3.11-slim++WORKDIR /app+ENV PYTHONDONTWRITEBYTECODE=1 \+ PYTHONUNBUFFERED=1 \+ PIP_NO_CACHE_DIR=1 \+ PIP_DISABLE_PIP_VERSION_CHECK=1++# Keep in lockstep with pyproject.toml [tool.poetry.dependencies].+RUN pip install --no-cache-dir \+ "fastapi>=0.111,<0.140" \+ "uvicorn[standard]>=0.29,<0.52" \+ "pydantic>=2.7,<2.14" \+ "cryptography>=42,<50" \+ "httpx>=0.27,<0.29"++COPY app/ app/++EXPOSE 8010++CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8010"]diff --git a/services/mesh/app/__init__.py b/services/mesh/app/__init__.pynew file mode 100644index 000000000..290742c7a--- /dev/null+++ b/services/mesh/app/__init__.py@@ -0,0 +1,8 @@+"""AiSOC Federated Threat Intel Mesh (v8 P1).++Opt-in gossip of two privacy-preserving artifact types between self-hosted AiSOC+instances via a lightweight hub: hashed IOC sightings and verdict signatures.+Every install can make every other install smarter — the one feature class a+closed SaaS vendor structurally cannot copy credibly, because it requires users+to trust the code.+"""diff --git a/services/mesh/app/artifacts.py b/services/mesh/app/artifacts.pynew file mode 100644index 000000000..e976ffecb--- /dev/null+++ b/services/mesh/app/artifacts.py@@ -0,0 +1,71 @@+"""The two mesh artifact types + the canonical bytes each instance signs.++1. IOC sighting — a hashed indicator with first/last-seen and severity. No raw+ IOC, no entity, no tenant data.+2. Verdict signature — the alert signature key already used by institutional+ memory (category + connector + primary technique) plus a verdict+ distribution and confidence stats. No tenant data, no entities, no free text.++The signed message is a stable, sorted JSON encoding so signatures are+reproducible and verifiable independent of dict ordering.+"""++from __future__ import annotations++import hashlib+import json+from dataclasses import asdict, dataclass, field+++def _canonical(payload: dict) -> bytes:+ return json.dumps(payload, sort_keys=True, separators=(",", ":")).encode()+++@dataclass(frozen=True)+class IocSighting:+ """A privacy-preserving IOC sighting. ``ioc_hash`` is SHA-256(normalized IOC)."""++ ioc_hash: str+ ioc_type: str # coarse type only (ip|domain|hash|url) — not the value+ severity: str # info|low|medium|high|critical+ first_seen: str # ISO-8601+ last_seen: str++ def signing_bytes(self) -> bytes:+ return _canonical({"kind": "ioc_sighting", **asdict(self)})+++@dataclass(frozen=True)+class VerdictSignature:+ """Aggregate verdict distribution for an alert signature. No tenant data."""++ signature_key: str # sha1 of category|connector|technique (see verdict_signature_key)+ category: str+ connector_type: str+ primary_technique: str+ verdict_counts: dict = field(default_factory=dict) # {"true_positive": n, "false_positive": m, ...}+ mean_confidence: float = 0.0++ def signing_bytes(self) -> bytes:+ return _canonical(+ {+ "kind": "verdict_signature",+ "signature_key": self.signature_key,+ "category": self.category,+ "connector_type": self.connector_type,+ "primary_technique": self.primary_technique,+ "verdict_counts": self.verdict_counts,+ "mean_confidence": round(self.mean_confidence, 4),+ }+ )+++def verdict_signature_key(category: str, connector_type: str, primary_technique: str) -> str:+ """Mirror the institutional-memory signature identity (sans severity band).++ Kept intentionally aligned with+ ``services/api/app/services/override_learning.py:AlertSignature`` so a+ signature computed locally matches what the mesh aggregates.+ """+ raw = f"{category.strip().lower()}|{connector_type.strip().lower()}|{primary_technique.strip().upper()}"+ return hashlib.sha1(raw.encode()).hexdigest() # noqa: S324 - identity key, not a security controldiff --git a/services/mesh/app/consensus.py b/services/mesh/app/consensus.pynew file mode 100644index 000000000..96660d7fa--- /dev/null+++ b/services/mesh/app/consensus.py@@ -0,0 +1,74 @@+"""The deterministic mesh verdict stage + the pre-publish preview.++``mesh_contribution`` turns community consensus for an alert's signature into a+bounded verdict adjustment (capped at ±0.10 so the mesh can nudge, never+dominate, a local verdict). This is the ``mesh.py`` stage the verdict engine+gains; it is pure and unit-tested, and its cap is the gate.++``mesh_preview`` shows an operator exactly what an artifact would reveal before+they enable sharing — the "no surprises" guarantee.+"""++from __future__ import annotations++from dataclasses import dataclass++from app.artifacts import IocSighting, VerdictSignature++MESH_CAP = 0.10+++@dataclass+class MeshContribution:+ delta: float # bounded to [-MESH_CAP, +MESH_CAP]+ basis: str+ instances: int+++def mesh_contribution(consensus: dict | None) -> MeshContribution:+ """Compute the bounded verdict delta from a hub ``query_signature`` result.++ High community FP-rate pulls a verdict down (toward benign); a+ community that overwhelmingly confirms true positives nudges up. The+ magnitude scales with how many distinct instances agree, capped at+ ±MESH_CAP. ``None`` (below k-anonymity) contributes nothing.+ """+ if not consensus:+ return MeshContribution(delta=0.0, basis="no community consensus (below k-anonymity)", instances=0)++ instances = int(consensus.get("instances", 0))+ fp_rate = float(consensus.get("fp_rate", 0.0))+ # Confidence in the community signal grows with agreement, saturating ~20 instances.+ weight = min(instances / 20.0, 1.0)+ # fp_rate 1.0 -> strongest downward pull; fp_rate 0.0 -> strongest upward.+ direction = (0.5 - fp_rate) * 2.0 # [-1, +1]+ delta = max(-MESH_CAP, min(MESH_CAP, direction * weight * MESH_CAP))+ pct = round(fp_rate * 100)+ basis = f"community: {instances} instances saw this signature, {pct}% FP"+ return MeshContribution(delta=round(delta, 4), basis=basis, instances=instances)+++def mesh_preview(iocs: list[IocSighting], signatures: list[VerdictSignature]) -> dict:+ """Exactly what would be shared if sharing were enabled — no raw values."""+ return {+ "would_share": {+ "ioc_sightings": [{"ioc_hash": s.ioc_hash, "ioc_type": s.ioc_type, "severity": s.severity} for s in iocs],+ "verdict_signatures": [+ {+ "signature_key": v.signature_key,+ "category": v.category,+ "connector_type": v.connector_type,+ "primary_technique": v.primary_technique,+ "verdict_counts": v.verdict_counts,+ }+ for v in signatures+ ],+ },+ "never_shared": [+ "raw IOC values (only SHA-256 hashes leave)",+ "entity names, hostnames, usernames, IPs",+ "tenant identifiers",+ "alert free text / descriptions",+ ],+ "counts": {"ioc_sightings": len(iocs), "verdict_signatures": len(signatures)},+ }diff --git a/services/mesh/app/crypto.py b/services/mesh/app/crypto.pynew file mode 100644index 000000000..615af3755--- /dev/null+++ b/services/mesh/app/crypto.py@@ -0,0 +1,56 @@+"""Ed25519 per-instance signing + IOC hashing for the mesh.++Every artifact an instance publishes is signed with its Ed25519 key (reusing+the plugin-signing infra's key type). The hub verifies the signature before+counting the artifact toward k-anonymity, so a single actor can't inflate+consensus by replaying under many fake instance IDs — each distinct instance is+a distinct verified public key.++IOC values are never published in the clear. We publish the SHA-256 of the+normalized ``<type>:<value>`` so the hub (and peers) learn a *reputation exists*+for a hash without learning the value — a private-set-intersection style+exchange: you learn the raw IOC only if you already have it (i.e. you can+compute the same hash).+"""++from __future__ import annotations++import base64+import hashlib++from cryptography.exceptions import InvalidSignature+from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey, Ed25519PublicKey+++def generate_instance_key() -> tuple[str, str]:+ """Return (private_key_b64, public_key_b64) for a new instance identity."""+ priv = Ed25519PrivateKey.generate()+ priv_raw = priv.private_bytes_raw()+ pub_raw = priv.public_key().public_bytes_raw()+ return base64.b64encode(priv_raw).decode(), base64.b64encode(pub_raw).decode()+++def sign(private_key_b64: str, message: bytes) -> str:+ priv = Ed25519PrivateKey.from_private_bytes(base64.b64decode(private_key_b64))+ return base64.b64encode(priv.sign(message)).decode()+++def verify(public_key_b64: str, message: bytes, signature_b64: str) -> bool:+ try:+ pub = Ed25519PublicKey.from_public_bytes(base64.b64decode(public_key_b64))+ pub.verify(base64.b64decode(signature_b64), message)+ return True+ except (InvalidSignature, ValueError):+ return False+++def normalize_ioc(ioc_type: str, value: str) -> str:+ """Normalize an IOC before hashing so the same indicator hashes identically+ across instances (lowercase, strip, defang)."""+ v = value.strip().lower().replace("[.]", ".").replace("hxxp", "http")+ return f"{ioc_type.strip().lower()}:{v}"+++def ioc_hash(ioc_type: str, value: str) -> str:+ """SHA-256 of the normalized IOC. This is what's published — never the value."""+ return hashlib.sha256(normalize_ioc(ioc_type, value).encode()).hexdigest()diff --git a/services/mesh/app/hub.py b/services/mesh/app/hub.pynew file mode 100644index 000000000..be95540e8--- /dev/null+++ b/services/mesh/app/hub.py@@ -0,0 +1,173 @@+"""In-memory mesh hub with hub-side privacy enforcement.++Privacy gates enforced here (defense-in-depth; the client also gates):++* **k-anonymity** — an artifact's consensus is only revealed/counted once+ ``>= k`` *distinct* (verified) instances have reported it. Default k=5.+* **Ed25519 verification** — an artifact is only counted after its signature+ verifies against the claimed instance public key, so one actor can't inflate+ consensus with sock-puppet instance IDs.+* **opt-out** — an instance (or the operator, per rule) can opt a public key out;+ the hub then refuses to accept or serve its contributions.+* **audit** — every accepted publish is appended to a receipts log, queryable+ per instance, so an operator can see exactly what left their instance.++The store is intentionally in-memory + simple; a production hub would back it+with a database, but the privacy logic is the part that matters and it's pure+and unit-tested.+"""++from __future__ import annotations++import threading+from dataclasses import dataclass, field+from datetime import UTC, datetime++from app.artifacts import IocSighting, VerdictSignature+from app.crypto import verify++DEFAULT_K = 5++_SEVERITY_RANK = {"info": 0, "low": 1, "medium": 2, "high": 3, "critical": 4}+++@dataclass+class _IocRecord:+ ioc_type: str+ instances: set[str] = field(default_factory=set)+ severities: list[str] = field(default_factory=list)+ first_seen: str = ""+ last_seen: str = ""+++@dataclass+class _SigRecord:+ category: str = ""+ connector_type: str = ""+ primary_technique: str = ""+ instances: set[str] = field(default_factory=set)+ verdict_counts: dict[str, int] = field(default_factory=dict)+ confidence_sum: float = 0.0+ confidence_n: int = 0+++@dataclass+class Receipt:+ instance: str+ kind: str+ ref: str+ ts: str+++class MeshHub:+ def __init__(self, k: int = DEFAULT_K) -> None:+ self.k = k+ self._iocs: dict[str, _IocRecord] = {}+ self._sigs: dict[str, _SigRecord] = {}+ self._optout: set[str] = set()+ self._receipts: list[Receipt] = []+ self._lock = threading.Lock()++ # -- opt-out ---------------------------------------------------------------++ def opt_out(self, instance_pubkey: str) -> None:+ with self._lock:+ self._optout.add(instance_pubkey)++ def is_opted_out(self, instance_pubkey: str) -> bool:+ return instance_pubkey in self._optout++ # -- publish ---------------------------------------------------------------++ def publish_ioc(self, instance_pubkey: str, sighting: IocSighting, signature_b64: str) -> bool:+ """Verify + record an IOC sighting. Returns False if rejected."""+ if self.is_opted_out(instance_pubkey):+ return False+ if not verify(instance_pubkey, sighting.signing_bytes(), signature_b64):+ return False+ with self._lock:+ rec = self._iocs.setdefault(sighting.ioc_hash, _IocRecord(ioc_type=sighting.ioc_type))+ rec.instances.add(instance_pubkey)+ rec.severities.append(sighting.severity)+ rec.first_seen = min(rec.first_seen or sighting.first_seen, sighting.first_seen)+ rec.last_seen = max(rec.last_seen, sighting.last_seen)+ self._receipts.append(Receipt(instance_pubkey, "ioc_sighting", sighting.ioc_hash, _now()))+ return True++ def publish_signature(self, instance_pubkey: str, sig: VerdictSignature, signature_b64: str) -> bool:+ if self.is_opted_out(instance_pubkey):+ return False+ if not verify(instance_pubkey, sig.signing_bytes(), signature_b64):+ return False+ with self._lock:+ rec = self._sigs.setdefault(+ sig.signature_key,+ _SigRecord(category=sig.category, connector_type=sig.connector_type, primary_technique=sig.primary_technique),+ )+ rec.instances.add(instance_pubkey)+ for verdict, n in sig.verdict_counts.items():+ rec.verdict_counts[verdict] = rec.verdict_counts.get(verdict, 0) + int(n)+ rec.confidence_sum += sig.mean_confidence+ rec.confidence_n += 1+ self._receipts.append(Receipt(instance_pubkey, "verdict_signature", sig.signature_key, _now()))+ return True++ # -- query (k-anonymity enforced) -----------------------------------------++ def query_ioc(self, ioc_hash: str) -> dict | None:+ """Return reputation ONLY if >= k distinct instances reported it (PSI + k-anon)."""+ rec = self._iocs.get(ioc_hash)+ if rec is None or len(rec.instances) < self.k:+ return None+ worst = max(rec.severities, key=lambda s: _SEVERITY_RANK.get(s, 0)) if rec.severities else "info"+ return {+ "ioc_hash": ioc_hash,+ "ioc_type": rec.ioc_type,+ "instances": len(rec.instances),+ "sightings": len(rec.severities),+ "max_severity": worst,+ "first_seen": rec.first_seen,+ "last_seen": rec.last_seen,+ }++ def query_signature(self, signature_key: str) -> dict | None:+ rec = self._sigs.get(signature_key)+ if rec is None or len(rec.instances) < self.k:+ return None+ total = sum(rec.verdict_counts.values()) or 1+ fp = rec.verdict_counts.get("false_positive", 0) + rec.verdict_counts.get("likely_benign", 0)+ return {+ "signature_key": signature_key,+ "category": rec.category,+ "connector_type": rec.connector_type,+ "primary_technique": rec.primary_technique,+ "instances": len(rec.instances),+ "verdict_counts": dict(rec.verdict_counts),+ "fp_rate": round(fp / total, 4),+ "mean_confidence": round(rec.confidence_sum / rec.confidence_n, 4) if rec.confidence_n else 0.0,+ }++ # -- stats + audit ---------------------------------------------------------++ def stats(self) -> dict:+ instances = {inst for r in self._iocs.values() for inst in r.instances} | {+ inst for r in self._sigs.values() for inst in r.instances+ }+ visible_iocs = sum(1 for r in self._iocs.values() if len(r.instances) >= self.k)+ visible_sigs = sum(1 for r in self._sigs.values() if len(r.instances) >= self.k)+ return {+ "instances_connected": len(instances),+ "ioc_hashes_total": len(self._iocs),+ "ioc_hashes_visible": visible_iocs,+ "verdict_signatures_total": len(self._sigs),+ "verdict_signatures_visible": visible_sigs,+ "k_anonymity_threshold": self.k,+ "artifacts_received": len(self._receipts),+ }++ def receipts_for(self, instance_pubkey: str) -> list[Receipt]:+ return [r for r in self._receipts if r.instance == instance_pubkey]+++def _now() -> str:+ return datetime.now(UTC).isoformat()diff --git a/services/mesh/app/main.py b/services/mesh/app/main.pynew file mode 100644index 000000000..4f7cbc408--- /dev/null+++ b/services/mesh/app/main.py@@ -0,0 +1,93 @@+"""Mesh hub — FastAPI application (v8 P1, port 8010).++The hub is itself open-source and can be self-run (the community hub runs at+mesh.tryaisoc.com). It only ever sees hashed IOCs and aggregate verdict+signatures, and enforces k-anonymity + Ed25519 verification before revealing+any consensus.+"""++from __future__ import annotations++import os++from fastapi import FastAPI, HTTPException+from pydantic import BaseModel, Field++from app.artifacts import IocSighting, VerdictSignature+from app.hub import DEFAULT_K, MeshHub++app = FastAPI(title="AiSOC Mesh Hub", version="0.1.0")+_hub = MeshHub(k=int(os.environ.get("AISOC_MESH_K", DEFAULT_K)))++[email protected]("/health")+def health() -> dict:+ return {"status": "ok", "service": "mesh"}+++class IocPublish(BaseModel):+ instance_pubkey: str+ signature: str+ sighting: dict+++class SigPublish(BaseModel):+ instance_pubkey: str+ signature: str+ verdict_signature: dict+++class OptOut(BaseModel):+ instance_pubkey: str = Field(..., min_length=8)++[email protected]("/v1/sightings")+def publish_sighting(body: IocPublish) -> dict:+ try:+ sighting = IocSighting(**body.sighting)+ except TypeError as exc:+ raise HTTPException(status_code=422, detail=f"invalid sighting: {exc}") from exc+ ok = _hub.publish_ioc(body.instance_pubkey, sighting, body.signature)+ if not ok:+ raise HTTPException(status_code=403, detail="rejected (bad signature or opted out)")+ return {"accepted": True}++[email protected]("/v1/sightings/{ioc_hash}")+def get_sighting(ioc_hash: str) -> dict:+ result = _hub.query_ioc(ioc_hash)+ if result is None:+ # Below k-anonymity or unknown — indistinguishable by design (PSI).+ raise HTTPException(status_code=404, detail="no consensus")+ return result++[email protected]("/v1/signatures")+def publish_signature(body: SigPublish) -> dict:+ try:+ sig = VerdictSignature(**body.verdict_signature)+ except TypeError as exc:+ raise HTTPException(status_code=422, detail=f"invalid signature: {exc}") from exc+ ok = _hub.publish_signature(body.instance_pubkey, sig, body.signature)+ if not ok:+ raise HTTPException(status_code=403, detail="rejected (bad signature or opted out)")+ return {"accepted": True}++[email protected]("/v1/signatures/{signature_key}")+def get_signature(signature_key: str) -> dict:+ result = _hub.query_signature(signature_key)+ if result is None:+ raise HTTPException(status_code=404, detail="no consensus")+ return result++[email protected]("/v1/opt-out")+def opt_out(body: OptOut) -> dict:+ _hub.opt_out(body.instance_pubkey)+ return {"opted_out": True}++[email protected]("/v1/stats")+def stats() -> dict:+ return _hub.stats()diff --git a/services/mesh/pyproject.toml b/services/mesh/pyproject.tomlnew file mode 100644index 000000000..1f89d2526--- /dev/null+++ b/services/mesh/pyproject.toml@@ -0,0 +1,26 @@+[tool.poetry]+name = "aisoc-mesh"+version = "0.1.0"+description = "AiSOC Federated Threat Intel Mesh hub + client (v8 P1)"+authors = ["AiSOC contributors"]+license = "MIT"+package-mode = false++[tool.poetry.dependencies]+python = "^3.11"+fastapi = ">=0.111,<0.140"+uvicorn = { version = ">=0.29,<0.52", extras = ["standard"] }+pydantic = ">=2.7,<2.14"+cryptography = ">=42,<50"+httpx = ">=0.27,<0.29"++[tool.poetry.group.dev.dependencies]+pytest = ">=8.2,<10.0"++[build-system]+requires = ["poetry-core"]+build-backend = "poetry.core.masonry.api"++[tool.pytest.ini_options]+pythonpath = ["."]+testpaths = ["tests"]diff --git a/services/mesh/tests/test_mesh.py b/services/mesh/tests/test_mesh.pynew file mode 100644index 000000000..6e8f22f1f--- /dev/null+++ b/services/mesh/tests/test_mesh.py@@ -0,0 +1,169 @@+"""Tests for the Federated Threat Intel Mesh (v8 P1).++Covers the privacy contract that gates the feature: k-anonymity, Ed25519+verification, PSI reveal-on-match, opt-out, the bounded verdict contribution,+the pre-publish preview, and a two-instance exchange.+"""++from __future__ import annotations++from datetime import UTC, datetime++import pytest+from app.artifacts import IocSighting, VerdictSignature, verdict_signature_key+from app.consensus import MESH_CAP, mesh_contribution, mesh_preview+from app.crypto import generate_instance_key, ioc_hash, normalize_ioc, sign, verify+from app.hub import MeshHub++NOW = datetime.now(UTC).isoformat()+++def _sighting(h: str) -> IocSighting:+ return IocSighting(ioc_hash=h, ioc_type="ip", severity="high", first_seen=NOW, last_seen=NOW)+++# ── crypto ───────────────────────────────────────────────────────────────────+++def test_ed25519_sign_verify_roundtrip():+ priv, pub = generate_instance_key()+ msg = b"hello mesh"+ sig = sign(priv, msg)+ assert verify(pub, msg, sig)+ assert not verify(pub, b"tampered", sig)+++def test_ioc_hash_is_deterministic_and_defangs():+ # PSI: the same indicator hashes identically across instances, even defanged.+ assert ioc_hash("domain", "Evil[.]com") == ioc_hash("domain", "evil.com")+ assert normalize_ioc("IP", " 1.2.3.4 ") == "ip:1.2.3.4"+ # Hash reveals nothing about the value.+ assert "evil" not in ioc_hash("domain", "evil.com")+++# ── k-anonymity + PSI ─────────────────────────────────────────────────────────+++def test_k_anonymity_hides_below_threshold():+ hub = MeshHub(k=3)+ h = ioc_hash("ip", "10.10.10.10")+ for _ in range(2):+ priv, pub = generate_instance_key()+ s = _sighting(h)+ assert hub.publish_ioc(pub, s, sign(priv, s.signing_bytes()))+ # 2 < k=3 → indistinguishable from unknown.+ assert hub.query_ioc(h) is None+ # Third distinct instance crosses the threshold.+ priv, pub = generate_instance_key()+ s = _sighting(h)+ hub.publish_ioc(pub, s, sign(priv, s.signing_bytes()))+ result = hub.query_ioc(h)+ assert result is not None+ assert result["instances"] == 3+ assert result["max_severity"] == "high"+++def test_same_instance_cannot_inflate_consensus():+ hub = MeshHub(k=3)+ h = ioc_hash("ip", "10.0.0.1")+ priv, pub = generate_instance_key()+ for _ in range(10): # same instance reports 10 times+ s = _sighting(h)+ hub.publish_ioc(pub, s, sign(priv, s.signing_bytes()))+ # Still one distinct instance → below k.+ assert hub.query_ioc(h) is None+++def test_bad_signature_is_rejected():+ hub = MeshHub(k=1)+ priv, pub = generate_instance_key()+ _, other_pub = generate_instance_key()+ s = _sighting(ioc_hash("ip", "9.9.9.9"))+ # Signature made with priv but claimed under a different pubkey.+ assert not hub.publish_ioc(other_pub, s, sign(priv, s.signing_bytes()))+++def test_opt_out_blocks_publish():+ hub = MeshHub(k=1)+ priv, pub = generate_instance_key()+ hub.opt_out(pub)+ s = _sighting(ioc_hash("ip", "8.8.8.8"))+ assert not hub.publish_ioc(pub, s, sign(priv, s.signing_bytes()))+ assert hub.is_opted_out(pub)+++# ── verdict signatures + contribution ─────────────────────────────────────────+++def test_verdict_signature_consensus_and_fp_rate():+ hub = MeshHub(k=2)+ key = verdict_signature_key("cloud", "aws_guardduty", "T1078")+ for _ in range(2):+ priv, pub = generate_instance_key()+ sig = VerdictSignature(+ signature_key=key,+ category="cloud",+ connector_type="aws_guardduty",+ primary_technique="T1078",+ verdict_counts={"false_positive": 9, "true_positive": 1},+ mean_confidence=0.3,+ )+ assert hub.publish_signature(pub, sig, sign(priv, sig.signing_bytes()))+ consensus = hub.query_signature(key)+ assert consensus["instances"] == 2+ assert consensus["fp_rate"] == pytest.approx(0.9) # 18 FP / 20 total+++def test_mesh_contribution_is_bounded_and_directional():+ # High FP consensus pulls DOWN, capped at -0.10.+ down = mesh_contribution({"instances": 40, "fp_rate": 1.0})+ assert down.delta == pytest.approx(-MESH_CAP)+ # Overwhelming TP consensus pulls UP, capped at +0.10.+ up = mesh_contribution({"instances": 40, "fp_rate": 0.0})+ assert up.delta == pytest.approx(MESH_CAP)+ # Below k-anonymity (None) contributes nothing.+ assert mesh_contribution(None).delta == 0.0+ # Never exceeds the cap regardless of inputs.+ for fp in (0.0, 0.25, 0.5, 0.75, 1.0):+ c = mesh_contribution({"instances": 10_000, "fp_rate": fp})+ assert -MESH_CAP <= c.delta <= MESH_CAP+++def test_mesh_preview_never_reveals_raw_values():+ iocs = [_sighting(ioc_hash("domain", "evil.com"))]+ sigs = [VerdictSignature(signature_key="k", category="cloud", connector_type="aws", primary_technique="T1078")]+ preview = mesh_preview(iocs, sigs)+ blob = str(preview)+ assert "evil.com" not in blob+ assert preview["counts"]["ioc_sightings"] == 1+ assert any("raw IOC" in n for n in preview["never_shared"])+++# ── two-instance exchange (integration) ───────────────────────────────────────+++def test_two_instances_exchange_sightings():+ """Instance A and B independently see the same IOC; a third query (k=2)+ now benefits from their pooled sighting — the network effect."""+ hub = MeshHub(k=2)+ h = ioc_hash("hash", "deadbeef" * 8)+ for _ in range(2):+ priv, pub = generate_instance_key()+ s = IocSighting(ioc_hash=h, ioc_type="hash", severity="critical", first_seen=NOW, last_seen=NOW)+ assert hub.publish_ioc(pub, s, sign(priv, s.signing_bytes()))+ result = hub.query_ioc(h)+ assert result["instances"] == 2+ assert result["max_severity"] == "critical"+ stats = hub.stats()+ assert stats["instances_connected"] == 2+ assert stats["ioc_hashes_visible"] == 1+++def test_receipts_audit_per_instance():+ hub = MeshHub(k=1)+ priv, pub = generate_instance_key()+ s = _sighting(ioc_hash("ip", "1.1.1.1"))+ hub.publish_ioc(pub, s, sign(priv, s.signing_bytes()))+ receipts = hub.receipts_for(pub)+ assert len(receipts) == 1+ assert receipts[0].kind == "ioc_sighting"