Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
docs(connectors): add endpoint decision-matrix page (Stage 2 #9) (#93)
Beenu Arorab88702f00ca8
diff --git a/apps/docs/docs/connectors/endpoint-decision-matrix.md b/apps/docs/docs/connectors/endpoint-decision-matrix.mdnew file mode 100644index 000000000..c50d324f6--- /dev/null+++ b/apps/docs/docs/connectors/endpoint-decision-matrix.md@@ -0,0 +1,198 @@+---+sidebar_position: 25+title: Endpoint connector decision matrix+description: Pick the right endpoint data source for AiSOC — EDR/XDR alerts, osquery fleet managers, the AiSOC osquery extension, or universal capture.+---++# Endpoint connector decision matrix++AiSOC ships with **four distinct paths** for getting endpoint telemetry into the+pipeline. They are not mutually exclusive — most production deployments use two+or three at once — but each has a different sweet spot. This page is a quick+decision aid for operators who are wiring up endpoint coverage for the first+time and aren't sure which connector to enable.++If you already know which class of source you want, skip to the per-connector+walkthrough linked from the [Connectors catalog](/docs/connectors).++## TL;DR — pick by what you already have++| If you already run… | Start with | Why |+|---|---|---|+| A commercial EDR / XDR (CrowdStrike, SentinelOne, Defender, Cortex, Carbon Black) | The matching **EDR connector** | You get vendor-curated alerts immediately; no agent to deploy |+| osquery + a fleet manager (osctrl, FleetDM) | **[osctrl](/docs/connectors/osctrl)** or **[FleetDM](/docs/connectors/fleetdm)** | Reuse your fleet for distributed-query results _and_ live-query response actions |+| Bare-metal Linux with `auditd`, no fleet manager | **Universal capture** (syslog/HEC) today; **aisoc-host-agent** when it ships | Lightweight path with no extra control plane |+| Nothing yet, want to start small | The **[AiSOC osquery extension](/docs/connectors/osquery-extensions)** layered on osquery | Surfaces AiSOC alerts and persistence baselines _inside_ the osquery shell |++## The four paths in detail++```mermaid+flowchart LR+ subgraph Path1["1. EDR / XDR alerts"]+ EDR[CrowdStrike, SentinelOne,<br/>Defender XDR, Cortex,<br/>Carbon Black, Trellix, Trend]+ end+ subgraph Path2["2. osquery fleet manager"]+ OSCTRL[osctrl]+ FLEET[FleetDM]+ end+ subgraph Path3["3. AiSOC osquery extension"]+ EXT[aisoc-extension binary<br/>5 virtual tables]+ end+ subgraph Path4["4. Universal capture"]+ UC[Webhook / syslog /<br/>email relay / HEC]+ end++ Path1 --> Ingest[ingest service]+ Path2 --> Ingest+ Path3 --> Ingest+ Path4 --> Ingest+ Ingest --> Detect[Detection +<br/>triage agents]+```++### 1. Commercial EDR / XDR connectors++These are the highest signal-to-noise sources because the vendor has already+correlated raw telemetry into alerts. AiSOC pulls the alert stream, normalises+severity into the four-tier ladder (`info`/`low`/`medium`/`high`), and feeds it+straight into detection routing.++**Available today:** [CrowdStrike Falcon](https://github.com/beenuar/AiSOC/blob/main/services/connectors/app/connectors/crowdstrike.py),+SentinelOne, [Microsoft Defender XDR](/docs/connectors/azure-defender),+Palo Alto Cortex XDR, Cortex XSIAM, VMware Carbon Black, Trellix Helix,+Trend Vision One.++**Pick this when** you already pay for an EDR and the security team treats it+as the source of truth for "is this host compromised".++**Trade-offs:**++- ✅ Zero new agents to deploy.+- ✅ Vendor-curated severity, MITRE mapping, and process-tree context.+- ❌ You don't see the raw events the vendor decided weren't alerts.+- ❌ Live-query response actions are vendor-specific (most playbook actions+ today target the osquery backends, not EDR APIs).++### 2. osquery fleet managers — osctrl and FleetDM++Both [osctrl](/docs/connectors/osctrl) and [FleetDM](/docs/connectors/fleetdm)+expose distributed-query results from a fleet of osquery agents. AiSOC polls+the manager's REST API, turns each result row into a normalised endpoint event,+and synthesises severity from the originating osquery table (persistence /+execution tables → `high`, FIM tables → `medium`, inventory → `info`).++The same connectors back the `osquery_live_query` playbook step, which lets a+playbook run on-demand allowlisted queries against affected hosts and feed the+rows back into the case as triage evidence.++**Pick this when** you want raw endpoint telemetry _and_ live response actions+from the same control plane, without paying for an EDR. This is the path most+self-hosted AiSOC deployments take.++**FleetDM vs osctrl** — see the [matching matrix in the FleetDM page](/docs/connectors/fleetdm#when-to-choose-fleetdm-vs-osctrl).+Short version: pick FleetDM if your team already runs it; pick osctrl if you+want the smaller infra footprint (PostgreSQL only, no MySQL or Redis).++**Trade-offs:**++- ✅ Open-source, self-hosted, no per-endpoint licensing.+- ✅ Same connector powers ingestion _and_ live-query playbook actions.+- ❌ You have to operate the fleet manager (osctrl or Fleet) yourself.+- ❌ Raw rows are noisy by default — detection authoring matters more than+ with EDR alert ingestion.++### 3. AiSOC osquery extension++The [aisoc-extension binary](/docs/connectors/osquery-extensions) goes the+other direction: instead of pulling osquery results into AiSOC, it surfaces+**AiSOC operational data inside the osquery shell** as five virtual tables+(`aisoc_pending_actions`, `aisoc_alert_cache`, `aisoc_attck_persistence`,+`aisoc_kernel_modules_verified`, `aisoc_browser_extensions`).++This is not an ingestion connector — it does not move data into AiSOC. It is+the **complement** to paths 2 and 3: once your fleet is sending data in via+osctrl/FleetDM, the extension lets responders query AiSOC's view of the host+from the same osquery shell they already use for IR triage.++**Pick this when** you've already wired an osquery fleet and want responders+to be able to answer "what does AiSOC currently know about this host?" via SQL.++### 4. Universal capture (webhook / syslog / HEC / email relay)++If none of the above fit — for example, you have a Linux box with `auditd`+running and no fleet manager, or a custom in-house EDR — the+[Universal capture](/docs/connectors/universal-capture) connector accepts+webhook POSTs, CEF syslog, Splunk HEC payloads, and email. Anything that can+emit one of those formats can land events in the same OCSF stream every other+connector feeds.++**Pick this when** the source isn't in the catalog and won't be soon, or when+you're prototyping a new agent locally before committing to writing a proper+connector.++**Trade-offs:**++- ✅ Works for literally anything that can POST or send mail.+- ❌ You own the field mapping — there is no schema-driven wizard.+- ❌ Best treated as a stopgap; for permanent integrations, write a proper+ connector ([Hello, connector tutorial](/docs/connectors/hello-connector)).++## Side-by-side comparison++| Dimension | EDR / XDR | osctrl / FleetDM | osquery extension | Universal capture |+|---|---|---|---|---|+| **What flows in** | Vendor alerts | osquery rows + (FleetDM) host posture | _Nothing_ — surfaces AiSOC data in the shell | Whatever you POST |+| **Severity source** | Vendor → 4-tier ladder | Synthesised from osquery table | n/a | You control it |+| **Live-query response** | Vendor-specific (limited today) | ✅ via `osquery_live_query` playbook step | n/a (read-only on host) | n/a |+| **Self-hosted-friendly** | Depends on vendor | ✅ Fully | ✅ Fully | ✅ Fully |+| **Per-endpoint cost** | EDR licence | Free (osquery + fleet manager) | Free | Free |+| **Setup complexity** | Low — paste API token | Medium — operate the fleet manager | Medium — deploy binary + systemd unit | High — design your own field mapping |+| **MITRE coverage** | Vendor-curated | Detection-rule-driven (you author) | n/a | Detection-rule-driven (you author) |+| **Platform support** | Vendor-defined | Linux / macOS / Windows (osquery) | Linux / macOS / Windows (osquery) | Anything |++## Common combinations++Most production AiSOC deployments end up running **at least two** of these in+parallel:++- **EDR + osquery fleet** — EDR for the curated alert stream, osquery fleet+ for the raw telemetry your detection rules need to spot what the EDR misses+ (custom persistence, lateral movement via SaaS tokens, etc.).+- **osquery fleet + osquery extension** — operators get both the+ detection-rule firehose and the in-shell AiSOC view for IR triage.+- **EDR + universal capture** — EDR is the primary source; universal capture+ catches the long tail (legacy Windows servers, IoT/OT gateways, custom+ internal tools).++There is no penalty for enabling multiple sources for the same host — events+land in the same OCSF stream and the dedupe key on detection rules handles+collapsing duplicate alerts.++## Roadmap signals++The endpoint connector surface is one of the most actively expanded areas of+AiSOC. Two work items are in the public roadmap and will land here when they+ship:++- **`aisoc-host-agent` daemon** — a lightweight Go daemon (mirroring the+ existing osquery deployment pattern) for hosts where running osquery + a+ fleet manager is overkill. Will give bare-metal Linux deployments a+ first-class path without forcing the operator to choose universal capture.+- **`auditd` ingest profile** — a pre-built ingest template plus+ `detections/endpoint/auditd-*.yaml` rules that consume `auditd` events via+ the host-agent in `file_tail` mode.++Both are tracked in the [community roadmap](https://github.com/beenuar/AiSOC/issues)+and will get dedicated walkthrough pages on the same model as+[osctrl](/docs/connectors/osctrl) and [FleetDM](/docs/connectors/fleetdm) when+they merge. Until then, the four paths above cover every endpoint shape AiSOC+can ingest today.++## Related++- [Connectors overview](/docs/connectors) — the full 50-connector catalog.+- [Hello, connector tutorial](/docs/connectors/hello-connector) — write your+ own if none of these fit.+- [Detection coverage](/docs/detections/coverage) — which endpoint detections+ fire on which sources.+- [Capabilities](/docs/concepts/capabilities) — the agent action surface,+ including which capabilities require an osquery fleet behind them.diff --git a/apps/docs/docs/detections/coverage.md b/apps/docs/docs/detections/coverage.mdindex f5912e24a..198dd75fb 100644--- a/apps/docs/docs/detections/coverage.md+++ b/apps/docs/docs/detections/coverage.md@@ -11,12 +11,12 @@ sidebar_position: 2# Detection Coverage-Generated: `2026-05-11T03:45:32Z`+Generated: `2026-05-12T20:45:55Z`## Headline numbers- **Curated v1.0 detections**: `416` (target: ≥ 300)-- **Total rules considered**: `1047` (quality floor: 0.55)+- **Total rules considered**: `1048` (quality floor: 0.55)- **Unique MITRE techniques covered**: `117`## Coverage by buyer familydiff --git a/apps/docs/docs/detections/hello-hunt.md b/apps/docs/docs/detections/hello-hunt.mdnew file mode 100644index 000000000..d9e25d3ea--- /dev/null+++ b/apps/docs/docs/detections/hello-hunt.md@@ -0,0 +1,317 @@+---+sidebar_position: 3+title: Hello, hunt — write your first AiSOC detection+description: A walkthrough that ships a runnable AiSOC detection rule end to end. Author the YAML, pin it with positive and negative fixtures, run it against the rule engine, and surface it in the marketplace — no vendor account required.+---++# Hello, hunt++This tutorial walks you end-to-end through the work of adding a new detection rule to AiSOC. By the end you will have:++- A `community-tier` Sigma-style YAML rule under `detections/community/<category>/`.+- A **positive** fixture under `detections/fixtures/positive/` that the rule must match.+- A **negative** fixture under `detections/fixtures/negative/` that the rule must *not* match.+- Local proof that `scripts/validate_detections.py` is happy with all three files.+- Local proof that the rule fires on the positive fixture and stays silent on the negative one when run through the real `services/api/app/services/rule_engine.py`.+- A marketplace entry that surfaces the rule under the `Community` badge after `pnpm marketplace:sync`.++The example rule targets [AWS root-account console logins](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#lock-away-credentials), because:++- The CloudTrail `ConsoleLogin` shape is well documented and easy to reason about.+- "Root logged in successfully" is a real signal — most production AWS environments lock the root user away after initial setup, so any successful console login under `userIdentity.type: Root` is worth a human eyeballing.+- The rule needs only three field comparisons, which keeps the tutorial focused on the *contract* rather than on detection cleverness.++When you're ready to write a real rule, copy the example, swap the field names, and tighten the condition. The shape stays the same.++## Where the example lives++The reference detection plus its two fixtures live at:++```text+detections/community/cloud/hello-hunt-aws-root-login.yaml+detections/fixtures/positive/hello-hunt-aws-root-login.json+detections/fixtures/negative/hello-hunt-aws-root-login.json+```++Three files, one rule. That's the whole contract.++## Detection tiers — pick the right shelf first++AiSOC sorts detections into three tiers, and each tier has a different bar:++| Tier | Path | `id` prefix | Authoring style | Bar |+| ----------- | ------------------------------------------------- | ----------------------------- | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |+| `native` | `detections/{cloud,identity,endpoint,...}/` | `det-` | Spec-generated YAML | Strict. Must come from `scripts/detection_specs/*.py`, must have positive + negative fixtures, must round-trip through `scripts/generate_detections.py`, must replay cleanly. CI will fail if any of those drift. |+| `imported` | `detections/{sigma,car,splunk,chronicle}-imports/`| `<source>-...` | Importer-generated | Provenance required (`source`, `source_id`, `source_commit`, `license`, `imported_at`, `imported_by`, `upstream_path`). Fixtures optional. Rules that don't run on our engine land in `_quarantine/` automatically. |+| `community` | `detections/community/<category>/` | `community-<github-handle>-*` | Hand-authored YAML | Permissive. No spec required, fixtures encouraged but optional. The validator emits a `WARN` instead of a `FAIL` when there's no spec to round-trip against. |++This tutorial sticks to the **community** tier on purpose — it's the only tier you can use without modifying `scripts/detection_specs/*.py`. Native rules belong to the curated v1.0 set; the path from "I have a rule idea" to "it ships in `det-*` form" goes through a maintainer review, and that's a different doc.++If you eventually want your rule promoted to native, see [Graduating from `community/` to `native/`](#graduating-from-community-to-native) at the bottom.++## Step 1 — Pick a stable `id`++```yaml+id: community-aisoc-hello-hunt-aws-root-login+```++Three rules to internalise:++1. **Lowercase, kebab-case, no spaces.** The `id` is a wire identifier — it ends up in `match_summary` payloads, in `events.matched_rule_ids`, in marketplace JSON, and in case timelines. Treat it like a primary key.+2. **Prefix with `community-<your-handle>-`.** This is enforced socially, not technically — the validator doesn't reject other shapes — but `marketplace/curated.json` and the in-app filters group by prefix, and reviewers will ask you to rename if you skip it.+3. **Never change it after merge.** Renaming a rule breaks every alert row in production that references it. If the rule needs a behavioural overhaul, give the new version a new `id` and deprecate the old one.++The example uses `aisoc` as the handle because the AiSOC project itself is the author. Real contributions use your GitHub username (`community-jdoe-aws-root-login`) or your org name (`community-acme-aws-root-login`).++## Step 2 — Fill in the metadata block++```yaml+name: "[Hello Hunt] AWS Root Account Console Login"+description: |+ Tutorial detection from apps/docs/docs/detections/hello-hunt.md.+ Fires when AWS CloudTrail records a successful console login under the+ `Root` user identity. Real environments should keep the AWS root account+ unused except for break-glass — any successful console login is worth a+ human eyeballing.+version: 1.0.0+severity: high+category: cloud+tags:+ - mitre.attack.T1078.004+ - tlp.white+ - tutorial+log_source:+ product: aws+ service: cloudtrail+```++A few non-obvious things:++- **`severity` has four tiers, not five.** The validator accepts `low | medium | high | critical`. Vendor 5-tier ladders (Azure, SCC, GitHub) collapse into this set in the connector's `normalize()` — see [Hello, connector](/docs/connectors/hello-connector) for the rationale. Using `informational` here is a hard FAIL.+- **`category` is shared with the connector taxonomy.** Valid values are `network | endpoint | cloud | identity | application | data-exfil`. The detection coverage page groups by this field and the connector router uses it as a routing hint, so picking the wrong one means your rule runs against the wrong event stream.+- **`tags` are the MITRE wiring.** The marketplace MITRE filter only sees rules whose `tags` include at least one entry of the form `mitre.attack.T1234` or `mitre.attack.T1234.567`. Skip the tag, lose the visibility. Use [`attack.mitre.org`](https://attack.mitre.org/) to pick the technique — `T1078.004` (Valid Accounts: Cloud Accounts) is the right one for "someone logged in as root".+- **`log_source` is a hint, not a filter.** The rule engine doesn't gate evaluation on it today — it'll happily try your rule against any flattened event. The field exists so reviewers and downstream tooling know what shape the rule expects. Fill it in anyway; if/when log-source routing lands, your rule will already be ready.++## Step 3 — Write the `detection` block++This is the actual logic. The community tier supports the same Sigma-style shape that the rule engine's [`_sigma_fallback`](https://github.com/beenuar/AiSOC/blob/main/services/api/app/services/rule_engine.py) understands:++```yaml+detection:+ selection:+ eventName: ConsoleLogin+ userIdentity.type: Root+ responseElements.ConsoleLogin: Success+ condition: selection+```++Three things are happening here that you only see if you read the engine:++1. **Field names use dot notation for nested keys.** Before evaluation, every event passes through `_flatten_dict`, which turns `{"userIdentity": {"type": "Root"}}` into `{"useridentity.type": "Root"}`. Both keys and incoming JSON paths are lowercased, so your rule field can be written as `userIdentity.type`, `useridentity.type`, or `USERIDENTITY.TYPE` and they all collide on the same flattened key. The example uses the original AWS casing because that's what the CloudTrail docs say, and matching the upstream spelling makes the rule easier to audit.+2. **Values are matched as `lower(rule) in lower(event)`.** A `selection` entry of `eventName: ConsoleLogin` matches any event whose `eventname` field *contains* the substring `consolelogin`. That's loose by design — it means `ConsoleLogin`, `consoleLogin`, and `ConsoleLoginAttempt` all match. If you need exact equality, write a more specific value, or pair with a second selection that's intentionally narrow.+3. **List values are OR-ed.** `eventName: [ConsoleLogin, AssumeRole]` matches if *any* listed value is a substring of the event field. Inside a single `selection` block, multiple keys are AND-ed (all must match). Across selection blocks, you compose with `condition`.++The `condition` field is parsed as a small boolean expression over selection names. The example uses the simplest case (`condition: selection` — a single selection block must match), but the engine also supports `selection1 and selection2`, `selection1 or selection2`, and `not exclusion`. Anything more complex than that and you're better off graduating to a native spec.++### A note on `keywords` blocks++If a selection value is a YAML list at the top level (instead of a dict), the engine treats it as a **keywords** block — every keyword must appear *somewhere* in the flattened event payload (concatenated). Useful for "does this event mention any of these IOCs?" hunts, but the wider the search the noisier the rule, so prefer field-scoped `selection` blocks for production work.++## Step 4 — Document the rough edges++```yaml+false_positives:+ - Documented break-glass access — verify the change-management ticket+ before closing.+ - Initial AWS account setup before an IAM admin user exists.+playbook: tpl-credential-access+enabled: true+author: AiSOC Tutorial+created: '2026-05-12'+modified: '2026-05-12'+references:+ - https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#lock-away-credentials+```++These fields are optional from the validator's perspective but required for human review:++- **`false_positives`** is the field reviewers stare at first. If it's empty, expect a "what does this miss / over-fire on?" question on the PR. Be specific — "documented break-glass access" is a good answer; "noisy" is not.+- **`playbook`** is a soft binding to a [playbook template](/docs/concepts/playbooks). When a matching alert lands, the case auto-creator uses this hint to suggest a starting playbook. `tpl-credential-access` is the generic credential-access response template that ships with v1.0.+- **`enabled: true`** controls whether the rule runs in production. Community rules ship as `true` by default; flip to `false` if you need to land the rule but don't want it firing yet (e.g. you're staging a backfill).+- **`references`** is what an analyst follows when they're triaging an alert at 2am and have never heard of MITRE T1078.004. Link upstream vendor docs, ATT&CK pages, and (if relevant) the original blog post that prompted the rule.++## Step 5 — Write the positive fixture++The fixture is a single CloudTrail event that **must** trigger the rule. Drop it at `detections/fixtures/positive/<rule-slug>.json` (slug == `id` minus the `community-aisoc-` prefix, so the file is `hello-hunt-aws-root-login.json`):++```json+{+ "eventName": "ConsoleLogin",+ "eventSource": "signin.amazonaws.com",+ "userIdentity": {+ "type": "Root",+ "accountId": "123456789012",+ "arn": "arn:aws:iam::123456789012:root"+ },+ "sourceIPAddress": "203.0.113.42",+ "responseElements": {+ "ConsoleLogin": "Success"+ },+ "eventTime": "2026-05-12T18:00:00Z"+}+```++Three rules for fixtures:++1. **Use a real upstream shape.** Copy the JSON straight from the vendor's docs or from a sanitised production sample. AWS `203.0.113.0/24` is RFC 5737 documentation space; `12345...` is a documentation account ID. Never put real customer data in a fixture file.+2. **Keep it minimal.** Only include the fields the rule looks at, plus enough surrounding context that a reviewer can tell what the event represents. Bloated fixtures rot faster.+3. **Match every field the rule reads.** If the rule has three field comparisons, the positive fixture must satisfy all three. The validator doesn't enforce this for community rules, but the rule engine quietly returns zero matches if you forget one — and you'll only catch that when you run the engine yourself in [Step 7](#step-7--run-the-rule-against-the-rule-engine).++## Step 6 — Write the negative fixture++The negative fixture has the same shape but **must not** match. Drop it at `detections/fixtures/negative/hello-hunt-aws-root-login.json`:++```json+{+ "eventName": "ConsoleLogin",+ "eventSource": "signin.amazonaws.com",+ "userIdentity": {+ "type": "IAMUser",+ "userName": "alice",+ "accountId": "123456789012",+ "arn": "arn:aws:iam::123456789012:user/alice"+ },+ "sourceIPAddress": "203.0.113.42",+ "responseElements": {+ "ConsoleLogin": "Success"+ },+ "eventTime": "2026-05-12T18:00:00Z"+}+```++The trick: change *exactly one* thing from the positive fixture so that the rule no longer fires. Here, `userIdentity.type` flips from `Root` to `IAMUser`. Everything else stays identical so a reviewer can see at a glance what the rule is keying on.++This is also the format you'll use later when you tune the rule against a false positive — the FP becomes a new negative fixture, and the rule has to be tightened until it stops matching.++## Step 7 — Run the rule against the rule engine++The validator catches schema and provenance errors, but it does **not** execute community rules — that means a rule with a typo in the field name validates fine and quietly never fires in production. Don't trust the validator for matching behaviour. Drive the engine yourself:++```bash+python3 - <<'PY'+import json+import sys+from pathlib import Path++sys.path.insert(0, "services/api")+from app.services import rule_engine # noqa: E402++rule_body = Path(+ "detections/community/cloud/hello-hunt-aws-root-login.yaml"+).read_text()++positive = json.loads(+ Path("detections/fixtures/positive/hello-hunt-aws-root-login.json").read_text()+)+negative = json.loads(+ Path("detections/fixtures/negative/hello-hunt-aws-root-login.json").read_text()+)++pos_hits = rule_engine._sigma_fallback(rule_body, [positive])+neg_hits = rule_engine._sigma_fallback(rule_body, [negative])++print(f"positive matched: {len(pos_hits)} (expected 1)")+print(f"negative matched: {len(neg_hits)} (expected 0)")++assert len(pos_hits) == 1, "positive fixture should fire the rule"+assert len(neg_hits) == 0, "negative fixture must not fire the rule"+print("OK — both fixtures behave as expected")+PY+```++Expected output:++```text+positive matched: 1 (expected 1)+negative matched: 0 (expected 0)+OK — both fixtures behave as expected+```++If the positive fixture returns `0`, the rule's field path almost certainly disagrees with the actual flattened event. Re-read [Step 3](#step-3--write-the-detection-block) — the engine lowercases everything, so a typo like `userIdentity.Type` (uppercase `T`) is invisible until you run it through `_flatten_dict`. If the negative fixture returns `1`, your rule is too loose; tighten the values or add a second selection.++## Step 8 — Pin the contract with the validator++```bash+python3 scripts/validate_detections.py+```++For the example rule you should see (somewhere in the long output):++```text+WARN [community] detections/community/cloud/hello-hunt-aws-root-login.yaml no spec+PASS [community] detections/community/cloud/hello-hunt-aws-root-login.yaml+```++The `WARN` is expected — community rules are hand-authored, so there's no entry in `scripts/detection_specs/*.py` to round-trip them against. The `PASS` is what matters: it means the YAML is syntactically valid, has every required field, the severity and category are in the allowed set, and the `id` doesn't collide with any other rule in the corpus.++If you see a `FAIL` on the rule, read the message — it will tell you exactly which field is wrong. Common ones:++- `severity must be one of {low, medium, high, critical}` — you wrote `info` or `informational`. Use `low`.+- `category must be one of ...` — you used a connector category like `saas` or `vcs` that doesn't exist on the detection side.+- `duplicate id ...` — the `id` you picked is already used by another rule. Add your handle to the prefix.++## Step 9 — Surface it in the marketplace++```bash+pnpm marketplace:build+pnpm marketplace:sync+pnpm marketplace:check+```++What each step does:++1. **`marketplace:build`** regenerates `marketplace/index.json` from the on-disk corpus. Your community rule appears under `categories.detections` with `verified: false` and `source: "community"`.+2. **`marketplace:sync`** mirrors `marketplace/index.json` to `apps/web/public/marketplace/index.json` so the in-app catalog grid picks it up. Skipping this step is the most common reason a rule passes validation locally but doesn't show in the UI.+3. **`marketplace:check`** asserts that the two copies match. CI fails on the smallest diff, so always run sync before opening the PR.++After this, the rule shows up in the in-app marketplace at `/marketplace` with a `Community` badge, gated by the MITRE filter you set in `tags`.++## Step 10 — (Optional) Replay the rule against your tenant's live events++If you have a running AiSOC dev stack and want to see the rule fire end to end:++```bash+# 1. Restart the API so it reloads the detection corpus from disk.+docker compose -f docker-compose.dev.yml restart api++# 2. Replay the positive fixture into the ingest pipeline.+curl -X POST http://localhost:8000/v1/ingest/batch \+ -H "Content-Type: application/json" \+ -H "X-Tenant-ID: demo" \+ -d "@detections/fixtures/positive/hello-hunt-aws-root-login.json"++# 3. Watch the rule fire.+curl -s "http://localhost:8000/v1/alerts?rule_id=community-aisoc-hello-hunt-aws-root-login" \+ -H "X-Tenant-ID: demo" | jq '.items[0]'+```++You should see one alert come back with `severity: "high"` and the original CloudTrail event embedded under `raw`. Drop the rule ID into the [Explain Drawer](/docs/api/rest) and you'll get the full lineage: which selection block matched, which field values, and which playbook template the rule recommends.++## Graduating from `community/` to `native/`++When you've run the rule against a few weeks of real telemetry and you want it promoted into the curated v1.0 set:++1. **Open a promotion PR.** Tag a maintainer. The PR should include the false-positive rate you observed, the volume of events evaluated, and at least one example of a real alert the rule produced.+2. **Add a spec entry.** Native rules are generated from `scripts/detection_specs/<category>.py` so the corpus stays auditable. The maintainer will help you author the spec; the existing entries in `cloud.py` are good templates.+3. **Move the YAML.** From `detections/community/<category>/<slug>.yaml` to `detections/<category>/<slug>.yaml`. Change the `id` prefix from `community-...` to `det-` and bump `version`.+4. **Re-run validation.** `python3 scripts/validate_detections.py --strict-fixtures` must pass for native rules — fixtures are no longer optional, and the spec round-trip is enforced.+5. **Mark it verified.** The marketplace build will switch `verified: true` and `source: "native"` automatically based on the file path.++That's the full path. Promotion is intentionally manual — the curated set is one of AiSOC's sharpest selling points, and we'd rather move slowly than dilute it.++## Related++- [Detection coverage](/docs/detections/coverage) — what we ship in the curated v1.0 set, by family.+- [Hello, connector](/docs/connectors/hello-connector) — the sister tutorial for the ingest side.+- [Plugin SDK overview](/docs/plugins/overview) — for connectors and enrichers distributed outside the monorepo.+- [Contributing guidelines](/docs/contributing/guidelines) — broader expectations for PRs that touch `detections/`.diff --git a/apps/docs/docs/plugins/hello-plugin.md b/apps/docs/docs/plugins/hello-plugin.mdnew file mode 100644index 000000000..d5b663ee9--- /dev/null+++ b/apps/docs/docs/plugins/hello-plugin.md@@ -0,0 +1,334 @@+---+sidebar_position: 3+title: Hello, plugin — write your first AiSOC enricher+description: A walkthrough that ships a runnable AiSOC plugin end to end. Author the manifest, implement the EnricherPlugin contract, register it with the runtime, and pin the whole thing with a smoke test — no vendor account, no API key, just the Python plugin SDK front to back.+---++# Hello, plugin++This tutorial walks you end-to-end through the work of adding a new plugin to AiSOC. By the end you will have:++- An `aisoc-plugin.yaml` manifest the plugin loader can validate.+- A `plugin.py` that subclasses `EnricherPlugin` and exposes a `create_plugin()` factory.+- A working `on_load()` lifecycle hook that reads tenant config out of `PluginContext`.+- A working `enrich()` method that returns a typed `EnrichmentResult` the platform can write to indicator records.+- A `PluginRegistry` flow that mirrors what the AiSOC runtime does at boot.+- A smoke test that pins all of the above against the real loader, so the docs page can never silently drift away from the code.++The example enricher is intentionally trivial — it computes a deterministic SHA-256 hash of the indicator value and returns the digest as enrichment metadata. There is no network call, no external API, no credential. That keeps the tutorial:++- runnable in air-gapped environments,+- reproducible in CI without secrets,+- focused on the *contract* rather than on third-party authentication,+- safe to copy as a starting point — when you swap the hash for a vendor SDK the manifest and entry-point shape stay identical.++When you're ready to write a real enricher, copy the example, replace the hashing call with whatever vendor SDK you need, and keep the manifest + `create_plugin` factory shape exactly as-is.++## Where the example lives++The reference plugin and its smoke test live at:++```text+plugins/community/_examples/hello-plugin/aisoc-plugin.yaml+plugins/community/_examples/hello-plugin/plugin.py+plugins/community/_examples/hello-plugin/README.md+packages/plugin-sdk-py/tests/test_hello_plugin_example.py+```++The `_examples/` directory is **deliberately not** picked up by [`scripts/build_marketplace.py`](https://github.com/beenuar/AiSOC/blob/main/scripts/build_marketplace.py). The marketplace builder scans `plugins/community/<id>/plugin.yaml`, and the tutorial avoids both signals on purpose:++1. It lives one directory deeper, under `_examples/hello-plugin/`.+2. Its manifest is named `aisoc-plugin.yaml` (the SDK loader filename), not `plugin.yaml` (the marketplace filename).++That means the example can never accidentally ship to a real tenant. The smoke test pins this invariant — if anyone "fixes" either of those, CI fails. The same pattern is used by [Hello, connector](/docs/connectors/hello-connector) and [Hello, hunt](/docs/detections/hello-hunt).++## Plugin types — pick the right base class first++The Python SDK ships three plugin types, each with a different contract:++| Type | Base class | Contract method | Use it when… |+| ----------- | --------------------------- | -------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |+| `enricher` | `EnricherPlugin` | `async def enrich(req, ctx) -> EnrichmentResult` | You take an indicator (IP, domain, URL, hash, email) and return structured intel about it. |+| `action` | `ActionPlugin` | `async def execute(req, ctx) -> ActionResult` | You take a response action (block IP, isolate host, page on-call) and report success or failure back. |+| `connector` | `ConnectorPlugin` *(plugin)*| `async def fetch_alerts(...)` | You poll a vendor API and emit normalized alerts. **For first-class connectors use [`BaseConnector`](/docs/connectors/hello-connector) instead — the connector plugin type exists for marketplace-shipped third-party integrations.** |++This tutorial sticks to `enricher` because it's the simplest contract — one input record in, one output record out, no scheduler, no retry policy, no normalisation. When you've internalised the manifest + `create_plugin()` shape here, the action and connector plugin types will feel familiar.++## Step 1 — Pick a stable plugin `id`++```yaml+id: aisoc.hello-plugin+```++Three rules to internalise:++1. **Lowercase, dotted, no spaces.** The `id` is a wire identifier — it ends up in `marketplace/index.json`, in tenant install records, and in audit logs whenever the registry resolves an enricher. Treat it like a primary key.+2. **Prefix with your namespace.** Use `<your-org>.` or `<your-handle>.` so two contributors don't ship `vt-enricher` and collide. The `aisoc.` prefix is reserved for first-party tutorial and reference plugins; real contributions use `acme.virustotal` or `jdoe.greynoise`.+3. **Never change it after merge.** Renaming a plugin orphans every tenant install that references the old id. If the plugin needs a v2 with breaking config changes, give it a new id and deprecate the old one — the marketplace publishing flow has a `deprecated` field for exactly this reason. See [Publishing plugins](/docs/plugins/publishing).++The example uses `aisoc.hello-plugin` because the AiSOC project itself is the author.++## Step 2 — Write the manifest++The manifest is parsed by [`load_manifest`](https://github.com/beenuar/AiSOC/blob/main/packages/plugin-sdk-py/src/aisoc_plugin_sdk/loader.py) and validated against [`PluginManifest`](https://github.com/beenuar/AiSOC/blob/main/packages/plugin-sdk-py/src/aisoc_plugin_sdk/plugin.py). Anything that doesn't match the schema is rejected at load time with a `PluginLoadError`.++```yaml title="plugins/community/_examples/hello-plugin/aisoc-plugin.yaml"+id: aisoc.hello-plugin+name: Hello Plugin (Tutorial)+version: 1.0.0+plugin_type: enricher+description: >+ Tutorial enricher that hashes the indicator value with SHA-256 locally.+ Reference implementation for apps/docs/docs/plugins/hello-plugin.md.+ Deliberately offline so it can run in air-gapped environments and CI+ without any external API calls.+author: AiSOC Tutorial+tags:+ - tutorial+ - enricher+ - offline+```++A few non-obvious things:++- **`plugin_type` is regex-validated.** The schema enforces `^(enricher|action|connector)$`. Misspelling it (`enrichers`, `Enricher`, `connect`) fails the load with a Pydantic validation error before your `plugin.py` is even imported.+- **`version` is treated as SemVer.** The marketplace orders installable versions by SemVer comparison, and the publishing flow uses it to decide whether a tenant has an upgrade available. Use real SemVer (`1.0.0`, `1.0.1`, `2.0.0-rc.1`) — calendar versions like `2025.05` will sort but won't trigger upgrade notifications cleanly.+- **`tags` are free-form** and mostly used for marketplace filtering. There is no enforced taxonomy (yet). Keep them lowercase and hyphen-separated for consistency with the connectors layer.+- **Two manifest filenames exist.** The SDK loader (`load_manifest`) reads `aisoc-plugin.yaml`. The marketplace builder (`scripts/build_marketplace.py`) reads `plugin.yaml`. The two formats are nearly identical, but the marketplace one carries extra publish metadata (signature URL, registry URL, install instructions). When you graduate from `_examples/` to `plugins/community/<your-id>/`, you will write **both** files. See [Publishing plugins](/docs/plugins/publishing) for the marketplace shape.++## Step 3 — Implement the plugin class++Now the code. Two ways to do this — class-based and decorator-based. The tutorial uses the class-based path because it makes the `on_load`/`on_unload` lifecycle hooks explicit, which you'll want as soon as a real enricher needs an HTTP client or a cached secret.++```python title="plugins/community/_examples/hello-plugin/plugin.py"+from __future__ import annotations++import hashlib++from aisoc_plugin_sdk import (+ AiSOCPlugin,+ EnricherPlugin,+ EnrichmentRequest,+ EnrichmentResult,+ PluginContext,+ PluginManifest,+)+++class HelloPlugin(EnricherPlugin):+ """Deterministic, offline enricher used by the hello-plugin tutorial."""++ @property+ def manifest(self) -> PluginManifest:+ return PluginManifest(+ id="aisoc.hello-plugin",+ name="Hello Plugin (Tutorial)",+ version="1.0.0",+ description=(+ "Tutorial enricher that hashes indicator values locally. "+ "Reference implementation for "+ "apps/docs/docs/plugins/hello-plugin.md."+ ),+ author="AiSOC Tutorial",+ tags=["tutorial", "enricher", "offline"],+ plugin_type="enricher",+ )+```++Three things to call out:++1. **`manifest` is a `@property`, not a method.** The SDK defines `manifest` as `@property @abstractmethod` on `AiSOCPlugin`. If you write `def manifest(self)` instead of `@property def manifest(self)`, the registry will receive a *bound method* instead of a `PluginManifest` instance and every lookup will explode at runtime. The smoke test pins the property contract.+2. **The manifest values must mirror `aisoc-plugin.yaml`.** The loader trusts the YAML for discovery, but every code path past the load step reads the manifest off the *plugin instance*. If the two drift, the marketplace will list one version and the runtime will report another. Keep them in lockstep, or have a build step that reads the YAML and constructs the manifest from it.+3. **You're inheriting from `EnricherPlugin`, not `AiSOCPlugin` directly.** This is what makes the registry route enrichment requests to your plugin. Inheriting from the wrong base class is a silent bug — the plugin loads fine, registers fine, and never receives any enrichment work.++## Step 4 — Implement the lifecycle hook++`on_load(ctx)` is called exactly once, after the plugin is registered and before the runtime sends it any work. The `PluginContext` argument carries the API base URL, a scoped API token, and the tenant's plugin config dict.++```python+async def on_load(self, ctx: PluginContext) -> None:+ self._algorithm = (ctx.config.get("algorithm") or "sha256").lower()+ if self._algorithm not in hashlib.algorithms_guaranteed:+ raise ValueError(+ f"Unsupported hash algorithm: {self._algorithm!r}. "+ f"Pick one of: {sorted(hashlib.algorithms_guaranteed)}"+ )+```++What's happening:++- **`ctx.config` is the per-tenant configuration.** Whatever the tenant set in their plugin install settings ends up here. The tutorial reads a single optional `algorithm` key — a real enricher might read API endpoints, rate-limit budgets, or per-customer feature flags.+- **The hook can raise.** If `on_load` raises, the runtime marks the plugin install as failed and surfaces the error message to the tenant. Use this to fail fast on bad config — empty API keys, unreachable endpoints, malformed allow-lists. Failing here is cheaper than failing later in `enrich()`, where the same error will repeat for every indicator.+- **There is also `on_unload()`.** It's called when the plugin is uninstalled or when the runtime shuts down. The tutorial doesn't override it because there's nothing to clean up. A real enricher with a long-lived `httpx.AsyncClient` should `await client.aclose()` here.++## Step 5 — Implement `enrich()`++This is the actual work the platform calls.++```python+async def enrich(+ self, request: EnrichmentRequest, ctx: PluginContext+) -> EnrichmentResult:+ algorithm = getattr(self, "_algorithm", "sha256")+ digest = hashlib.new(algorithm, request.indicator_value.encode("utf-8")).hexdigest()++ return EnrichmentResult(+ indicator_type=request.indicator_type,+ indicator_value=request.indicator_value,+ enrichments={+ "hello_plugin.algorithm": algorithm,+ "hello_plugin.digest": digest,+ "hello_plugin.length": len(digest),+ },+ tags=["hello-plugin"],+ malicious=None,+ confidence=None,+ raw={"input": request.indicator_value, "digest": digest},+ )+```++The contract:++- **`request.indicator_type` and `request.indicator_value` come from the indicator that triggered enrichment.** The five types AiSOC routes today are `ip | domain | url | hash | email`. A real enricher should branch on `indicator_type` and short-circuit (or return an empty result) for types it doesn't support — the tutorial hashes everything because hash-of-anything is well-defined.+- **`enrichments` is a flat dict that's merged into the indicator record.** Namespace your keys with `<plugin-id>.<field>` (the tutorial uses `hello_plugin.*`) so two enrichers writing to the same indicator can't stomp each other.+- **`tags` are appended to the indicator's tag list.** Use them for downstream filtering — e.g. `["malicious", "vt-detected"]` or `["benign", "alexa-top-1k"]`.+- **`malicious` is a tri-state.** `True` means the enricher is confident it's bad. `False` means the enricher is confident it's clean. `None` means the enricher has no opinion. **Don't return `False` just because your API returned no hits** — that's an opinion you don't have. The tutorial returns `None` because hashing a value tells you nothing about its reputation.+- **`confidence` is `[0.0, 1.0]` or `None`.** Pydantic enforces the range; out-of-band values raise at construction time. Skip the field unless your upstream actually returns a confidence score.+- **`raw` is for audit.** Store the upstream API response (or a redacted version) so investigators can reproduce the decision later. Don't put secrets in here — the indicator record is readable by anyone with case access.++## Step 6 — Expose `create_plugin()`++The loader doesn't import your class directly. It looks for a top-level `create_plugin()` factory in `plugin.py` and uses whatever it returns:++```python+def create_plugin() -> AiSOCPlugin:+ """Factory called by ``load_plugin_from_directory``."""+ return HelloPlugin()+```++Why a factory and not the class itself:++- **Per-tenant isolation.** Every tenant install gets its own plugin instance, so per-tenant state (cached HTTP client, last-seen timestamp, rate-limit bucket) lives on the instance and can't leak across tenants.+- **Lazy construction.** The class can defer expensive work (loading a model, opening a file) until the runtime actually needs it. The loader pays the cost of `create_plugin()`; the import of `plugin.py` stays cheap.+- **Test-friendly.** The smoke test calls `load_plugin_from_directory(...)` exactly the way the runtime does. If `create_plugin` is missing, returns `None`, or returns something that isn't an `AiSOCPlugin`, the loader raises `PluginLoadError` with a precise message. You don't need to mock anything.++## Step 7 — Run the smoke test++The companion smoke test lives at `packages/plugin-sdk-py/tests/test_hello_plugin_example.py` and pins the entire contract: the files exist, the example is excluded from the marketplace, the loader accepts the manifest + factory shape, the lifecycle hook validates config, and the enrichment is deterministic.++```bash+cd packages/plugin-sdk-py+.venv/bin/python -m pytest tests/test_hello_plugin_example.py -v+```++Expected output:++```text+tests/test_hello_plugin_example.py::test_hello_plugin_example_files_exist PASSED+tests/test_hello_plugin_example.py::test_hello_plugin_is_excluded_from_marketplace PASSED+tests/test_hello_plugin_example.py::test_hello_plugin_loads_via_loader PASSED+tests/test_hello_plugin_example.py::test_on_load_defaults_to_sha256 PASSED+tests/test_hello_plugin_example.py::test_on_load_accepts_configured_algorithm PASSED+tests/test_hello_plugin_example.py::test_on_load_rejects_unknown_algorithm PASSED+tests/test_hello_plugin_example.py::test_enrich_is_deterministic PASSED+tests/test_hello_plugin_example.py::test_enrich_uses_configured_algorithm PASSED+tests/test_hello_plugin_example.py::test_hello_plugin_registers_as_enricher PASSED+9 passed+```++The two tests worth understanding before you write your own plugin:++### `test_hello_plugin_loads_via_loader`++```python+def test_hello_plugin_loads_via_loader() -> None:+ plugin = load_plugin_from_directory(HELLO_PLUGIN_DIR)++ assert isinstance(plugin, EnricherPlugin)+ assert plugin.manifest.id == "aisoc.hello-plugin"+ assert plugin.manifest.plugin_type == "enricher"+```++This is the single most useful test you can write for a plugin. It calls `load_plugin_from_directory` exactly the way the runtime does, which proves:++- the manifest YAML is parseable and schema-valid,+- the entry point file exists and is importable,+- `create_plugin()` is defined and returns the expected base class,+- the manifest property returns a real `PluginManifest` (not a method, not `None`).++If this test passes, the runtime will be able to load your plugin. If it fails, the error message points at exactly which contract you broke.++### `test_enrich_is_deterministic`++```python+async def test_enrich_is_deterministic(ctx: PluginContext) -> None:+ plugin = load_plugin_from_directory(HELLO_PLUGIN_DIR)+ await plugin.on_load(ctx)++ request = EnrichmentRequest(+ indicator_type="ip", indicator_value="203.0.113.42",+ )+ expected_digest = hashlib.sha256(b"203.0.113.42").hexdigest()++ result_a = await plugin.enrich(request, ctx)+ result_b = await plugin.enrich(request, ctx)++ assert result_a.enrichments["hello_plugin.digest"] == expected_digest+ assert result_a.model_dump() == result_b.model_dump()+```++Two reasons this matters:++1. **The same input must always yield the same enrichment.** That's what makes the result cacheable, replayable, and trustworthy in case investigations. If your enricher hits a vendor API, mock the API in the test (with `respx` or `pytest-httpx`) so the test stays deterministic — you're testing *your code*, not the vendor's uptime.+2. **Snapshot equality on `model_dump()`** is the cheapest way to catch accidental breaking changes. If you add a new key to `enrichments`, this assertion still passes (because it's the same on both calls). If you start mutating shared state across calls, it fails immediately.++## Step 8 — Register with the runtime++In production, the AiSOC plugin runtime constructs a `PluginRegistry`, loads every installed plugin from disk, and calls `load_all()` once per tenant. The smoke test mirrors this so you can validate it locally:++```python+async def test_hello_plugin_registers_as_enricher(ctx: PluginContext) -> None:+ plugin = load_plugin_from_directory(HELLO_PLUGIN_DIR)+ registry = PluginRegistry()+ registry.register(plugin)++ await registry.load_all(ctx)++ assert len(registry) == 1+ enrichers = registry.enrichers()+ assert len(enrichers) == 1+ assert enrichers[0].manifest.id == "aisoc.hello-plugin"+ assert registry.get("aisoc.hello-plugin") is plugin+```++Three things this proves:++- `PluginRegistry.register()` accepts the plugin and stores it under `manifest.id`. Registering the same id twice raises — the runtime relies on this to surface duplicate installs.+- `load_all()` calls `on_load(ctx)` for every plugin and propagates exceptions. If your `on_load` raises, the registry stays in a half-loaded state and the runtime surfaces the error to the tenant.+- `enrichers()` returns only `EnricherPlugin` instances. The same registry can hold actions and connectors side-by-side; the type-segmented accessors (`enrichers()`, `actions()`, `connectors()`) are how the runtime routes work.++## What you don't get from the tutorial++The tutorial is intentionally narrow. Real plugins eventually need:++- **Authentication.** Most real enrichers need an API key or OAuth token. Read it from `ctx.config`, never from env vars — env vars are global and the runtime sets them per-process, not per-tenant. The credential vault encrypts secrets at rest; see [Operations → Credentials](/docs/operations/credentials).+- **An HTTP client.** Open one `httpx.AsyncClient` in `on_load`, store it on `self`, reuse it from every `enrich()`, and close it in `on_unload`. Don't open a fresh client per request — connection pooling matters even for low-volume enrichers.+- **Error handling.** The current `enrich()` will raise if the algorithm is missing or invalid. A real enricher should catch upstream API errors, classify them (timeout vs. 4xx vs. 5xx), and either return an empty `EnrichmentResult` or raise — the runtime treats unhandled exceptions as fatal for the request, not for the plugin.+- **Rate limiting.** If your vendor enforces a request-per-second budget, enforce it in the plugin with `asyncio.Semaphore` or `aiolimiter`. The runtime won't do it for you, and bursting will get the tenant's API key throttled or banned.+- **Observability.** The `AiSOCClient` (exported from `aisoc_plugin_sdk`) gives you authenticated access to the AiSOC API for emitting plugin-side events and metrics. Use it sparingly — every call goes back over the network.++When you wire any of these in, the contract you wrote in this tutorial — manifest, `create_plugin()`, `on_load`, `enrich`, registry — does not change. That's the value of the SDK.++## Graduating from `_examples/` to the marketplace++When you're ready to ship the plugin to real tenants:++1. Copy `plugins/community/_examples/hello-plugin/` to `plugins/community/<your-id>/`.+2. Rename `aisoc-plugin.yaml` → `plugin.yaml` and add the marketplace fields (`signature_url`, `registry_url`, `install_command`). The full schema lives in [Publishing plugins](/docs/plugins/publishing).+3. Sign the plugin with your maintainer Ed25519 key (`scripts/sign_plugin.py`).+4. Add an entry to `marketplace/index.json` and run `pnpm marketplace:sync`.+5. Open a PR. Maintainers will review the plugin, validate the signature against your registered public key, and merge.++After merge the plugin shows up in every tenant's in-app marketplace under the **Community** badge. Tenants install it with one click, and the runtime calls the same `load_plugin_from_directory()` you tested locally.++That's the whole loop.diff --git a/apps/docs/sidebars.ts b/apps/docs/sidebars.tsindex 5df112d45..5ef24189e 100644--- a/apps/docs/sidebars.ts+++ b/apps/docs/sidebars.ts@@ -37,6 +37,7 @@ const sidebars: SidebarsConfig = {items: ["connectors/index","connectors/hello-connector",+ "connectors/endpoint-decision-matrix","connectors/api-coverage","connectors/universal-capture","connectors/azure-entra",@@ -66,7 +67,7 @@ const sidebars: SidebarsConfig = {{type: "category",label: "Detections",- items: ["detections/coverage"],+ items: ["detections/coverage", "detections/hello-hunt"],},{type: "category",@@ -94,6 +95,7 @@ const sidebars: SidebarsConfig = {items: ["plugins/overview","plugins/python-sdk",+ "plugins/hello-plugin","plugins/go-sdk","plugins/publishing",],diff --git a/apps/web/public/marketplace/curated.json b/apps/web/public/marketplace/curated.jsonindex 7ac577bd4..1c163978e 100644--- a/apps/web/public/marketplace/curated.json+++ b/apps/web/public/marketplace/curated.json@@ -1,10 +1,10 @@{"$schema": "https://aisoc.example.com/schemas/curated/v1.json","version": "1.0.0",- "generated": "2026-05-11T03:45:32Z",+ "generated": "2026-05-12T20:45:55Z","stats": {- "considered": 1047,- "eligible": 1043,+ "considered": 1048,+ "eligible": 1044,"selected": 416,"min_per_family": 25,"min_total": 300,diff --git a/apps/web/public/marketplace/index.json b/apps/web/public/marketplace/index.jsonindex 330225aac..65b4633b1 100644--- a/apps/web/public/marketplace/index.json+++ b/apps/web/public/marketplace/index.json@@ -1,7 +1,7 @@{"$schema": "https://example.com/schemas/marketplace/v1.json","version": "1.0.0",- "generated": "2026-05-11T03:45:31Z",+ "generated": "2026-05-12T20:45:54Z","categories": [{"id": "playbooks",@@ -20,18 +20,20 @@}],"stats": {- "total": 7104,+ "total": 7105,"playbooks": 62,- "detections": 6993,+ "detections": 6994,"plugins": 49,"verified": 985,- "community": 0,+ "community": 1,"by_tier": {"beta": 6,+ "community": 1,"imported": 6113,"stable": 985},"detections_by_tier": {+ "community": 1,"imported": 6113,"stable": 880},@@ -164,7 +166,7 @@"T1078.001": 10,"T1078.002": 14,"T1078.003": 11,- "T1078.004": 97,+ "T1078.004": 98,"T1082": 49,"T1083": 28,"T1087": 21,@@ -534,8 +536,11 @@"T1690": 1},"unique_techniques": 493,- "total_with_mitre": 5777,+ "total_with_mitre": 5778,"by_tier": {+ "community": {+ "T1078.004": 1+ },"stable": {"T1003": 2,"T1003.001": 3,@@ -28555,6 +28560,30 @@"enabled": true,"path": "detections/playbooks/cloud-takeover-response.yaml"},+ {+ "id": "community-aisoc-hello-hunt-aws-root-login",+ "type": "detection",+ "name": "[Hello Hunt] AWS Root Account Console Login",+ "description": "Tutorial detection from apps/docs/docs/detections/hello-hunt.md.\nFires when AWS CloudTrail records a successful console login under the\n`Root` user identity. Real environments should keep the AWS root account\nunused except for break-glass \u2014 any successful console login is worth a\nhuman eyeballing.\n\nThis rule is intentionally minimal so first-time contributors can read\nit end to end. See `apps/docs/docs/detections/hello-hunt.md` for a full\nwalkthrough.",+ "version": "1.0.0",+ "author": "AiSOC Tutorial",+ "tags": [+ "tlp.white",+ "tutorial"+ ],+ "severity": "high",+ "category": "cloud",+ "mitre_techniques": [+ "T1078.004"+ ],+ "log_source": "aws",+ "playbook": "tpl-credential-access",+ "verified": false,+ "source": "community",+ "tier": "community",+ "enabled": true,+ "path": "detections/community/cloud/hello-hunt-aws-root-login.yaml"+ },{"id": "container-escape-response-v1","type": "detection",diff --git a/detections/community/cloud/hello-hunt-aws-root-login.yaml b/detections/community/cloud/hello-hunt-aws-root-login.yamlnew file mode 100644index 000000000..5bf58f073--- /dev/null+++ b/detections/community/cloud/hello-hunt-aws-root-login.yaml@@ -0,0 +1,39 @@+id: community-aisoc-hello-hunt-aws-root-login+name: "[Hello Hunt] AWS Root Account Console Login"+description: |+ Tutorial detection from apps/docs/docs/detections/hello-hunt.md.+ Fires when AWS CloudTrail records a successful console login under the+ `Root` user identity. Real environments should keep the AWS root account+ unused except for break-glass — any successful console login is worth a+ human eyeballing.++ This rule is intentionally minimal so first-time contributors can read+ it end to end. See `apps/docs/docs/detections/hello-hunt.md` for a full+ walkthrough.+version: 1.0.0+severity: high+category: cloud+tags:+ - mitre.attack.T1078.004+ - tlp.white+ - tutorial+log_source:+ product: aws+ service: cloudtrail+detection:+ selection:+ eventName: ConsoleLogin+ userIdentity.type: Root+ responseElements.ConsoleLogin: Success+ condition: selection+false_positives:+ - Documented break-glass access — verify the change-management ticket+ before closing.+ - Initial AWS account setup before an IAM admin user exists.+playbook: tpl-credential-access+enabled: true+author: AiSOC Tutorial+created: '2026-05-12'+modified: '2026-05-12'+references:+ - https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#lock-away-credentialsdiff --git a/detections/fixtures/negative/hello-hunt-aws-root-login.json b/detections/fixtures/negative/hello-hunt-aws-root-login.jsonnew file mode 100644index 000000000..8cbf7c888--- /dev/null+++ b/detections/fixtures/negative/hello-hunt-aws-root-login.json@@ -0,0 +1,15 @@+{+ "eventName": "ConsoleLogin",+ "eventSource": "signin.amazonaws.com",+ "userIdentity": {+ "type": "IAMUser",+ "userName": "alice",+ "accountId": "123456789012",+ "arn": "arn:aws:iam::123456789012:user/alice"+ },+ "sourceIPAddress": "203.0.113.42",+ "responseElements": {+ "ConsoleLogin": "Success"+ },+ "eventTime": "2026-05-12T18:00:00Z"+}diff --git a/detections/fixtures/positive/hello-hunt-aws-root-login.json b/detections/fixtures/positive/hello-hunt-aws-root-login.jsonnew file mode 100644index 000000000..e0d2ae6d0--- /dev/null+++ b/detections/fixtures/positive/hello-hunt-aws-root-login.json@@ -0,0 +1,14 @@+{+ "eventName": "ConsoleLogin",+ "eventSource": "signin.amazonaws.com",+ "userIdentity": {+ "type": "Root",+ "accountId": "123456789012",+ "arn": "arn:aws:iam::123456789012:root"+ },+ "sourceIPAddress": "203.0.113.42",+ "responseElements": {+ "ConsoleLogin": "Success"+ },+ "eventTime": "2026-05-12T18:00:00Z"+}diff --git a/marketplace/curated.json b/marketplace/curated.jsonindex 7ac577bd4..1c163978e 100644--- a/marketplace/curated.json+++ b/marketplace/curated.json@@ -1,10 +1,10 @@{"$schema": "https://aisoc.example.com/schemas/curated/v1.json","version": "1.0.0",- "generated": "2026-05-11T03:45:32Z",+ "generated": "2026-05-12T20:45:55Z","stats": {- "considered": 1047,- "eligible": 1043,+ "considered": 1048,+ "eligible": 1044,"selected": 416,"min_per_family": 25,"min_total": 300,diff --git a/marketplace/index.json b/marketplace/index.jsonindex 330225aac..65b4633b1 100644--- a/marketplace/index.json+++ b/marketplace/index.json@@ -1,7 +1,7 @@{"$schema": "https://example.com/schemas/marketplace/v1.json","version": "1.0.0",- "generated": "2026-05-11T03:45:31Z",+ "generated": "2026-05-12T20:45:54Z","categories": [{"id": "playbooks",@@ -20,18 +20,20 @@}],"stats": {- "total": 7104,+ "total": 7105,"playbooks": 62,- "detections": 6993,+ "detections": 6994,"plugins": 49,"verified": 985,- "community": 0,+ "community": 1,"by_tier": {"beta": 6,+ "community": 1,"imported": 6113,"stable": 985},"detections_by_tier": {+ "community": 1,"imported": 6113,"stable": 880},@@ -164,7 +166,7 @@"T1078.001": 10,"T1078.002": 14,"T1078.003": 11,- "T1078.004": 97,+ "T1078.004": 98,"T1082": 49,"T1083": 28,"T1087": 21,@@ -534,8 +536,11 @@"T1690": 1},"unique_techniques": 493,- "total_with_mitre": 5777,+ "total_with_mitre": 5778,"by_tier": {+ "community": {+ "T1078.004": 1+ },"stable": {"T1003": 2,"T1003.001": 3,@@ -28555,6 +28560,30 @@"enabled": true,"path": "detections/playbooks/cloud-takeover-response.yaml"},+ {+ "id": "community-aisoc-hello-hunt-aws-root-login",+ "type": "detection",+ "name": "[Hello Hunt] AWS Root Account Console Login",+ "description": "Tutorial detection from apps/docs/docs/detections/hello-hunt.md.\nFires when AWS CloudTrail records a successful console login under the\n`Root` user identity. Real environments should keep the AWS root account\nunused except for break-glass \u2014 any successful console login is worth a\nhuman eyeballing.\n\nThis rule is intentionally minimal so first-time contributors can read\nit end to end. See `apps/docs/docs/detections/hello-hunt.md` for a full\nwalkthrough.",+ "version": "1.0.0",+ "author": "AiSOC Tutorial",+ "tags": [+ "tlp.white",+ "tutorial"+ ],+ "severity": "high",+ "category": "cloud",+ "mitre_techniques": [+ "T1078.004"+ ],+ "log_source": "aws",+ "playbook": "tpl-credential-access",+ "verified": false,+ "source": "community",+ "tier": "community",+ "enabled": true,+ "path": "detections/community/cloud/hello-hunt-aws-root-login.yaml"+ },{"id": "container-escape-response-v1","type": "detection",diff --git a/packages/plugin-sdk-py/tests/test_hello_plugin_example.py b/packages/plugin-sdk-py/tests/test_hello_plugin_example.pynew file mode 100644index 000000000..bc28aacd6--- /dev/null+++ b/packages/plugin-sdk-py/tests/test_hello_plugin_example.py@@ -0,0 +1,192 @@+"""Smoke test for the ``plugins/community/_examples/hello-plugin`` tutorial.++This test pins the contract that the hello-plugin tutorial documents in+``apps/docs/docs/plugins/hello-plugin.md``. If the tutorial drifts from+the reference implementation, this test will fail and force the docs to+be updated in lockstep.++It deliberately avoids importing the example as a Python module — the+tutorial loads it through ``load_plugin_from_directory`` exactly the way+the AiSOC plugin runtime would, which doubles as proof that the+manifest + entry-point shape on disk is loadable in production.+"""++from __future__ import annotations++import hashlib+from pathlib import Path++import pytest++from aisoc_plugin_sdk import (+ EnricherPlugin,+ EnrichmentRequest,+ PluginContext,+ PluginRegistry,+)+from aisoc_plugin_sdk.loader import load_plugin_from_directory++REPO_ROOT = Path(__file__).resolve().parents[3]+HELLO_PLUGIN_DIR = REPO_ROOT / "plugins" / "community" / "_examples" / "hello-plugin"+++# ── Sanity: the example exists where the docs say it does ────────────────────+++def test_hello_plugin_example_files_exist() -> None:+ """The tutorial points at concrete files; make sure they survive renames."""++ assert HELLO_PLUGIN_DIR.is_dir(), (+ f"Expected hello-plugin example at {HELLO_PLUGIN_DIR}. "+ "If this directory moved, update apps/docs/docs/plugins/hello-plugin.md."+ )+ assert (HELLO_PLUGIN_DIR / "aisoc-plugin.yaml").is_file()+ assert (HELLO_PLUGIN_DIR / "plugin.py").is_file()+ assert (HELLO_PLUGIN_DIR / "README.md").is_file()+++def test_hello_plugin_is_excluded_from_marketplace() -> None:+ """Examples under ``_examples/`` must never ship in the marketplace.++ The marketplace builder (``scripts/build_marketplace.py``) discovers+ plugins by walking ``plugins/community/<id>/plugin.yaml``. The tutorial+ avoids both signals on purpose: it lives one directory deeper, and its+ manifest is named ``aisoc-plugin.yaml`` (the SDK loader filename), not+ ``plugin.yaml`` (the marketplace filename). If anyone "fixes" either+ of those, this test will catch it.+ """++ assert HELLO_PLUGIN_DIR.parent.name == "_examples", (+ "Tutorial example must stay under plugins/community/_examples/ so the "+ "marketplace builder ignores it."+ )+ assert not (HELLO_PLUGIN_DIR / "plugin.yaml").exists(), (+ "Tutorial example must NOT define a plugin.yaml — that would make "+ "scripts/build_marketplace.py index it and ship it to real tenants."+ )+++# ── Loader contract ──────────────────────────────────────────────────────────+++def test_hello_plugin_loads_via_loader() -> None:+ """Walks the same code path the runtime uses to load real plugins."""++ plugin = load_plugin_from_directory(HELLO_PLUGIN_DIR)++ assert isinstance(plugin, EnricherPlugin)+ assert plugin.manifest.id == "aisoc.hello-plugin"+ assert plugin.manifest.plugin_type == "enricher"+ assert plugin.manifest.version == "1.0.0"+ assert "tutorial" in plugin.manifest.tags+++# ── Lifecycle hook ───────────────────────────────────────────────────────────+++def ctx() -> PluginContext:+ return PluginContext(+ api_base_url="http://localhost:8000",+ api_token="test-token",+ config={},+ )+++async def test_on_load_defaults_to_sha256(ctx: PluginContext) -> None:+ plugin = load_plugin_from_directory(HELLO_PLUGIN_DIR)++ await plugin.on_load(ctx)++ # The tutorial promises the default algorithm is sha256.+ assert getattr(plugin, "_algorithm") == "sha256"+++async def test_on_load_accepts_configured_algorithm() -> None:+ plugin = load_plugin_from_directory(HELLO_PLUGIN_DIR)+ ctx = PluginContext(+ api_base_url="http://localhost:8000",+ api_token="test-token",+ config={"algorithm": "sha512"},+ )++ await plugin.on_load(ctx)++ assert getattr(plugin, "_algorithm") == "sha512"+++async def test_on_load_rejects_unknown_algorithm() -> None:+ plugin = load_plugin_from_directory(HELLO_PLUGIN_DIR)+ ctx = PluginContext(+ api_base_url="http://localhost:8000",+ api_token="test-token",+ config={"algorithm": "definitely-not-a-real-hash"},+ )++ with pytest.raises(ValueError, match="Unsupported hash algorithm"):+ await plugin.on_load(ctx)+++# ── Enrichment is deterministic ──────────────────────────────────────────────+++async def test_enrich_is_deterministic(ctx: PluginContext) -> None:+ plugin = load_plugin_from_directory(HELLO_PLUGIN_DIR)+ await plugin.on_load(ctx)++ request = EnrichmentRequest(+ indicator_type="ip",+ indicator_value="203.0.113.42",+ )++ expected_digest = hashlib.sha256(b"203.0.113.42").hexdigest()++ result_a = await plugin.enrich(request, ctx)+ result_b = await plugin.enrich(request, ctx)++ assert result_a.indicator_value == "203.0.113.42"+ assert result_a.enrichments["hello_plugin.algorithm"] == "sha256"+ assert result_a.enrichments["hello_plugin.digest"] == expected_digest+ assert result_a.enrichments["hello_plugin.length"] == len(expected_digest)+ assert "hello-plugin" in result_a.tags+ assert result_a.malicious is None # tutorial enricher has no opinion+ assert result_a.confidence is None+ # Same input must always yield the same enrichment — this is the+ # property the docs page leans on for snapshot examples.+ assert result_a.model_dump() == result_b.model_dump()+++async def test_enrich_uses_configured_algorithm() -> None:+ plugin = load_plugin_from_directory(HELLO_PLUGIN_DIR)+ ctx = PluginContext(+ api_base_url="http://localhost:8000",+ api_token="test-token",+ config={"algorithm": "sha512"},+ )+ await plugin.on_load(ctx)++ request = EnrichmentRequest(indicator_type="domain", indicator_value="example.com")+ result = await plugin.enrich(request, ctx)++ assert result.enrichments["hello_plugin.algorithm"] == "sha512"+ assert result.enrichments["hello_plugin.digest"] == hashlib.sha512(+ b"example.com"+ ).hexdigest()+++# ── Registry integration ─────────────────────────────────────────────────────+++async def test_hello_plugin_registers_as_enricher(ctx: PluginContext) -> None:+ plugin = load_plugin_from_directory(HELLO_PLUGIN_DIR)+ registry = PluginRegistry()+ registry.register(plugin)++ await registry.load_all(ctx)++ assert len(registry) == 1+ enrichers = registry.enrichers()+ assert len(enrichers) == 1+ assert enrichers[0].manifest.id == "aisoc.hello-plugin"+ # Lookup by ID works the same way the runtime resolves enrichers.+ assert registry.get("aisoc.hello-plugin") is plugindiff --git a/plugins/community/_examples/hello-plugin/README.md b/plugins/community/_examples/hello-plugin/README.mdnew file mode 100644index 000000000..dab8210d1--- /dev/null+++ b/plugins/community/_examples/hello-plugin/README.md@@ -0,0 +1,46 @@+# Hello Plugin (Tutorial)++Reference implementation for [apps/docs/docs/plugins/hello-plugin.md](../../../../apps/docs/docs/plugins/hello-plugin.md).++A deterministic, offline AiSOC enricher that hashes indicator values with+SHA-256 and returns the digest as enrichment metadata. No network calls, no+credentials, no external dependencies beyond `aisoc-plugin-sdk` itself.++## Why this lives under `_examples/`++Anything under `plugins/community/_examples/` is documentation only:++- It is **not** picked up by `scripts/build_marketplace.py`.+- It does **not** appear in the in-app marketplace.+- It is **not** loaded by the plugin runtime in any environment.++The directory exists so contributors have a runnable, reviewable starting+point that can never accidentally ship to a real tenant. When you're ready+to graduate, copy the directory to `plugins/community/<your-id>/` and the+marketplace builder will pick it up on the next `pnpm marketplace:build`.++## Files++| Path | Purpose |+| --------------------- | --------------------------------------------------------- |+| `aisoc-plugin.yaml` | Plugin manifest (id, name, version, plugin_type). |+| `plugin.py` | Plugin entry point with `HelloPlugin` + `create_plugin()`. |+| `README.md` | This file. |++## Run the smoke test++```bash+cd packages/plugin-sdk-py+.venv/bin/python -m pytest tests/test_hello_plugin_example.py -v+```++The smoke test pins:++1. The example is **not** importable as a real plugin (kept under `_examples/`).+2. `load_plugin_from_directory()` accepts the manifest + `create_plugin()` shape.+3. The enrichment is deterministic for the same input.+4. The `on_load()` hook validates the algorithm config.+5. The `PluginRegistry` correctly registers it as an enricher.++If you change the manifest, the algorithm, or the enrichment shape, the+test will fail and the docs page will need a matching update.diff --git a/plugins/community/_examples/hello-plugin/aisoc-plugin.yaml b/plugins/community/_examples/hello-plugin/aisoc-plugin.yamlnew file mode 100644index 000000000..5fb748a57--- /dev/null+++ b/plugins/community/_examples/hello-plugin/aisoc-plugin.yaml@@ -0,0 +1,14 @@+id: aisoc.hello-plugin+name: Hello Plugin (Tutorial)+version: 1.0.0+plugin_type: enricher+description: >+ Tutorial enricher that hashes the indicator value with SHA-256 locally.+ Reference implementation for apps/docs/docs/plugins/hello-plugin.md.+ Deliberately offline so it can run in air-gapped environments and CI+ without any external API calls.+author: AiSOC Tutorial+tags:+ - tutorial+ - enricher+ - offlinediff --git a/plugins/community/_examples/hello-plugin/plugin.py b/plugins/community/_examples/hello-plugin/plugin.pynew file mode 100644index 000000000..57698a76f--- /dev/null+++ b/plugins/community/_examples/hello-plugin/plugin.py@@ -0,0 +1,102 @@+"""Reference implementation for the "Hello, plugin" tutorial.++Walks through every step of writing an AiSOC enricher with the Python plugin+SDK. The full walkthrough lives at apps/docs/docs/plugins/hello-plugin.md —+this file is the runnable companion.++The enricher is intentionally trivial: it computes a deterministic SHA-256+hash of the indicator value and returns it as enrichment metadata. No+network calls, no external APIs, no credentials. That keeps the example:++- usable in air-gapped environments,+- reproducible in CI without secrets,+- focused on the *contract* rather than on third-party authentication.++When you're ready to write a real enricher, copy this file, replace the+hashing call with whatever vendor SDK you need, and keep the manifest ++``create_plugin`` factory shape exactly as-is.+"""++from __future__ import annotations++import hashlib++from aisoc_plugin_sdk import (+ AiSOCPlugin,+ EnricherPlugin,+ EnrichmentRequest,+ EnrichmentResult,+ PluginContext,+ PluginManifest,+)+++class HelloPlugin(EnricherPlugin):+ """Deterministic, offline enricher used by the hello-plugin tutorial."""++ @property+ def manifest(self) -> PluginManifest:+ return PluginManifest(+ id="aisoc.hello-plugin",+ name="Hello Plugin (Tutorial)",+ version="1.0.0",+ description=(+ "Tutorial enricher that hashes indicator values locally. "+ "Reference implementation for "+ "apps/docs/docs/plugins/hello-plugin.md."+ ),+ author="AiSOC Tutorial",+ tags=["tutorial", "enricher", "offline"],+ plugin_type="enricher",+ )++ async def on_load(self, ctx: PluginContext) -> None:+ # The tutorial uses on_load() to demonstrate the lifecycle hook.+ # A real enricher might open a long-lived HTTP client, prime a cache,+ # or validate that required config keys are present. We just stash the+ # configured hash algorithm so we can reuse it from enrich().+ self._algorithm = (ctx.config.get("algorithm") or "sha256").lower()+ if self._algorithm not in hashlib.algorithms_guaranteed:+ raise ValueError(+ f"Unsupported hash algorithm: {self._algorithm!r}. "+ f"Pick one of: {sorted(hashlib.algorithms_guaranteed)}"+ )++ async def enrich(+ self, request: EnrichmentRequest, ctx: PluginContext+ ) -> EnrichmentResult:+ # Hash the indicator value with the configured algorithm. This is+ # deterministic and offline, so the same input always yields the same+ # enrichment — perfect for a tutorial and for snapshot tests.+ algorithm = getattr(self, "_algorithm", "sha256")+ digest = hashlib.new(algorithm, request.indicator_value.encode("utf-8")).hexdigest()++ return EnrichmentResult(+ indicator_type=request.indicator_type,+ indicator_value=request.indicator_value,+ enrichments={+ "hello_plugin.algorithm": algorithm,+ "hello_plugin.digest": digest,+ "hello_plugin.length": len(digest),+ },+ tags=["hello-plugin"],+ # The tutorial enricher never has an opinion on maliciousness —+ # leaving this as None is the honest answer and prevents the+ # alert UI from drawing a red badge based on a meaningless signal.+ malicious=None,+ confidence=None,+ raw={"input": request.indicator_value, "digest": digest},+ )+++def create_plugin() -> AiSOCPlugin:+ """Factory called by ``load_plugin_from_directory``.++ The loader looks for a top-level ``create_plugin`` callable in+ ``plugin.py`` and uses whatever it returns as the plugin instance.+ Returning a fresh instance per call keeps every load isolated, which+ matters when the same process loads the same plugin under multiple+ tenants.+ """++ return HelloPlugin()