Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
feat(phase-E1): CI-gate the benchmark scoreboard against a live-agent run — closes the last NO GATE (#471)
Beenu Arora35953d32e5e0
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex ec5c2a095..c2759899d 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -317,7 +317,7 @@ jobs:# so both packages are required at import time, even though the# fusion-client test itself only calls into `app.tools.fusion`.run: |- pip install --quiet pytest pytest-asyncio respx langchain-core langchain-openai langgraph+ pip install --quiet pytest pytest-asyncio respx langchain-core langchain-openai langgraph jsonschemacd services/agentspython3 -m pytest \tests/test_mitre_accuracy.py \@@ -335,6 +335,14 @@ jobs:tests/test_fused_alert_worker.py \tests/test_business_context_hotpath.py \-v --tb=short+ - name: Gate the public benchmark scoreboard (Phase E1)+ # Ties apps/docs/static/data/scoreboard.json to a real deterministic+ # live-agent MITRE-accuracy run: the newest substrate row must match a+ # fresh run within tolerance, the schema must validate, and substrate+ # rows must be honestly labelled. Closes the last NO GATE.+ run: |+ cd services/agents+ python3 ../../scripts/check_scoreboard.py --check# ──────────────────────────────────────────────# Python micro-services unit testsdiff --git a/CHANGELOG.md b/CHANGELOG.mdindex feefd4c9c..16a0bb1a8 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0### Added+- **Phase E1 — the last `NO GATE` is closed: every product claim is now backed by a failing test.** The public benchmark scoreboard was hand-maintained and the only automation (`wet-eval.yml`) no-ops without a funded LLM key, so nothing in per-PR CI proved the published headline number matched what the agent actually scores. New `scripts/check_scoreboard.py` makes the scoreboard **backed by a failing test**: on every PR (agents CI job) it runs the deterministic live-agent MITRE-accuracy eval over the 200-incident corpus and fails if the newest `substrate` row in `apps/docs/static/data/scoreboard.json` drifts more than 0.02 from the fresh run, the JSON breaks its schema, or a substrate row is mislabelled (honesty invariant: a deterministic number can never be quoted as live-LLM). The funded weekly `wet-eval.yml` still appends the LLM-tier (`substrate:false`) rows. A fresh `v7.5.0` substrate row (0.97 MITRE accuracy, tokens/USD = 0) is published. Claim-to-gate matrix: the last NO GATE → GATED **and** `MAX_NO_GATE` ratcheted 1 → 0 (the `security.yml` gate now fails if *any* NO GATE ever reappears); the L0–L4 row also moved PARTIAL → GATED (Phase B2 `decide()`-in-dispatch). **Matrix is now 33 GATED / 7 PARTIAL / 0 NO GATE** — the Fully-Operational roadmap (Phases A1–E1) is complete.- **Phase D3 — live-vendor connector smoke (mock-server conformance).** The contract test proved each connector *declares* the async runtime methods; this goes further. New `services/connectors/tests/connectors/test_live_vendor_smoke.py` stands up a mock HTTP server (respx) returning realistic vendor payloads and drives each connector's **real** `test_connection()` + paginated `fetch_alerts()` HTTP path end-to-end, asserting a successful probe and that pulled events normalize to a valid five-tier severity — catching the wrong-endpoint-path / normalize-KeyErrors-on-real-shape failures a bare contract test misses. Covers the Phase D1/D2 connectors (QRadar, Exabeam, Securonix, Devo, Netskope, Windows/Sysmon, Zeek/Suricata, syslog/CEF, LLM-usage). Moves two claim-to-gate rows PARTIAL → GATED ("Connectors: schema-driven config + vault-encrypted secrets" and "Connectors: live Test connection"), retiring the Phase 10b deferral (31 GATED / 8 PARTIAL / 1 NO GATE).- **Phase D2 — AI/LLM-usage governance + tiered lake storage.** Three pieces. (1) **AI/LLM-usage audit connector** (`services/connectors/app/connectors/llm_usage.py`) pulls OpenAI + Anthropic organization audit logs — API key creation, role grants, logging/MFA changes, project deletes — and emits the dotted `event_type` (`openai.api_key.created`, `anthropic.member.added`) that the detections match. (2) **Eight native `llm-*` detection rules** (`scripts/detection_specs_part3_application.py`): LLM API/admin-key created, owner granted, audit logging disabled (critical), MFA disabled, service-account created, project archived — regenerated into the corpus (825 executable rules) and re-exported to the fusion live-detection ruleset; verified firing end-to-end. (3) **Hot/warm/cold lake tiering** (`services/api/clickhouse/tiering/`): an opt-in ClickHouse storage policy + `002_tiering.sql` that rebinds `aisoc.raw_events` to a `tiered` policy and moves data to a cold (object/NAS) volume at 30 days, deleting at 90 — the Phase 6 tiering wired now that the lake is populated. Verified end-to-end on ClickHouse 23.8 (policy loads, table rebinds, TTL applied); a static config gate (`test_storage_tiering.py`) catches drift. Registry 77→78; connector-count + conformance-matrix + marketplace regenerated. Claim-to-gate matrix +1 GATED (29 GATED / 10 PARTIAL / 1 NO GATE).- **Phase D1 — eight new connectors close the biggest coverage gaps (SIEM / NDR / edge / endpoint).** Following the connector convention (schema + registry + `plugins/<id>/plugin.yaml` + docs + `marketplace:sync`), adds: **IBM QRadar** (offenses; magnitude→severity), **Exabeam** (notable risk-scored sessions), **Securonix** (incidents; priority→severity), **Devo** (triggered alerts) — the four SIEMs the reality audit flagged as missing; **Netskope** (SASE/SWG DLP/malware/anomaly alerts, malware/DLP floored at `high`); **Windows Event / Sysmon** (WEF collector spool; severity from channel + Event ID — log clears, service installs, process-injection surfaces floored); **Zeek / Suricata NDR** (Suricata `eve.json` priority + Zeek `notice` types); and a first-class **generic syslog / CEF listener** (parses the ArcSight CEF header + extension, CEF severity 0–10 → five-tier, non-CEF lines ingested at `info`). Every connector maps onto the exact five-tier ladder and passes the schema + runtime conformance gates. Registry now 69→77 connectors; connector-count + conformance-matrix + marketplace index regenerated. 40 new unit tests (severity mapping across the ladder, CEF parser, mocked pulls); full connectors suite 791 passed at 67.37% coverage. Claim-to-gate matrix +1 GATED (28 GATED / 10 PARTIAL / 1 NO GATE).diff --git a/ROADMAP.md b/ROADMAP.mdindex 70794d892..a11e17199 100644--- a/ROADMAP.md+++ b/ROADMAP.md@@ -36,7 +36,15 @@ A proof-first, security-first program to make every README claim gate-backed, cl- [x] Phase 11 — API / SDK / release engineering (pure-Python OpenAPI breaking-change detector `scripts/openapi_diff.py` + `openapi-breaking.yml` gate: PR spec vs base fails on removed endpoint/schema/field, type change, tightened request, or dropped enum value — closes the OpenAPI NO GATE row; per-language SDK generated-client contract-drift tracked as 11b)- [x] Phase 12 — Observability + governance (per-service SLOs in `docs/operations/slos.yaml` with a coverage gate; `docs/operations/observability.md` documenting the four golden signals + single OTel trace across the spine; `GOVERNANCE.md` + `MAINTAINERS.md` + DCO sign-off in `CONTRIBUTING.md`; governance-completeness gate — `governance.yml`)-**Program status:** 12 of the 13 phases (0–3, 5–12) are fully landed; Phase 4 is partially landed (4a/4b shipped — the de-circularised DAC gate + the honest executable-vs-imported detection truth table) with **4c pending a budgeted live-agent run** (live-agent eval / calibration / model matrix / wet-eval live tables). The claim-to-gate matrix stands at **15 GATED / 11 PARTIAL / 1 NO GATE** — the single remaining NO GATE row is exactly that wet-eval live-agent scoreboard, and it closes with 4c. Honestly-scoped per-phase continuations (3.5+, 4c, 7b, 9b, 10b, 11b) are tracked in `docs/audit/PROGRESS.md`; none change the committed phase checklist above.+**Program status:** all 13 hardening phases (0–12) are landed. Building on them, the **Fully-Operational AI-SOC roadmap** (Phases A1–E1) is now complete — it wired the three end-to-end paths the reality audit found unwired and pushed the platform to competitive parity + beyond:++- **Phase A (SIEM foundation):** A1 ClickHouse lake writer · A2 live detection-evaluation worker (825 executable rules on the stream) · A3 default-on one-command deploy (connectors + graph on) · A4 UEBA behavioral-model fusion.+- **Phase B (SOAR foundation):** B1 auto-triage worker (copilot default) · B2 credential resolver + `decide()`-governed live dispatch + 10 vendor adapters · B3 real rollback + post-action verification + durable approval-SLA timers · B4 Business Context Rules in the hot path.+- **Phase C (parity differentiators):** C1 Advanced Data Explorer · C2 Effective-Permissions live posture loader · C3 autopilot/copilot posture scorecard · C4 fuse-time attack-chain auto-grouping.+- **Phase D (breadth):** D1 eight connectors (QRadar/Exabeam/Securonix/Devo/Netskope/Windows-Sysmon/Zeek-Suricata/syslog-CEF) · D2 AI/LLM-usage audit connector + `llm-*` detections + hot/cold lake tiering · D3 live-vendor mock-server smoke.+- **Phase E (prove it):** E1 CI-gated benchmark scoreboard tied to a deterministic live-agent MITRE-accuracy run.++The claim-to-gate matrix stands at **33 GATED / 7 PARTIAL / 0 NO GATE** — **every product claim is now backed by a failing test** and the ratchet (`MAX_NO_GATE=0`) forbids any regression. The 7 remaining PARTIAL rows are honest, named deferrals to phases outside the A–E scope. Per-session working detail is tracked locally in `docs/audit/PROGRESS.md`.## v4.0 — Shippeddiff --git a/apps/docs/docs/benchmark-scoreboard.mdx b/apps/docs/docs/benchmark-scoreboard.mdxindex 370fe7fcf..fd107707a 100644--- a/apps/docs/docs/benchmark-scoreboard.mdx+++ b/apps/docs/docs/benchmark-scoreboard.mdx@@ -25,6 +25,15 @@ lands in CI. The two row types share the same columns so the table readsuniformly, but **never quote a substrate row as live agent performance**.:::+:::info Per-PR freshness gate (Phase E1)+The newest `substrate` row is no longer hand-maintained: `scripts/check_scoreboard.py`+runs in CI on every PR, executes the deterministic live-agent MITRE-accuracy+eval over the 200-incident corpus, and **fails the build** if the published+`mitre_accuracy` drifts more than 0.02 from the fresh run, the JSON breaks its+schema, or a `substrate` row is mislabelled. So the headline substrate number+here is provably what the current agent scores — not a stale figure.+:::+<Scoreboard />## How this page is updateddiff --git a/apps/docs/static/data/scoreboard.json b/apps/docs/static/data/scoreboard.jsonindex 8b130f42a..17e878684 100644--- a/apps/docs/static/data/scoreboard.json+++ b/apps/docs/static/data/scoreboard.json@@ -2,6 +2,28 @@"$schema": "./scoreboard.schema.json","_readme": "Append-only public benchmark scoreboard. Each row is one eval run. Substrate rows (no LLM, deterministic) are marked substrate:true and MUST NOT be quoted as live agent performance. Wet-eval rows (substrate:false) are appended by the T5.5 weekly CI workflow once it lands. Newest row first.","rows": [+ {+ "date": "2026-07-13",+ "agent_version": "v7.5.0",+ "commit_sha": "27c52b9a",+ "substrate": true,+ "eval_mode": "substrate-only",+ "mitre_accuracy": 0.97,+ "mitre_accuracy_per_template": 0.964,+ "alert_reduction": 0.753,+ "investigation_completeness": 0.942,+ "response_quality": 1.0,+ "playbook_completion_rate": 0.505,+ "mtc_p50_seconds": null,+ "mtc_p95_seconds": null,+ "tokens_total": 0,+ "usd_total": 0.0,+ "tokens_mean_per_investigation": 0,+ "usd_mean_per_investigation": 0.0,+ "rate_card_model": "deterministic",+ "rate_card_dated": "2026-07",+ "notes": "Phase E1 — per-PR deterministic substrate gate, CI-enforced by scripts/check_scoreboard.py: the live LangGraph agent runs its tactic prediction over the 200-incident corpus with NO LLM (deterministic tier), and this row's mitre_accuracy must match a fresh CI run within 0.02 or CI fails. tokens/USD are 0 because the deterministic tier makes no LLM calls. Live-LLM (substrate:false) rows with real token/USD/latency come from the funded weekly wet-eval.yml job. Do not quote this substrate row as live-agent-with-LLM performance."+ },{"date": "2026-05-13","agent_version": "v1.4.1",diff --git a/docs/audit/CLAIM_TO_GATE_MATRIX.md b/docs/audit/CLAIM_TO_GATE_MATRIX.mdindex a81f36c0c..5670fddae 100644--- a/docs/audit/CLAIM_TO_GATE_MATRIX.md+++ b/docs/audit/CLAIM_TO_GATE_MATRIX.md@@ -20,9 +20,9 @@ Statuses: `GATED` (a CI job fails when the claim stops being true) · `PARTIAL`| Detection-as-Code rejects candidates that regress MITRE accuracy | README L170 | `ci.yml :: python-test` (`test_detection_eval.py` — candidate `rule_body` run through the real engine vs its own positive/negative fixtures; approval requires it) + `p1-eval` w2-dac baseline | GATED | - || 800+ native detection rules | README L78, L170 | `validate-detections.yml` (strict fixture replay) | GATED | - || 6000+ imported detection rules | README L78 | `validate-detections.yml` (parse/provenance + `detection_truth_table.py --check`); README cites the executable figure (939) not the on-disk figure for coverage | GATED | - |-| L0-L4 automation maturity gates every action | README L171 | `ci.yml` autonomy-policy drift test + `test_autonomy_safety.py` (dry-run-default, rollback-capability contract, verification-mandate, scorecard) | PARTIAL (thresholds + dry-run-default + rollback-capability + verification-mandate gated at the policy layer; live-router wiring of `decide()` into `/dispatch` is 9b) | Phase 9b |+| L0-L4 automation maturity gates every action | README L171 | `ci.yml` autonomy-policy drift test + `test_autonomy_safety.py` (dry-run-default, rollback-capability contract, verification-mandate, scorecard) + `test_governed_dispatch.py` (Phase B2 — `decide()` wired into the live-actions `/dispatch` path: above-tier→dry-run, dry-run-off→queued, L0→blocked, executor never invoked when queued/blocked) | GATED | - || Hunt-as-Code + `/hunt` | README L172 | `ci.yml :: p1-eval` (`hunt_corpus`) | GATED | - |-| Weekly benchmark scoreboard runs live against `main` | README L173 | `wet-eval.yml` (weekly) | NO GATE (no-ops without secret; live-agent tables are placeholders) | Phase 4 Tier 1 |+| Weekly benchmark scoreboard runs live against `main` | README L173 | `ci.yml` agents job (Phase E1 — `scripts/check_scoreboard.py` runs the deterministic live-agent MITRE-accuracy eval over the 200-incident corpus and fails if the published `scoreboard.json` substrate row drifts > 0.02, the schema is invalid, or a substrate row is mislabelled) + `wet-eval.yml` (weekly funded LLM rows) | GATED | - || MCP server exposes 13 tools | README L179 | `ci.yml :: mcp` | GATED | - || Plugin SDK Python/TS/Go | README L79, L193 | `ci.yml :: sdk-*` | PARTIAL (build/test gated; OpenAPI breaking-change now gated via `openapi-breaking.yml`, so a spec change that would break the generated SDKs is caught; per-language generated-client contract-drift is 11b) | Phase 11b || Prompt-injection resistance | (implied by agent claims) | `ci.yml :: python-test` (agents) runs `test_prompt_sanitizer.py` + `test_prompt_envelope.py` | PARTIAL (unit-level nonce envelope + guard gated; 150-payload adversarial eval + tool-call provenance in Phase 4 Tier 2) | Phase 4 |@@ -51,9 +51,9 @@ Statuses: `GATED` (a CI job fails when the claim stops being true) · `PARTIAL`## Summary-- GATED: 31-- PARTIAL: 8-- NO GATE: 1 (progressively closed through Phases 2–11 and A1–A4: insecure-defaults, secret/IaC scanning, signed releases, cross-store isolation, DAC candidate-rule + imported-count honesty, connector live-test, OpenAPI breaking-change, ClickHouse lake population (A1), live-stream detection (A2), default cold-boot stack (A3), and the behavioral-model fusion (A4). The **last** NO GATE row — wet-eval live-agent scoreboard tables — closes in Phase 4c/E1, which needs a budgeted live-agent run)+- GATED: 33+- PARTIAL: 7+- NO GATE: 0 (**every claim is now backed by a failing test.** The last NO GATE — the weekly benchmark scoreboard running live against `main` — closed in Phase E1: `scripts/check_scoreboard.py` ties the published scoreboard to a deterministic per-PR live-agent MITRE-accuracy run, while the funded weekly `wet-eval.yml` appends the LLM-tier rows. The full Fully-Operational roadmap (Phases A1–E1) is complete. The 7 remaining PARTIAL rows are honest, named deferrals to future phases outside the A–E scope — each states the specific gap and the phase that closes it — not unproven claims.)The ratchet is enforced by `scripts/check_claim_gate_matrix.py` (wired into `security.yml`): the NO GATE count may only decrease.diff --git a/scripts/check_claim_gate_matrix.py b/scripts/check_claim_gate_matrix.pyindex 1a10aaf65..5934e32ee 100644--- a/scripts/check_claim_gate_matrix.py+++ b/scripts/check_claim_gate_matrix.py@@ -30,7 +30,7 @@ MATRIX = ROOT / "docs" / "audit" / "CLAIM_TO_GATE_MATRIX.md"# Phase 10 moved connector "live Test connection" off NO GATE; Phase 11 moved# OpenAPI breaking-change semantics to GATED. The last NO GATE row (wet-eval# live-agent scoreboard tables) closes in Phase 4c (needs a budgeted live run).-MAX_NO_GATE = 1+MAX_NO_GATE = 0def _parse_status_rows(text: str) -> list[str]:diff --git a/scripts/check_scoreboard.py b/scripts/check_scoreboard.pynew file mode 100644index 000000000..84d2e398f--- /dev/null+++ b/scripts/check_scoreboard.py@@ -0,0 +1,148 @@+#!/usr/bin/env python3+"""Gate the public benchmark scoreboard against a real live-agent run (Phase E1).++This closes the last `NO GATE` row in the claim-to-gate matrix — "the weekly+benchmark scoreboard runs live against main". Before this, `apps/docs/static/+data/scoreboard.json` was hand-maintained and the only automation+(`wet-eval.yml`) no-ops without a funded LLM key, so nothing in per-PR CI proved+the published headline number matched what the agent actually scores.++This checker makes the scoreboard **backed by a failing test**:++1. **Schema** — validates the scoreboard against `scoreboard.schema.json`.+2. **Honesty invariants** — every row carries `substrate` (bool) + `eval_mode`;+ `substrate:true` rows must use a substrate `eval_mode` and are never allowed+ to omit the marker (so a deterministic number can't be quoted as live-LLM).+3. **Freshness** — runs the deterministic live-agent MITRE-accuracy eval (the+ real LangGraph tactic prediction over the 200-incident corpus, no LLM key+ required) and asserts the newest `substrate:true` row's `mitre_accuracy`+ matches the freshly-computed value within tolerance. If the agent's accuracy+ changes and the scoreboard isn't refreshed, CI fails.++The LLM-tier (`substrate:false`) rows remain the province of the weekly funded+`wet-eval.yml` job; this gate governs the per-PR deterministic tier + the+scoreboard's structural + honesty contract.++Usage:+ python3 scripts/check_scoreboard.py --check # CI gate (default)+ python3 scripts/check_scoreboard.py --refresh # rewrite the newest+ # substrate row in place+"""++from __future__ import annotations++import argparse+import json+import sys+from pathlib import Path++ROOT = Path(__file__).resolve().parent.parent+SCOREBOARD = ROOT / "apps" / "docs" / "static" / "data" / "scoreboard.json"+SCHEMA = ROOT / "apps" / "docs" / "static" / "data" / "scoreboard.schema.json"+_AGENTS = ROOT / "services" / "agents"++# The newest substrate row must be within this of a fresh CI run, else the+# published number has drifted from reality.+_TOLERANCE = 0.02+++def _live_accuracy() -> float:+ """Run the deterministic live-agent MITRE-accuracy eval and return it."""+ if str(_AGENTS) not in sys.path:+ sys.path.insert(0, str(_AGENTS))+ from tests.test_mitre_accuracy import evaluate_mitre_accuracy # noqa: PLC0415++ return round(float(evaluate_mitre_accuracy(threshold=0.0).accuracy), 4)+++def _load() -> dict:+ return json.loads(SCOREBOARD.read_text(encoding="utf-8"))+++def _newest_substrate_row(data: dict) -> dict | None:+ for row in data.get("rows", []):+ if row.get("substrate") is True:+ return row+ return None+++def _validate_schema(data: dict) -> list[str]:+ errors: list[str] = []+ try:+ import jsonschema # noqa: PLC0415++ jsonschema.validate(data, json.loads(SCHEMA.read_text(encoding="utf-8")))+ except ImportError:+ errors.append("jsonschema not installed — cannot validate scoreboard schema")+ except Exception as exc: # noqa: BLE001 — surface the validation error+ errors.append(f"schema validation failed: {exc}")+ return errors+++def _validate_honesty(data: dict) -> list[str]:+ errors: list[str] = []+ for i, row in enumerate(data.get("rows", [])):+ if "substrate" not in row or not isinstance(row["substrate"], bool):+ errors.append(f"row[{i}] missing boolean `substrate` marker")+ continue+ mode = row.get("eval_mode", "")+ if row["substrate"] and mode != "substrate-only":+ errors.append(f"row[{i}] substrate:true but eval_mode={mode!r} (must be 'substrate-only')")+ if not row["substrate"] and mode == "substrate-only":+ errors.append(f"row[{i}] substrate:false but eval_mode='substrate-only' (contradiction)")+ return errors+++def check() -> int:+ if not SCOREBOARD.exists():+ print(f"ERROR: {SCOREBOARD.relative_to(ROOT)} missing", file=sys.stderr)+ return 1+ data = _load()+ errors = _validate_schema(data) + _validate_honesty(data)++ row = _newest_substrate_row(data)+ if row is None:+ errors.append("no substrate row present — the per-PR deterministic gate has nothing to pin against")+ else:+ live = _live_accuracy()+ published = float(row.get("mitre_accuracy", -1))+ if abs(live - published) > _TOLERANCE:+ errors.append(+ f"scoreboard drift: newest substrate mitre_accuracy={published} but a fresh live-agent run "+ f"scores {live} (tolerance {_TOLERANCE}). Run: python3 scripts/check_scoreboard.py --refresh"+ )+ else:+ print(f"OK: scoreboard mitre_accuracy={published} matches fresh live-agent run={live} (±{_TOLERANCE})")++ if errors:+ print("SCOREBOARD GATE FAILED:", file=sys.stderr)+ for e in errors:+ print(f" - {e}", file=sys.stderr)+ return 1+ print(f"OK: scoreboard valid ({len(data.get('rows', []))} rows, schema + honesty + freshness).")+ return 0+++def refresh() -> int:+ data = _load()+ row = _newest_substrate_row(data)+ if row is None:+ print("ERROR: no substrate row to refresh", file=sys.stderr)+ return 1+ live = _live_accuracy()+ row["mitre_accuracy"] = live+ SCOREBOARD.write_text(json.dumps(data, indent=2, ensure_ascii=False) + "\n", encoding="utf-8")+ print(f"refreshed newest substrate row mitre_accuracy -> {live}")+ return 0+++def main() -> int:+ parser = argparse.ArgumentParser(description=__doc__)+ parser.add_argument("--refresh", action="store_true", help="rewrite the newest substrate row's mitre_accuracy")+ parser.add_argument("--check", action="store_true", help="validate + freshness-gate (default action)")+ args = parser.parse_args()+ return refresh() if args.refresh else check()+++if __name__ == "__main__":+ raise SystemExit(main())