Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

ci(security): require the three scanner checks that genuinely report (#1064)

Beenu Arorafb83e66c6e9c
diff --git a/.github/required-checks.json b/.github/required-checks.json
index 443908c22..177507161 100644
--- a/.github/required-checks.json
+++ b/.github/required-checks.json
@@ -18,13 +18,38 @@
"commit that lands, because runs before that point were correct about a tree",
"that had not yet been fixed. A window holding no runs is tolerated for",
"seven days and then fails, so parking this date in the future cannot",
- "silence the gate for long."
+ "silence the gate for long.",
+ "The Security workflow's scanner jobs were hard gates that blocked nothing:",
+ "Trivy fails outright on a fixable HIGH/CRITICAL and had been red on main for",
+ "nine consecutive runs with nothing to stop a merge, because only",
+ "security-audit was required. Three of the six are promoted here. All six",
+ "were checked for the defect that makes a required check worthless rather",
+ "than assumed safe: security.yml declares no paths filter and no job-level",
+ "if:, and across 40 pull_request runs every one of the six reported a real",
+ "conclusion and none reported skipped - GitHub counts a skipped check as",
+ "passing, so requiring one that skips on pull requests would recreate the",
+ "defect this repository has now fixed twice.",
+ "Two are deliberately NOT required, and the reason is the same shape one",
+ "level down. 'Semgrep (ratcheted)' and 'IaC scan (checkov, ratcheted)' each",
+ "carry a continue-on-error step whose outcome no later step reads, so the",
+ "job can report success over a step that failed and the Actions API cannot",
+ "see it. check_required_check_substance.py --static refuses both by name;",
+ "adding either here fails the gate rather than strengthening anything. The",
+ "remedy it prints is to give the step an id and add a blocking step reading",
+ "steps.<id>.outcome == 'failure', or to drop continue-on-error.",
+ "Semgrep is additionally non-deterministic on an unchanged tree - the",
+ "measurement is recorded in .security/allowlist.yml, where four pull",
+ "requests on one base within thirteen minutes split 102/40 and 101/39 over",
+ "the identical 387 rules and 2267 files - so requiring it would red pull",
+ "requests that cannot have changed a finding and teach re-running until",
+ "green, which is how a required check stops meaning anything."
],
"branch": "main",
"enforced_from": "2026-09-29T12:00:00Z",
"contexts": [
"Backup → destroy → restore (Postgres via S3)",
"Claim-to-gate matrix (ratchet)",
+ "IaC scan (tfsec, ratcheted)",
"No competitor product named",
"No required check can skip its own assertions",
"No tool attribution",
@@ -42,6 +67,8 @@
"Python — Service unit tests wave 2 (threatintel)",
"Python — Service unit tests wave 2 (ueba)",
"Python — Tests",
+ "Secret scan (gitleaks)",
+ "Trivy filesystem",
"Web — Build",
"Web — Lint & Type-check",
"Zero open CodeQL alerts",
diff --git a/docs/audit/CLAIM_TO_GATE_MATRIX.md b/docs/audit/CLAIM_TO_GATE_MATRIX.md
index 13517763c..a48eaa243 100644
--- a/docs/audit/CLAIM_TO_GATE_MATRIX.md
+++ b/docs/audit/CLAIM_TO_GATE_MATRIX.md
@@ -42,7 +42,7 @@ Statuses: `GATED` (a CI job fails when the claim stops being true) · `PARTIAL`
| SAST | README badge (CodeQL) | `codeql.yml` (the scan) + `codeql-alert-gate.yml` (the invariant: `scripts/check_codeql_alerts.py` fails on any open CodeQL alert at any severity, `note` included, and refuses a ref whose declared languages were not analysed at the triggering commit — `analyze` never failed a build on findings, so this row named a scan and not a gate) | GATED | - |
| Dependency CVE scanning | (implied by security) | `security-audit.yml` | GATED | - |
| OpenSSF Scorecard | README badge L13 | `scorecard.yml` | GATED | - |
-| Container image / IaC / secret scanning (Trivy/checkov/tfsec/gitleaks/Semgrep) | (implied by security) | `security.yml` (claim-matrix ratchet, gitleaks and Trivy fail outright; Semgrep, checkov and tfsec fail on a changed finding count via `check_scanner_ratchet.py` against the ceilings in `.security/allowlist.yml`) + `ci.yml :: Python — Lint & Type-check` (the same gate's allow-list-schema and scanner-coverage half, which needs no scanner) | GATED (this row previously said the three observe scanners "report-and-ratchet" against `.security/allowlist.yml`, and neither half was true: nothing compared the printed counts to anything, and the allow-list had **no reader anywhere in the tree**. The ratchet now fails on more findings than the ceiling, on fewer (shrink-only, printing the measured number), on zero against a non-zero ceiling — a scanner that did not run — on a scanner version that is not the one the ceiling was measured with — for all three, which this row claimed before it was true: the arm read `if version and measured_with and ...` and tfsec's JSON carries no version field, so its counter returned `None` and the `measured_with` in the allow-list was never compared against anything. A version the gate cannot establish is now a failure rather than a skip, tfsec's is read from the binary's own `--version` (a pinned URL says what was asked for; a cached binary would still satisfy it), and a report version and a `--tool-version` that disagree fail in both directions. `_versions_pinned` also could not see a curled binary's pin, which silently exempted tfsec from the test that keeps pin and ceiling in step — on an expired or unreasoned suppression, and on a ceiling/scanner set mismatch in either direction. All three scanners are pinned exactly, because for a ratchet the scanner version is the behaviour. GitHub push-protection remains the always-on hard secret gate) | — |
+| Container image / IaC / secret scanning (Trivy/checkov/tfsec/gitleaks/Semgrep) | (implied by security) | `security.yml` (claim-matrix ratchet, gitleaks and Trivy fail outright; Semgrep, checkov and tfsec fail on a changed finding count via `check_scanner_ratchet.py` against the ceilings in `.security/allowlist.yml`) + `ci.yml :: Python — Lint & Type-check` (the same gate's allow-list-schema and scanner-coverage half, which needs no scanner). **Three of the six are required checks on `main`** — `Secret scan (gitleaks)`, `Trivy filesystem` and `IaC scan (tfsec, ratcheted)` — because a hard gate that is not a required context blocks nothing: Trivy fails outright on a fixable HIGH/CRITICAL and had been red on `main` for nine consecutive runs with nothing to stop a merge. All six were checked for the defect that makes a required check worthless rather than assumed safe — `security.yml` declares no `paths:` filter and no job-level `if:`, and across 40 `pull_request` runs each of the six reported a real conclusion and **none reported `skipped`**, which GitHub counts as passing. **Two are deliberately not required**: `Semgrep (ratcheted)` and `IaC scan (checkov, ratcheted)` each carry a `continue-on-error` step whose outcome no later step reads, so the job can report success over a failed step and the Actions API cannot see it — `check_required_check_substance.py --static` refuses both by name, so adding either would fail the grading-integrity gate rather than strengthen anything. Semgrep is additionally non-deterministic on an unchanged tree (four pull requests on one base within thirteen minutes split 102/40 and 101/39 over the identical 387 rules and 2,267 files, recorded in `.security/allowlist.yml`), so requiring it would red pull requests that cannot have changed a finding | GATED (this row previously said the three observe scanners "report-and-ratchet" against `.security/allowlist.yml`, and neither half was true: nothing compared the printed counts to anything, and the allow-list had **no reader anywhere in the tree**. The ratchet now fails on more findings than the ceiling, on fewer (shrink-only, printing the measured number), on zero against a non-zero ceiling — a scanner that did not run — on a scanner version that is not the one the ceiling was measured with — for all three, which this row claimed before it was true: the arm read `if version and measured_with and ...` and tfsec's JSON carries no version field, so its counter returned `None` and the `measured_with` in the allow-list was never compared against anything. A version the gate cannot establish is now a failure rather than a skip, tfsec's is read from the binary's own `--version` (a pinned URL says what was asked for; a cached binary would still satisfy it), and a report version and a `--tool-version` that disagree fail in both directions. `_versions_pinned` also could not see a curled binary's pin, which silently exempted tfsec from the test that keeps pin and ceiling in step — on an expired or unreasoned suppression, and on a ceiling/scanner set mismatch in either direction. All three scanners are pinned exactly, because for a ratchet the scanner version is the behaviour. GitHub push-protection remains the always-on hard secret gate) | — |
| Signed / attested release artifacts | (implied by "run next to crown jewels") | `release.yml` + `publish-images.yml` (cosign keyless signatures, CycloneDX SBOM attestation, BuildKit SLSA provenance + SPDX SBOM per image; SHA-pinned actions), `build-extensions.yml` (extension cosign) | GATED | - |
| Insecure defaults hard-fail in production | (implied by self-host) | `ci.yml` api tests (`test_security_defaults.py::test_enforce_*`; `enforce_secure_defaults` raises at boot in production) | GATED | - |
| OpenAPI stability for 3 SDKs + MCP | (implied by SDKs) | `check-openapi.yml` (drift) + `openapi-breaking.yml` (`scripts/openapi_diff.py` — PR spec vs base fails on removed endpoint/schema/field, type change, tightened request, or dropped enum value). A deliberate break goes through the `breaking-change-approved` PR label, which still runs the detector and records every break it permits plus the CHANGELOG `### BREAKING` note that justified it; `--allow-breaking` requires that note as an argument, so the bypass cannot be wired up without the record. Four of the tests assert the *workflow* reaches the flag — it previously had no caller at all | GATED | - |