Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

fix(action): never grade a repository clean across sources it could not read (#1017)

Beenu Aroraef6848e3f185
diff --git a/.github/workflows/aisoc-selfscan.yml b/.github/workflows/aisoc-selfscan.yml
index 3049b347d..fe06c8217 100644
--- a/.github/workflows/aisoc-selfscan.yml
+++ b/.github/workflows/aisoc-selfscan.yml
@@ -17,10 +17,23 @@ on:
permissions:
contents: read
+ # `security-events` covers code scanning ONLY. Dependabot alerts need
+ # `vulnerability-alerts`, and because naming any permission sets every
+ # unnamed one to `none`, omitting it here was not a default — it was a
+ # denial. That is why every digest recorded "Dependabot: skipped (token
+ # lacks permission…)" and then graded the repository A/100 anyway.
security-events: read
+ vulnerability-alerts: read
pull-requests: write
issues: write
+# Secret scanning is deliberately absent above and cannot be added: GitHub
+# documents that its alerts "cannot be read with this permission and require a
+# GitHub App or a personal access token", so `GITHUB_TOKEN` has no route to
+# them at all. The digest reports itself incomplete rather than grading as
+# though it had looked, which is the honest state until someone supplies an
+# app or a PAT.
+
concurrency:
group: aisoc-selfscan-${{ github.ref }}
cancel-in-progress: true
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 26f188df1..bb553ca1c 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -157,6 +157,70 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **`scripts/check_required_check_substance.py`** — a required check that reports success while its work was skipped is the third way a grading can be absent without anything going red, after a discarded push (`check_workflow_concurrency.py`) and a cancelled run (`check_main_run_cancellations.py`). It closes it from both ends, wired into `grading-integrity.yml`. Statically it enumerates every path through a required check's job and refuses one that runs strictly less assertive work than another path through the same job — so two ways of booting one stack both pass, and announcing that nothing was checked does not. Against the Actions API it requires every successful push run on `main` to have taken a complete path. Which steps count is derived by evaluating each `if:` under a simulated protected-branch push (`scripts/gh_expr.py`), not from a list of step names, so a rename cannot silently empty it. All 22 required contexts are inventoried in `.github/required-checks.json`, compared against branch protection whenever a token can read it, and the gate refuses a missing token, an empty fetch, a context whose job is not in the tree, and a window that has held no runs for seven days.
- Building it surfaced a defect in reading the Actions API that is worth recording, because any gate reading run history can hit it. **The workflow-runs endpoint is not stable across pages and can serve a stale but internally consistent snapshot.** Asking for 250 runs returned 250 rows holding 157 unique ids, with the first row of page 1 five weeks older than the newest run that existed; and on another attempt a single page came back ending six weeks short while a second read of the same endpoint agreed with it. So the window is one page, capped at 100 and refused above that rather than silently truncated; rows are deduplicated and sorted locally; and freshness is checked against the branch's **commit list** — a different endpoint — because no amount of re-reading the runs endpoint can detect that it is stale. A workflow whose push trigger carries a `paths:` filter is exempt from that oracle and is named in the output as not freshness-verified.
+- **The weekly security digest graded a repository A/100 across sources it
+ could not read, and could not tell a reader when it last ran.** Two defects
+ in `packages/aisoc-action`, both found by asking why issue #510 showed an
+ unqualified "grade A (100/100), 0 open findings" that had not moved since
+ 2026-08-24.
+
+ `fetchAlerts` degrades a 403/404 on any source into a note plus an empty
+ array. Zero alerts from a source the token cannot read is then
+ indistinguishable from zero alerts from a clean one, `postureGrade` sees an
+ empty queue and returns A/100 unconditionally, and the digest headlined that
+ as the repository's posture with the skip demoted to a blockquote beneath the
+ numbers. #510's own body records **Dependabot and secret scanning as
+ skipped** — two of its three declared sources — so the all-clear was derived
+ from code scanning alone. `fetchAlerts` now returns `scanned` and `skipped`
+ alongside the alerts, and a digest with any skipped source refuses to
+ headline a grade: it reads `incomplete (N of M sources readable)`, leads with
+ "this is not an all-clear", names which sources were unreadable, and scopes
+ the grade explicitly to the ones that answered. A fully-read clean queue
+ still publishes `grade A (100/100)` exactly as before.
+
+ The staleness was a second defect with the same root: nothing in the body was
+ tied to the run. A week where nothing changed rendered a byte-identical body,
+ GitHub's `PATCH` was a no-op, and `updated_at` froze — so **six consecutive
+ scheduled runs succeeded** (2026-08-31 through 2026-09-28) while the issue
+ read five weeks abandoned. The digest now carries a `**Generated** <UTC>`
+ stamp, so the body changes every week and a genuinely stopped generator is
+ visible on the issue itself rather than inferable only from the Actions tab.
+ `upsertDigestIssue` also logs on success with the issue number; previously
+ the run log ended at the triage headline whether the issue was written or
+ not, so "did the digest publish?" could not be answered from a green run.
+
+ Five tests pin all of it, and all five fail against the pre-fix tree.
+
+- **The digest's Dependabot blindness had a one-line root cause, and the docs
+ shipped it to every adopter.** `security-events: read` covers code scanning
+ only; Dependabot alerts need `vulnerability-alerts: read`, and GitHub's
+ workflow-syntax reference says so outright — "For Dependabot alerts, use the
+ `vulnerability-alerts` permission." The second half is what made it a denial
+ rather than a default: "If you specify the access for any of these
+ permissions, all of those that are not specified are set to `none`." So
+ `aisoc-selfscan.yml`, by naming four permissions and not that one, actively
+ denied the source it then graded as clean. It is now granted.
+
+ Secret scanning cannot be fixed the same way and is documented rather than
+ papered over: GitHub states its alerts "cannot be read with this permission
+ and require a GitHub App or a personal access token", so `GITHUB_TOKEN` has
+ no route to them at all and the digest will keep reporting itself incomplete
+ until someone supplies one.
+
+ Both copy-paste snippets in `apps/docs/docs/integrations/github-action.md`
+ recommended the same incomplete block while the action's default `sources` is
+ `dependabot,code-scanning,secret-scanning`, so anyone following the docs got
+ two of three sources silently skipped. Both are corrected and a `Permissions`
+ section states which grant each source needs and which one is unreachable.
+
+- **The digest's week-over-week delta has never rendered, and the docs promised
+ it.** `renderDigest` takes a `previous` result, and the action's only
+ production call site passes `null`, so every "(no change vs last week)" and
+ "(▲ +N vs last week)" branch is unreachable — while the integration page
+ advertised "the week-over-week change in act-now findings". The page now says
+ plainly that the field does not render and what restoring it would need.
+ Ironically, had the delta ever worked, the body would have changed weekly and
+ the frozen `updated_at` above would never have happened.
+
## [12.2.0] - 2026-09-28
### Fixed
diff --git a/apps/docs/docs/integrations/github-action.md b/apps/docs/docs/integrations/github-action.md
index 8b8ce8a49..6117530e5 100644
--- a/apps/docs/docs/integrations/github-action.md
+++ b/apps/docs/docs/integrations/github-action.md
@@ -35,7 +35,8 @@ on:
pull_request:
permissions:
contents: read
- security-events: read
+ security-events: read # code scanning
+ vulnerability-alerts: read # Dependabot — a separate permission, see below
pull-requests: write
jobs:
triage:
@@ -64,7 +65,8 @@ on:
- cron: '17 13 * * 1' # Mondays
permissions:
contents: read
- security-events: read
+ security-events: read # code scanning
+ vulnerability-alerts: read # Dependabot — a separate permission, see below
issues: write
jobs:
digest:
@@ -75,19 +77,56 @@ jobs:
mode: digest
```
-Refreshes a single `aisoc-digest`-labelled issue each week with an A–F posture
-grade and the week-over-week change in act-now findings.
+Refreshes a single `aisoc-digest`-labelled issue each week with the run's UTC
+timestamp, the sources it managed to read, and an A–F posture grade over those
+sources.
+
+The digest carries a week-over-week delta field, and **it does not render
+today**: the action's only call site passes no previous result, so the
+comparison has nothing to compare against. It is left in place because the
+renderer is shared, but nothing in the published body will show a change
+figure until a caller supplies last week's result.
## Inputs
| Input | Default | Description |
|---|---|---|
-| `github-token` | `${{ github.token }}` | Needs `security-events: read`; `pull-requests: write` for comments; `issues: write` for the digest. |
+| `github-token` | `${{ github.token }}` | Needs `security-events: read` for code scanning **and** `vulnerability-alerts: read` for Dependabot; `pull-requests: write` for comments; `issues: write` for the digest. See [Permissions](#permissions). |
| `mode` | `job-summary` | `job-summary` \| `pr-comment` \| `digest` |
| `min-severity` | `low` | Lowest severity to include (`info`→`critical`). |
| `fail-on` | `none` | Fail the job on `needs_review` or `true_positive` findings (gate mode). |
| `sources` | `dependabot,code-scanning,secret-scanning` | Which signals to pull. |
+## Permissions
+
+The three sources need three different grants, and only two of them are
+reachable by `GITHUB_TOKEN` at all.
+
+| Source | Permission | Reachable by `GITHUB_TOKEN`? |
+|---|---|---|
+| Code scanning | `security-events: read` | yes |
+| Dependabot | `vulnerability-alerts: read` | yes |
+| Secret scanning | — | **no**, needs a GitHub App or a PAT |
+
+Two traps are worth stating outright, because the digest on this repository
+fell into both and reported `grade A (100/100)` for weeks while blind to two of
+its three sources.
+
+**`security-events` does not cover Dependabot.** GitHub's workflow syntax
+reference is explicit: "For Dependabot alerts, use the `vulnerability-alerts`
+permission. Secret scanning alerts cannot be read with this permission and
+require a GitHub App or a personal access token."
+
+**Naming any permission denies every unnamed one.** "If you specify the access
+for any of these permissions, all of those that are not specified are set to
+`none`." So a `permissions:` block that lists `security-events` and omits
+`vulnerability-alerts` is not taking a default — it is denying Dependabot.
+
+A source the action cannot read is reported as skipped, and a digest with any
+skipped source refuses to publish a headline grade: it reads `incomplete (N of
+M sources readable)` and scopes the grade to what answered. Zero findings from
+a source nobody could read is not zero findings.
+
## Outputs
`total`, `escalate`, `review`, `suppress`, `headline` — wire them into
diff --git a/packages/aisoc-action/dist/index.js b/packages/aisoc-action/dist/index.js
index 98cc84928..693c4210c 100644
--- a/packages/aisoc-action/dist/index.js
+++ b/packages/aisoc-action/dist/index.js
@@ -324,34 +324,57 @@ function mapSecretScanning(a) {
timestamp: a.created_at
};
}
-async function safe(fetchFn, label, notes) {
+async function safe(fetchFn, label, notes, scanned, skipped) {
try {
- return await fetchFn();
+ const rows = await fetchFn();
+ scanned.push(label);
+ return rows;
} catch (err) {
const status = err?.status ?? err?.response?.status;
if (status === 403) notes.push(`${label}: skipped (token lacks permission or the feature is not enabled).`);
else if (status === 404) notes.push(`${label}: skipped (not enabled for this repository).`);
else notes.push(`${label}: skipped (${err?.message ?? "error"}).`);
+ skipped.push(label);
return [];
}
}
async function fetchAlerts(client, owner, repo, sources) {
const alerts = [];
const notes = [];
+ const scanned = [];
+ const skipped = [];
const q = "?state=open";
if (sources.includes("dependabot")) {
- const rows = await safe(() => client.paginate(`/repos/${owner}/${repo}/dependabot/alerts${q}`), "Dependabot", notes);
+ const rows = await safe(
+ () => client.paginate(`/repos/${owner}/${repo}/dependabot/alerts${q}`),
+ "Dependabot",
+ notes,
+ scanned,
+ skipped
+ );
alerts.push(...rows.map((r) => mapDependabot(r)));
}
if (sources.includes("code-scanning")) {
- const rows = await safe(() => client.paginate(`/repos/${owner}/${repo}/code-scanning/alerts${q}`), "Code scanning", notes);
+ const rows = await safe(
+ () => client.paginate(`/repos/${owner}/${repo}/code-scanning/alerts${q}`),
+ "Code scanning",
+ notes,
+ scanned,
+ skipped
+ );
alerts.push(...rows.map((r) => mapCodeScanning(r)));
}
if (sources.includes("secret-scanning")) {
- const rows = await safe(() => client.paginate(`/repos/${owner}/${repo}/secret-scanning/alerts${q}`), "Secret scanning", notes);
+ const rows = await safe(
+ () => client.paginate(`/repos/${owner}/${repo}/secret-scanning/alerts${q}`),
+ "Secret scanning",
+ notes,
+ scanned,
+ skipped
+ );
alerts.push(...rows.map((r) => mapSecretScanning(r)));
}
- return { alerts, notes };
+ return { alerts, notes, scanned, skipped };
}
// ../report-card/dist/index.js
@@ -424,26 +447,45 @@ function renderComment(result, notes) {
);
return lines.join("\n");
}
-function renderDigest(result, previous, notes) {
+function utcStamp(when) {
+ const iso = when.toISOString();
+ return `${iso.slice(0, 10)} ${iso.slice(11, 16)} UTC`;
+}
+function renderDigest(result, previous, notes, coverage = { scanned: [], skipped: [] }) {
const { grade, score } = postureGrade(result);
const s = result.summary;
const delta = previous ? s.truePositive - previous.summary.truePositive : null;
const deltaStr = delta === null ? "" : delta === 0 ? " (no change vs last week)" : delta > 0 ? ` (\u25B2 +${delta} vs last week)` : ` (\u25BC ${delta} vs last week)`;
- return [
- COMMENT_MARKER,
- `## \u{1F6E1}\uFE0F AiSOC weekly security posture \u2014 grade ${grade} (${score}/100)`,
+ const { scanned, skipped } = coverage;
+ const partial = skipped.length > 0;
+ const declared = scanned.length + skipped.length;
+ const stamp = utcStamp(coverage.generatedAt ?? /* @__PURE__ */ new Date());
+ const heading = partial ? `## \u{1F6E1}\uFE0F AiSOC weekly security posture \u2014 incomplete (${scanned.length} of ${declared} sources readable)` : `## \u{1F6E1}\uFE0F AiSOC weekly security posture \u2014 grade ${grade} (${score}/100)`;
+ const lines = [COMMENT_MARKER, heading, "", `**Generated** ${stamp}`];
+ if (partial) {
+ const scope = scanned.length ? `the sources that answered (${scanned.join(", ")})` : "no readable source";
+ lines.push(
+ "",
+ `> \u26A0\uFE0F **This is not an all-clear.** ${skipped.join(" and ")} could not be read, so a finding there is absent from the counts below rather than absent from the repository. Across ${scope}, the grade would be **${grade} (${score}/100)**.`
+ );
+ }
+ lines.push(
"",
- `- **${s.total}** open findings triaged`,
+ `- **${s.total}** open findings triaged${partial ? ` (from ${scanned.length ? scanned.join(", ") : "no readable source"})` : ""}`,
`- **${s.truePositive}** act-now${deltaStr}`,
`- **${s.needsReview}** need review`,
`- **${s.suppressed}** low-signal noise`,
"",
priorityLine(result),
- "",
- ...notes.length ? notes.map((n) => `> ${n}`) : [],
+ ""
+ );
+ if (scanned.length) lines.push(`> Sources read: ${scanned.join(", ")}.`);
+ if (notes.length) lines.push(...notes.map((n) => `> ${n}`));
+ lines.push(
"",
"<sub>Generated weekly by [AiSOC](https://github.com/beenuar/AiSOC). Deterministic; nothing leaves your CI.</sub>"
- ].join("\n");
+ );
+ return lines.join("\n");
}
// src/index.ts
@@ -464,7 +506,12 @@ async function run() {
const ctx = getContext();
const client = new GitHubClient(token);
info(`AiSOC: triaging security signals for ${ctx.owner}/${ctx.repo} (sources: ${sources.join(", ")})`);
- const { alerts, notes } = await fetchAlerts(client, ctx.owner, ctx.repo, sources);
+ const { alerts, notes, scanned, skipped } = await fetchAlerts(client, ctx.owner, ctx.repo, sources);
+ if (skipped.length) {
+ warning(
+ `AiSOC: ${skipped.length} of ${sources.length} declared source(s) could not be read (${skipped.join(", ")}). A source that returns nothing because the token cannot read it is not a source that found nothing \u2014 the posture grade is scoped to ${scanned.length ? scanned.join(", ") : "no readable source"}.`
+ );
+ }
const filtered = alerts.filter((a) => atLeast(a.severity, minSeverity));
const result = triageBatch(filtered, { deterministic: true });
setOutput("total", String(result.summary.total));
@@ -486,7 +533,7 @@ async function run() {
if (mode === "pr-comment") {
await upsertPrComment(client, ctx.owner, ctx.repo, ctx.prNumber, renderComment(result, notes));
} else if (mode === "digest") {
- await upsertDigestIssue(client, ctx.owner, ctx.repo, renderDigest(result, null, notes));
+ await upsertDigestIssue(client, ctx.owner, ctx.repo, renderDigest(result, null, notes, { scanned, skipped }));
}
if (failOn !== "none") {
const escalate = result.summary.truePositive;
@@ -518,8 +565,10 @@ async function upsertDigestIssue(client, owner, repo, body) {
const issues = await client.paginate(`/repos/${owner}/${repo}/issues?state=open&labels=aisoc-digest`);
if (issues[0]) {
await client.request("PATCH", `/repos/${owner}/${repo}/issues/${issues[0].number}`, { body });
+ info(`AiSOC: refreshed digest issue #${issues[0].number}.`);
} else {
await client.request("POST", `/repos/${owner}/${repo}/issues`, { title, body, labels: ["aisoc-digest"] });
+ info("AiSOC: opened a new digest issue.");
}
} catch (err) {
warning(`Could not create/update the digest issue (${err.message}).`);
diff --git a/packages/aisoc-action/src/action.test.ts b/packages/aisoc-action/src/action.test.ts
index cf14b8a83..0683432e4 100644
--- a/packages/aisoc-action/src/action.test.ts
+++ b/packages/aisoc-action/src/action.test.ts
@@ -57,9 +57,25 @@ describe("fetchAlerts", () => {
return [];
},
};
- const { alerts, notes } = await fetchAlerts(octokit, "o", "r", ["dependabot", "code-scanning", "secret-scanning"]);
+ const { alerts, notes, scanned, skipped } = await fetchAlerts(octokit, "o", "r", [
+ "dependabot",
+ "code-scanning",
+ "secret-scanning",
+ ]);
expect(alerts).toHaveLength(2);
expect(notes.join(" ")).toMatch(/Secret scanning: skipped/);
+ // A skipped source and a clean source both contribute zero alerts. The
+ // caller has to be able to tell them apart without parsing the prose.
+ expect(scanned).toEqual(["Dependabot", "Code scanning"]);
+ expect(skipped).toEqual(["Secret scanning"]);
+ });
+
+ it("reports a source that answered with nothing as read, not skipped", async () => {
+ const octokit: OctokitLike = { paginate: async () => [] };
+ const { alerts, skipped, scanned } = await fetchAlerts(octokit, "o", "r", ["dependabot", "code-scanning"]);
+ expect(alerts).toHaveLength(0);
+ expect(skipped).toEqual([]);
+ expect(scanned).toEqual(["Dependabot", "Code scanning"]);
});
});
@@ -95,4 +111,71 @@ describe("render", () => {
expect(md).toMatch(/vs last week/);
expect(priorityLine(result)).toContain("exploitable");
});
+
+ const AT = new Date("2026-09-28T20:06:00Z");
+
+ it("digest dates itself, so a stale one is visible on the issue", () => {
+ // Without a stamp the body is byte-identical any week nothing changed, so
+ // GitHub's PATCH is a no-op and `updated_at` freezes. Six consecutive
+ // weekly runs succeeded while issue #510 appeared five weeks stale, which
+ // reads as an abandoned generator rather than a quiet one.
+ const md = renderDigest(triageBatch([]), null, [], {
+ scanned: ["Dependabot", "Code scanning", "Secret scanning"],
+ skipped: [],
+ generatedAt: AT,
+ });
+ expect(md).toContain("**Generated** 2026-09-28 20:06 UTC");
+
+ const later = renderDigest(triageBatch([]), null, [], {
+ scanned: ["Dependabot", "Code scanning", "Secret scanning"],
+ skipped: [],
+ generatedAt: new Date("2026-10-05T20:06:00Z"),
+ });
+ expect(later).not.toEqual(md);
+ });
+
+ it("a fully-read clean queue still publishes the grade", () => {
+ const md = renderDigest(triageBatch([]), null, [], {
+ scanned: ["Dependabot", "Code scanning", "Secret scanning"],
+ skipped: [],
+ generatedAt: AT,
+ });
+ expect(md).toContain("grade A (100/100)");
+ expect(md).not.toMatch(/incomplete/i);
+ expect(md).not.toContain("not an all-clear");
+ expect(md).toContain("Sources read: Dependabot, Code scanning, Secret scanning.");
+ });
+
+ it("refuses to headline a grade when a declared source could not be read", () => {
+ // The defect this pins: `safe()` turns a 403 into an empty array, an empty
+ // queue grades A/100, and the digest published "grade A (100/100) — 0 open
+ // findings" with the skip demoted to a blockquote. Zero findings from a
+ // source nobody could read is not zero findings.
+ const md = renderDigest(triageBatch([]), null, [
+ "Dependabot: skipped (token lacks permission or the feature is not enabled).",
+ "Secret scanning: skipped (token lacks permission or the feature is not enabled).",
+ ], {
+ scanned: ["Code scanning"],
+ skipped: ["Dependabot", "Secret scanning"],
+ generatedAt: AT,
+ });
+ expect(md).toContain("incomplete (1 of 3 sources readable)");
+ expect(md).not.toMatch(/^## .*grade A \(100\/100\)/m);
+ expect(md).toContain("not an all-clear");
+ expect(md).toContain("Dependabot and Secret scanning could not be read");
+ // The grade survives, explicitly scoped to what answered.
+ expect(md).toContain("Across the sources that answered (Code scanning), the grade would be **A (100/100)**");
+ expect(md).toContain("(from Code scanning)");
+ });
+
+ it("names no readable source when every source was skipped", () => {
+ const md = renderDigest(triageBatch([]), null, ["Dependabot: skipped (not enabled for this repository)."], {
+ scanned: [],
+ skipped: ["Dependabot", "Code scanning", "Secret scanning"],
+ generatedAt: AT,
+ });
+ expect(md).toContain("incomplete (0 of 3 sources readable)");
+ expect(md).toContain("no readable source");
+ expect(md).not.toContain("Sources read:");
+ });
});
diff --git a/packages/aisoc-action/src/index.ts b/packages/aisoc-action/src/index.ts
index a64925cf2..5b70024f6 100644
--- a/packages/aisoc-action/src/index.ts
+++ b/packages/aisoc-action/src/index.ts
@@ -39,7 +39,14 @@ async function run(): Promise<void> {
const client = new GitHubClient(token);
info(`AiSOC: triaging security signals for ${ctx.owner}/${ctx.repo} (sources: ${sources.join(", ")})`);
- const { alerts, notes } = await fetchAlerts(client, ctx.owner, ctx.repo, sources);
+ const { alerts, notes, scanned, skipped } = await fetchAlerts(client, ctx.owner, ctx.repo, sources);
+ if (skipped.length) {
+ warning(
+ `AiSOC: ${skipped.length} of ${sources.length} declared source(s) could not be read (${skipped.join(", ")}). ` +
+ `A source that returns nothing because the token cannot read it is not a source that found nothing — ` +
+ `the posture grade is scoped to ${scanned.length ? scanned.join(", ") : "no readable source"}.`,
+ );
+ }
const filtered = alerts.filter((a) => atLeast(a.severity, minSeverity));
const result = triageBatch(filtered, { deterministic: true });
@@ -65,7 +72,12 @@ async function run(): Promise<void> {
if (mode === "pr-comment") {
await upsertPrComment(client, ctx.owner, ctx.repo, ctx.prNumber, renderComment(result, notes));
} else if (mode === "digest") {
- await upsertDigestIssue(client, ctx.owner, ctx.repo, renderDigest(result, null, notes));
+ // `previous` is null and this is the only production call site, so the
+ // week-over-week delta in `renderDigest` never renders. The field is kept
+ // because the renderer is shared and tested, but the docs no longer
+ // promise a change figure the action cannot produce — restoring it needs
+ // somewhere to persist last week's result, which does not exist yet.
+ await upsertDigestIssue(client, ctx.owner, ctx.repo, renderDigest(result, null, notes, { scanned, skipped }));
}
if (failOn !== "none") {
@@ -106,8 +118,13 @@ async function upsertDigestIssue(client: GitHubClient, owner: string, repo: stri
const issues = (await client.paginate(`/repos/${owner}/${repo}/issues?state=open&labels=aisoc-digest`)) as { number: number }[];
if (issues[0]) {
await client.request("PATCH", `/repos/${owner}/${repo}/issues/${issues[0].number}`, { body });
+ // Logged on success, not only on failure. The run log previously ended
+ // at the triage headline whether the issue was written or not, so "did
+ // the digest publish?" could not be answered from a green run.
+ info(`AiSOC: refreshed digest issue #${issues[0].number}.`);
} else {
await client.request("POST", `/repos/${owner}/${repo}/issues`, { title, body, labels: ["aisoc-digest"] });
+ info("AiSOC: opened a new digest issue.");
}
} catch (err) {
warning(`Could not create/update the digest issue (${(err as Error).message}).`);
diff --git a/packages/aisoc-action/src/render.ts b/packages/aisoc-action/src/render.ts
index db59caceb..fbfc76e63 100644
--- a/packages/aisoc-action/src/render.ts
+++ b/packages/aisoc-action/src/render.ts
@@ -81,25 +81,83 @@ export function renderComment(result: TriageResult, notes: string[]): string {
return lines.join("\n");
}
-export function renderDigest(result: TriageResult, previous: TriageResult | null, notes: string[]): string {
+/**
+ * Which declared sources the digest actually managed to read, and when it ran.
+ *
+ * Both halves exist because of the same defect. A source the token cannot read
+ * contributes zero findings, `postureGrade` sees an empty queue and returns
+ * A/100, and the digest headlined that as an all-clear with the "skipped" note
+ * demoted to a blockquote underneath — reporting clean because it could not
+ * look. And with nothing in the body tied to the run, a week where nothing
+ * changed rendered a byte-identical body, so GitHub's PATCH was a no-op and
+ * the issue's `updated_at` froze. A live weekly generator then read as an
+ * abandoned one: six consecutive scheduled runs succeeded while the issue
+ * appeared five weeks stale.
+ */
+export interface DigestCoverage {
+ scanned: string[];
+ skipped: string[];
+ /** Injectable so the rendering is deterministic under test. */
+ generatedAt?: Date;
+}
+
+function utcStamp(when: Date): string {
+ const iso = when.toISOString();
+ return `${iso.slice(0, 10)} ${iso.slice(11, 16)} UTC`;
+}
+
+export function renderDigest(
+ result: TriageResult,
+ previous: TriageResult | null,
+ notes: string[],
+ coverage: DigestCoverage = { scanned: [], skipped: [] },
+): string {
const { grade, score } = postureGrade(result);
const s = result.summary;
const delta = previous ? s.truePositive - previous.summary.truePositive : null;
const deltaStr =
delta === null ? "" : delta === 0 ? " (no change vs last week)" : delta > 0 ? ` (▲ +${delta} vs last week)` : ` (▼ ${delta} vs last week)`;
- return [
- COMMENT_MARKER,
- `## 🛡️ AiSOC weekly security posture — grade ${grade} (${score}/100)`,
+
+ const { scanned, skipped } = coverage;
+ const partial = skipped.length > 0;
+ const declared = scanned.length + skipped.length;
+ const stamp = utcStamp(coverage.generatedAt ?? new Date());
+
+ // A grade is only a posture statement if every declared source answered.
+ // When one did not, the headline says so instead of publishing a number a
+ // reader would take for an all-clear, and the grade is explicitly scoped to
+ // what was read.
+ const heading = partial
+ ? `## 🛡️ AiSOC weekly security posture — incomplete (${scanned.length} of ${declared} sources readable)`
+ : `## 🛡️ AiSOC weekly security posture — grade ${grade} (${score}/100)`;
+
+ const lines = [COMMENT_MARKER, heading, "", `**Generated** ${stamp}`];
+
+ if (partial) {
+ const scope = scanned.length ? `the sources that answered (${scanned.join(", ")})` : "no readable source";
+ lines.push(
+ "",
+ `> ⚠️ **This is not an all-clear.** ${skipped.join(" and ")} could not be read, so a finding there is absent from the counts below rather than absent from the repository. Across ${scope}, the grade would be **${grade} (${score}/100)**.`,
+ );
+ }
+
+ lines.push(
"",
- `- **${s.total}** open findings triaged`,
+ `- **${s.total}** open findings triaged${partial ? ` (from ${scanned.length ? scanned.join(", ") : "no readable source"})` : ""}`,
`- **${s.truePositive}** act-now${deltaStr}`,
`- **${s.needsReview}** need review`,
`- **${s.suppressed}** low-signal noise`,
"",
priorityLine(result),
"",
- ...(notes.length ? notes.map((n) => `> ${n}`) : []),
+ );
+
+ if (scanned.length) lines.push(`> Sources read: ${scanned.join(", ")}.`);
+ if (notes.length) lines.push(...notes.map((n) => `> ${n}`));
+
+ lines.push(
"",
"<sub>Generated weekly by [AiSOC](https://github.com/beenuar/AiSOC). Deterministic; nothing leaves your CI.</sub>",
- ].join("\n");
+ );
+ return lines.join("\n");
}
diff --git a/packages/aisoc-action/src/sources.ts b/packages/aisoc-action/src/sources.ts
index e856e55b4..79f39364b 100644
--- a/packages/aisoc-action/src/sources.ts
+++ b/packages/aisoc-action/src/sources.ts
@@ -17,6 +17,18 @@ export interface OctokitLike {
export interface FetchResult {
alerts: Alert[];
notes: string[];
+ /**
+ * Which declared sources were actually read, and which were not.
+ *
+ * `safe()` turns a 403/404 into a note and an empty array, which makes a
+ * source nobody could read indistinguishable from a source that was read
+ * and found clean — and downstream, zero findings grades as A/100. Callers
+ * need the two apart to avoid publishing an all-clear they did not earn, so
+ * the split is part of the result rather than something a reader has to
+ * recover by pattern-matching the prose in `notes`.
+ */
+ scanned: string[];
+ skipped: string[];
}
function sev(value: string | undefined): Severity {
@@ -93,14 +105,23 @@ export function mapSecretScanning(a: Record<string, any>): Alert {
};
}
-async function safe(fetchFn: () => Promise<unknown[]>, label: string, notes: string[]): Promise<unknown[]> {
+async function safe(
+ fetchFn: () => Promise<unknown[]>,
+ label: string,
+ notes: string[],
+ scanned: string[],
+ skipped: string[],
+): Promise<unknown[]> {
try {
- return await fetchFn();
+ const rows = await fetchFn();
+ scanned.push(label);
+ return rows;
} catch (err: any) {
const status = err?.status ?? err?.response?.status;
if (status === 403) notes.push(`${label}: skipped (token lacks permission or the feature is not enabled).`);
else if (status === 404) notes.push(`${label}: skipped (not enabled for this repository).`);
else notes.push(`${label}: skipped (${err?.message ?? "error"}).`);
+ skipped.push(label);
return [];
}
}
@@ -113,19 +134,39 @@ export async function fetchAlerts(
): Promise<FetchResult> {
const alerts: Alert[] = [];
const notes: string[] = [];
+ const scanned: string[] = [];
+ const skipped: string[] = [];
const q = "?state=open";
if (sources.includes("dependabot")) {
- const rows = await safe(() => client.paginate(`/repos/${owner}/${repo}/dependabot/alerts${q}`), "Dependabot", notes);
+ const rows = await safe(
+ () => client.paginate(`/repos/${owner}/${repo}/dependabot/alerts${q}`),
+ "Dependabot",
+ notes,
+ scanned,
+ skipped,
+ );
alerts.push(...rows.map((r) => mapDependabot(r as Record<string, any>)));
}
if (sources.includes("code-scanning")) {
- const rows = await safe(() => client.paginate(`/repos/${owner}/${repo}/code-scanning/alerts${q}`), "Code scanning", notes);
+ const rows = await safe(
+ () => client.paginate(`/repos/${owner}/${repo}/code-scanning/alerts${q}`),
+ "Code scanning",
+ notes,
+ scanned,
+ skipped,
+ );
alerts.push(...rows.map((r) => mapCodeScanning(r as Record<string, any>)));
}
if (sources.includes("secret-scanning")) {
- const rows = await safe(() => client.paginate(`/repos/${owner}/${repo}/secret-scanning/alerts${q}`), "Secret scanning", notes);
+ const rows = await safe(
+ () => client.paginate(`/repos/${owner}/${repo}/secret-scanning/alerts${q}`),
+ "Secret scanning",
+ notes,
+ scanned,
+ skipped,
+ );
alerts.push(...rows.map((r) => mapSecretScanning(r as Record<string, any>)));
}
- return { alerts, notes };
+ return { alerts, notes, scanned, skipped };
}