Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

docs(media): the published technical guide still showed the pre-v20.1.0 console (#1492)

Beenu Aroraf15279e5a10d
diff --git a/.pr-body.md b/.pr-body.md
new file mode 100644
index 000000000..d621d18aa
--- /dev/null
+++ b/.pr-body.md
@@ -0,0 +1,102 @@
+## What this is
+
+Close-out of the v20.1.0 media work. The screenshots themselves were already
+re-captured in the release (#1476) and are current — verified by what they
+show, not by the commit touching them: footer `v20.1.0`, `Evaluate on History`
+and `EASM` present, the removed `Investigation Chat` gone, 1,754 live KEV rows,
+a real local-model verdict, alerts labelled `aisoc_sample`.
+
+What was missed is everything downstream of those files.
+
+## The published PDF was still showing the old console
+
+`apps/web/public/papers/aisoc-technical-guide.pdf` — the "26 page PDF" the
+README front page advertises — was last rendered in #1229. The release swapped
+fifteen captures underneath it and never re-rendered, so the document a reader
+downloads showed a console two releases old.
+
+**The check for exactly this passed throughout.** `papers.yml` already triggers
+on `apps/web/public/screenshots/**`, with an inline comment explaining why a
+screenshot refresh changes the rendered PDF while leaving every other watched
+path untouched. The trigger was right; the comparison was blind to it, because
+the recorded digest covered only the markdown. The digest is now taken over the
+**source closure** — the markdown plus every local figure it embeds, keyed by
+path so repointing a caption at a different file counts as a change too.
+
+Evidence it detects the real defect, run on this tree before the re-render:
+
+```
+$ python3 scripts/render_white_paper.py --check
+[render_white_paper] rendered PDFs are out of date:
+ - aisoc-technical-guide.md changed since its PDF was rendered
+```
+
+It names the guide and leaves `l0-l4-automation-maturity.md` alone, which
+embeds no figures — so its recorded digest is unchanged and its PDF is not
+touched by this PR. Two self-test cases cover the new behaviour: the negative
+control (a figure re-captured after the render is refused) and the positive
+control that stops the gate from refusing everything. 8/8 pass.
+
+## Re-rendering exposed captions describing the previous images
+
+The captures were swapped under captions written for the ones they replaced, so
+the guide asserted things its own figures contradicted:
+
+| Caption said | Figure shows |
+|---|---|
+| "Twelve alerts the pipeline produced" | five alerts |
+| "confidence 80/100" | 85/100 |
+| "1,725 real Known Exploited Vulnerabilities entries" | 1,754 |
+| "The connector catalogue, reporting **84 connector types available**" | the connectors page, nothing connected, no catalogue in frame |
+| "The step palette … captured when the vocabulary was twenty-two members" | a nine-step playbook in the editor |
+
+Each is now written against what is actually in the frame. The step-palette
+caption's point — that `approval` is offered now that it can durably pause —
+survives, because it is visible in the new image as step 6.
+
+## Two assets decided rather than re-captured
+
+- **`attack-graph.png` is stale and stays, dated.** It is a v10.0.0 capture
+ (footer `v10.0.0`, and the nav still shows the removed `Investigation Chat`)
+ presented as current. The populated graph needs the `full` profile, so rather
+ than delete the only recorded one it is now dated in both the guide and the
+ provenance README, next to the CORE capture that shows the same page
+ degrading honestly. Re-capturing it is a `full`-profile job, not a CORE one.
+- **`connectors.png` is deleted.** It duplicated `connector-wizard.png`'s view
+ and carried a rendering artifact — a second sidebar bleeding into the right
+ edge — which was being shipped inside the PDF. Nothing else referenced it.
+
+**No screenshot was re-captured.** `git diff f2955c7c..origin/main -- apps/web/`
+is empty, so the console today is byte-identical to the one already in the
+committed images; a re-capture would reproduce the same pixels and risk a worse
+set.
+
+## Prose corrected
+
+- `README.md` called the stills "earlier runs" when the refresh had made them
+ **newer** than the recording. Still 248 lines, inside the 250 cap.
+- `apps/web/public/demo/README.md` now states that the recording's console is
+ v11.0.0, so the mismatch against the v20.1.0 stills is accounted for rather
+ than discovered. The asset stays a committed GIF.
+- `apps/web/public/screenshots/README.md` described the previous session
+ throughout — fourteen containers, 1,725 KEV, twelve alerts, a connector
+ wizard and a step palette that are not in the images any more. It now
+ describes what is in the directory, and carries the two-item refresh
+ checklist that would have caught both halves of this.
+
+## Verification
+
+- `render_white_paper.py --self-test` → 8/8, including both new controls
+- `render_white_paper.py --check` → clean after re-render; the regenerated PDF
+ is 26 pages, so the README badge stays true, and grew 2,056,523 → 2,480,949
+ bytes, consistent with the larger current captures
+- `readme_gates.py` OK · `check_comment_paths.py` OK (2,270 paths) ·
+ `check_repo_self_links.py` OK · `check_hosted_hostname.py` OK ·
+ `check_competitor_names.py` OK
+- `ruff 0.16.10 format --check .` → 2,588 files clean; `ruff check` on the
+ changed script clean
+
+## Out of scope
+
+Does not touch `services/fusion/**`, the per-service `_health.py` files,
+#1480, #1482 or #1446.
diff --git a/README.md b/README.md
index 9f92475af..e28664b17 100644
--- a/README.md
+++ b/README.md
@@ -28,7 +28,7 @@ history you already collected, and a human approves before anything reaches a ve
**[Watch the full three minutes](apps/web/public/demo/demo.mp4)** — install to AI verdict on one
server against the published images, terminal waits shortened and the recording saying so on screen.
-The stills below are earlier runs under the same rules: no seeded rows, no demo mode, no mockups.
+The stills below are from a v20.1.0 run under the same rules — the recording is older than they are.
([step by step](apps/docs/docs/deployment/walkthrough.mdx) · [what is real](apps/web/public/screenshots/README.md))
| | |
diff --git a/apps/web/content/papers/aisoc-technical-guide.md b/apps/web/content/papers/aisoc-technical-guide.md
index da65ea3b8..dbe0e2706 100644
--- a/apps/web/content/papers/aisoc-technical-guide.md
+++ b/apps/web/content/papers/aisoc-technical-guide.md
@@ -368,17 +368,16 @@ table, on an endpoint with no authentication at all.
### Pull
-![The connector catalogue](apps/web/public/screenshots/connector-wizard.png)
+![The connector surface with nothing connected](apps/web/public/screenshots/connector-wizard.png)
-*The connector wizard, reporting **84 connector types available** — generated
-from the registry, not hand-typed.*
+*An honest empty state: nothing is invented to fill the page, and the counters
+read zero rather than being hidden. `Universal capture (push)` is offered
+alongside the catalogue, so a vendor with no connector of its own can still
+POST JSON, send email, or speak CEF/HEC.*
-Settings → Connectors → Add connector. Credentials are encrypted at rest with
-Fernet before they touch the database.
-
-![The connector surface with nothing connected](apps/web/public/screenshots/connectors.png)
-
-*An honest empty state. Nothing is invented to fill the page.*
+Settings → Connectors → Add connector opens the catalogue, whose type count is
+generated from the registry rather than hand-typed. Credentials are encrypted
+at rest with Fernet before they touch the database.
Connecting a source is meant to be the only manual step: the scheduler polls
on a five-minute default, events flow to ingest, fusion promotes what matters,
@@ -410,8 +409,9 @@ useful head start", not "a finished migration".
![The alert queue](apps/web/public/screenshots/alerts-queue.png)
-*Twelve alerts the pipeline produced from pushed telemetry, each attributed to
-the connector that fed it.*
+*Five alerts the pipeline produced from pushed telemetry, each attributed to
+the connector that fed it — here `aisoc_sample`, the labelled synthetic set the
+setup wizard pushes through the same ingest endpoint a real connector uses.*
Alerts are ordered by a priority score: severity multiplied by asset
criticality, identity privilege and known-exploited-vulnerability exposure.
@@ -437,7 +437,7 @@ is not there.*
![A real local-model triage verdict](apps/web/public/screenshots/ai-triage-verdict.png)
*A verdict from the bundled local `llama3.2:3b-instruct-q4_K_M`: true
-positive, confidence 80/100, groundedness **not assessed**, and the model's
+positive, confidence 85/100, groundedness **not assessed**, and the model's
own rationale verbatim. No hosted provider was configured.*
What the agent does and does not do:
@@ -497,9 +497,9 @@ attribute to a credential precisely because it is silent.
![The CISA KEV catalogue](apps/web/public/screenshots/threat-intel-kev.png)
-*1,725 real Known Exploited Vulnerabilities entries, fetched from the public
-CISA catalogue. Not synthetic, not ours, unmodified — and the page
-distinguishes the catalogue size from the page it is showing.*
+*1,754 real Known Exploited Vulnerabilities entries, fetched from the public
+CISA catalogue at capture time. Not synthetic, not ours, unmodified — and the
+page distinguishes the catalogue size from the hundred rows it is showing.*
### Hunting
@@ -514,7 +514,11 @@ Hunting needs the `full` profile, because it reads the ClickHouse lake.
![The entity graph](apps/web/public/screenshots/attack-graph.png)
-*The graph over real ingested telemetry, with Neo4j running.*
+*The graph over real ingested telemetry, with Neo4j running. Dated honestly:
+this one is a v10.0.0 capture, kept because it is the only populated view
+recorded so far and the page needs the `full` profile to produce one. Every
+other figure in this guide is from v20.1.0, so the console chrome here is a
+release behind — the graph itself is unchanged.*
![The same page in CORE](apps/web/public/screenshots/attack-graph-core-degraded.png)
@@ -526,12 +530,14 @@ much of the product as the populated view.*
![The playbook library](apps/web/public/screenshots/playbooks.png)
-![The step palette](apps/web/public/screenshots/playbook-editor.png)
+![The playbook editor](apps/web/public/screenshots/playbook-editor.png)
-*Captured when the vocabulary was twenty-two members and `approval` was
-withheld for having no durable pause — a step that cannot suspend should not
-be offered. The engine now declares twenty-five step types and `approval` is a
-durable pause, so the palette offers it; the screenshot is the earlier run.*
+*A shipped pack opened in the editor: nine steps, left as an ordered list and
+right as the graph that runs. Step 6 is `Wait for human approval`, which is
+worth pointing at — `approval` was once withheld from the palette for having
+no durable pause, on the principle that a step which cannot suspend should not
+be offered. It suspends now, so it is offered, and a playbook can stop here
+until a person says otherwise.*
### The case lifecycle
diff --git a/apps/web/public/demo/README.md b/apps/web/public/demo/README.md
index 0815c671c..14cf202b9 100644
--- a/apps/web/public/demo/README.md
+++ b/apps/web/public/demo/README.md
@@ -29,6 +29,13 @@ In this recording both triage runs fell back to the deterministic path, which
the console labels (`model_used` reads `kafka:auto_triage:deterministic`), and
the closing card says so.
+The console in the recording is **v11.0.0**, which is older than the stills in
+`apps/web/public/screenshots/` — those were re-captured at v20.1.0. The
+navigation gained entries after this was recorded, so the two do not match
+pixel for pixel. The sequence the recording demonstrates is unchanged, and it
+is kept as recorded rather than re-cut; see "Re-recording" below for what
+replacing it would involve.
+
**That is no longer the common case.** Triage now asks the provider to
constrain its reply to a JSON object: measured over 50 alerts through the
gateway, replies triage could use went from 44 of 50 to 50 of 50. The recording
diff --git a/apps/web/public/papers/README.md b/apps/web/public/papers/README.md
index b3565e720..d9b369865 100644
--- a/apps/web/public/papers/README.md
+++ b/apps/web/public/papers/README.md
@@ -61,8 +61,12 @@ When adding a new paper:
Do not commit PDFs without their matching markdown source.
-**Editing a paper's markdown means running `make papers` and committing
-the result in the same change.** CI does not refresh PDFs for you. It
+**Editing a paper's markdown — or replacing a screenshot it embeds —
+means running `make papers` and committing the result in the same
+change.** The second half is not hypothetical: the v20.1.0 release
+refreshed fifteen console captures without re-rendering, and the
+published technical guide kept showing a console two releases old.
+CI does not refresh PDFs for you. It
once did, pushing a `chore(papers)` commit straight to `main`, but that
stopped working when `main` became branch-protected — the push is
rejected with `GH006`. Opening a pull request instead is not available
@@ -81,7 +85,11 @@ reports rather than repairs:
Staleness is judged by comparing each source's SHA-256 against the digest
recorded in `render-manifest.json` when its PDF was last rendered, which
-`make papers` updates. PDF bytes are deliberately not compared: WeasyPrint
+`make papers` updates. The digest covers the paper's **source closure** —
+the markdown plus every local figure it embeds, keyed by path — so
+refreshing a screenshot marks the PDF stale even though the markdown is
+untouched, and repointing a caption at a different file counts as a change
+too. PDF bytes are deliberately not compared: WeasyPrint
stamps a creation timestamp into its output and glyph metrics depend on
the host's installed fonts, so two correct renders of the same source are
not byte-identical. Commit `render-manifest.json` alongside the PDF.
diff --git a/apps/web/public/papers/aisoc-technical-guide.pdf b/apps/web/public/papers/aisoc-technical-guide.pdf
index ad70b3840..5658c3a92 100644
Binary files a/apps/web/public/papers/aisoc-technical-guide.pdf and b/apps/web/public/papers/aisoc-technical-guide.pdf differ
diff --git a/apps/web/public/papers/render-manifest.json b/apps/web/public/papers/render-manifest.json
index 7708715e8..b7c65ec59 100644
--- a/apps/web/public/papers/render-manifest.json
+++ b/apps/web/public/papers/render-manifest.json
@@ -1,6 +1,6 @@
{
"sources": {
- "aisoc-technical-guide.md": "5ce35ff60425fa757635e889c890585922f6b824e2d9c90e4ceae7ce6a994d50",
+ "aisoc-technical-guide.md": "0324f7f9cf8bcb155c82590393fead26ae74cf92f38391c82d6ce76e3533e298",
"l0-l4-automation-maturity.md": "4da4b02d8e7b6d2839a593f09303191cd71c6b9ef28b34555a1a2086cf4c8781"
}
}
diff --git a/apps/web/public/screenshots/README.md b/apps/web/public/screenshots/README.md
index a40461dc0..eee00e2fd 100644
--- a/apps/web/public/screenshots/README.md
+++ b/apps/web/public/screenshots/README.md
@@ -17,24 +17,21 @@ make up # CORE: 16 long-running services, no credentials of any kind
make smoke # one real event through the real spine
```
-Then telemetry was pushed through the documented ingest API
-(`POST /v1/ingest/batch`) and the console was driven with Playwright at
-1440×900, signed in as the administrator `make up` created.
-
-One detail, so the record is exact: these were captured immediately before
-ingest authentication landed, so the pushes carried no token. That changed what
-the *door* requires and nothing about what came through it — normalization,
-detection, promotion, correlation and triage are the same code either way.
-Re-capturing now would need `make ingest-token` first.
+The console was then driven with Playwright at 1440×900, signed in as the
+administrator `make up` created, and the security events were loaded through
+the setup wizard's **Load sample data**, which pushes them through the same
+documented ingest endpoint a real connector uses rather than inserting rows.
+The console attributes every resulting alert to `aisoc_sample`, which is
+visible in the captures themselves — that label is the point, not decoration.
What is real in these images, stated precisely:
-- **The deployment.** Fourteen containers, the images built from this
+- **The deployment.** Sixteen containers, the images built from this
repository, Postgres/Kafka/Qdrant/Ollama running locally.
- **The pipeline.** Normalization, detection, promotion, correlation and
alerting were all performed by the running services. No row was seeded into
the database.
-- **The threat intelligence.** 1,725 CISA Known Exploited Vulnerabilities
+- **The threat intelligence.** 1,754 CISA Known Exploited Vulnerabilities
entries, fetched from the public catalogue at capture time. Not ours, not
synthetic, unmodified.
- **The AI verdict.** Produced by the bundled local `llama3.2:3b-instruct-q4_K_M`
@@ -45,28 +42,31 @@ What is real in these images, stated precisely:
They are as much of the product as the populated ones.
What is *authored* rather than observed: the security events themselves. They
-were written by hand to be representative and pushed through the ingest API.
-They are not a real intrusion and no caption may imply they are.
+are the labelled `aisoc_sample` set, written to be representative and pushed
+through the ingest API. They are not a real intrusion and no caption may imply
+they are.
## The captures
+Unless a row says otherwise, every image below was captured from a v20.1.0
+stack on 2026-10-22 and the console footer in it reads `v20.1.0`.
+
| File | Surface | What is worth seeing |
|---|---|---|
| `dashboard.png` | `/dashboard` | Operations funnel over real counts. `MTTR` reads *not measured · no cases closed* rather than `0`. |
-| `alerts-queue.png` | `/alerts` | Twelve alerts the pipeline produced, each attributed to the connector that fed it. |
-| `investigation-rail.png` | `/alerts` → row | The rail's Story view. Progression says *no ATT&CK tactic is mapped … that is a gap in the detection's metadata rather than a statement about the activity.* |
-| `ai-triage-verdict.png` | `/alerts` → row → Details | A real local-model verdict: `true positive`, confidence 80/100, groundedness *not assessed*, and the model's own rationale verbatim. |
-| `threat-intel-kev.png` | `/threat-intel` | 1,725 real KEV entries; the page distinguishes the catalogue size from the page it is showing. |
+| `alerts-queue.png` | `/alerts` | The five alerts the pipeline produced, each attributed to the `aisoc_sample` source that fed it. |
+| `investigation-rail.png` | `/alerts` → row | The rail's Story view. Progression says *No ATT&CK tactic is mapped to this alert … that is a gap in the detection's metadata rather than a statement about the activity.* |
+| `ai-triage-verdict.png` | `/alerts` → row → Details | A real local-model verdict: `true positive`, confidence 85/100, groundedness *not assessed*, and the model's own rationale verbatim. |
+| `threat-intel-kev.png` | `/threat-intel` | 1,754 real KEV entries; the page distinguishes the catalogue size from the hundred rows it is showing. |
| `soc-operations.png` | `/dashboards/operations` | Pipeline health against a tenant with no connectors — *"Nothing is feeding the pipeline yet, so an empty alert queue is expected."* |
-| `connectors.png` | `/connectors` | The connector surface with nothing connected. |
-| `connector-wizard.png` | `/connectors` → Add connector | The catalogue, reporting **84 connector types available** — the generated count at the time of capture, not a hand-typed one. The generator now reports **87**; the image is the older run. |
-| `attack-graph.png` | `/graph` | The entity graph over real ingested telemetry, with Neo4j running. |
+| `connector-wizard.png` | `/connectors` | The connector surface with nothing connected: counters at zero rather than hidden, and `Universal capture (push)` offered for vendors with no connector of their own. |
+| `attack-graph.png` | `/graph` | The entity graph over real ingested telemetry, with Neo4j running. **A v10.0.0 capture, not v20.1.0** — the populated view needs the `full` profile, so this is the most recent one recorded. The console chrome is a release behind; the graph is not. |
| `attack-graph-core-degraded.png` | `/graph` | The same page in CORE, where Neo4j is not running: it names the failure (`API 503 … /api/v1/graph`) instead of drawing an invented graph. |
| `federated-search.png` | `/federated-search` | Honest empty state — no SIEM connected, and it says which four it would query. |
-| `playbooks.png` | `/playbooks` | The playbook library. |
-| `playbook-editor.png` | `/playbooks/{id}` → Add step | The step palette, captured when the vocabulary was twenty-two members and `approval` was withheld for having no pause/resume. Both have changed: the engine declares **twenty-five** step types and `approval` is a durable pause, so the palette now offers it. Re-capture before citing the tile count. |
-| `detection-rules.png` | `/detection` | The detection rule surface. |
-| `hunt.png` | `/hunt` | The hunt workbench. |
+| `playbooks.png` | `/playbooks` | The playbook library: 63 playbooks, split into 51 shipped pack entries and 12 custom. |
+| `playbook-editor.png` | `/playbooks/{id}` | A shipped pack open in the editor — nine steps as an ordered list and as the graph that runs. Step 6 is `Wait for human approval`, which the palette withheld until `approval` became a durable pause. |
+| `detection-rules.png` | `/detection` | The rule surface, reporting the catalogue figure (2,614) and `0 authored here` — the catalogue is a different measurement from the executable-rule count the README quotes. |
+| `hunt.png` | `/hunt` | The hunt workbench: plain English on the left, the generated KQL on the right. |
| `cases-empty.png` | `/cases` | No cases, said plainly. |
## Refreshing them
@@ -75,3 +75,17 @@ They are not a real intrusion and no caption may imply they are.
schedule, but it drives the **seeded demo stack**. Anything it produces is
demo-mode output and must be labelled as such wherever it appears — which is
why the captures above were taken by hand against a real CORE stack instead.
+
+Two things to do in the same commit as a refresh, because both have already
+been missed once:
+
+1. **Re-read the captions.** The v20.1.0 refresh replaced the images under
+ captions written for the previous ones, leaving the technical guide
+ claiming twelve alerts over a picture of five and a "step palette" over a
+ picture of a playbook. A screenshot and the sentence under it are one
+ claim.
+2. **Run `make papers`.** `apps/web/content/papers/` embeds these files, so a
+ refresh leaves every committed PDF showing the old console.
+ `scripts/render_white_paper.py --check` now digests each paper's markdown
+ *and the figures it embeds*, so it will tell you — it could not before, and
+ that is how a published PDF kept showing a console two releases old.
diff --git a/apps/web/public/screenshots/connectors.png b/apps/web/public/screenshots/connectors.png
deleted file mode 100644
index 41022c927..000000000
Binary files a/apps/web/public/screenshots/connectors.png and /dev/null differ
diff --git a/scripts/render_white_paper.py b/scripts/render_white_paper.py
index aee2bd0bb..56bd288c2 100755
--- a/scripts/render_white_paper.py
+++ b/scripts/render_white_paper.py
@@ -48,7 +48,8 @@ DEFAULT_OUT_DIR = REPO_ROOT / "apps" / "web" / "public" / "papers"
# Records the digest of each markdown source at the moment its PDF was
# rendered, so `--check` can tell a stale PDF from a current one.
#
-# The digest is taken over the *source markdown*, never the rendered PDF.
+# The digest is taken over the *source closure* — the markdown plus every
+# local figure it embeds — and never over the rendered PDF.
# WeasyPrint stamps a creation timestamp into its output and glyph metrics
# depend on which fonts the host happens to have installed, so two correct
# renders of one source do not produce equal bytes. A PDF byte comparison
@@ -197,6 +198,16 @@ COVER_HTML = """
_IMG_RE = re.compile(r"!\[[^\]]*\]\(([^)\s]+)")
+# Papers reference figures from the repository root, which is also the
+# `base_url` WeasyPrint renders them against. One name for that root, read by
+# both the missing-figure refusal and the staleness digest, so the two cannot
+# disagree about which file a caption points at.
+IMAGE_ROOT = REPO_ROOT
+
+
+def _resolve_image(ref: str) -> Path:
+ return IMAGE_ROOT / ref.lstrip("/")
+
def _assert_images_resolved(markdown_path: Path, output_path: Path) -> None:
"""Refuse a PDF whose figures point at files that are not there.
@@ -211,9 +222,7 @@ def _assert_images_resolved(markdown_path: Path, output_path: Path) -> None:
"""
source = markdown_path.read_text(encoding="utf-8")
missing = [
- ref
- for ref in _IMG_RE.findall(source)
- if not ref.startswith(("http://", "https://", "data:")) and not (repo_root() / ref.lstrip("/")).is_file()
+ ref for ref in _IMG_RE.findall(source) if not ref.startswith(("http://", "https://", "data:")) and not _resolve_image(ref).is_file()
]
if missing:
output_path.unlink(missing_ok=True)
@@ -329,7 +338,31 @@ def _discover() -> list[tuple[Path, Path]]:
def _source_digest(path: Path) -> str:
- return hashlib.sha256(path.read_bytes()).hexdigest()
+ """Digest a paper's markdown *and every local figure it embeds*.
+
+ A PDF is only as current as its screenshots. Digesting the markdown alone
+ meant refreshing a capture left the recorded digest unchanged, so
+ ``--check`` reported the PDF as current while it still showed the previous
+ console. That is not hypothetical: the v20.1.0 release replaced fifteen
+ console captures without re-rendering the technical guide, and this check
+ passed throughout — the workflow even runs on `screenshots/**` for exactly
+ this reason, so the trigger was right and the comparison was blind to it.
+ """
+ digest = hashlib.sha256()
+ digest.update(path.read_bytes())
+ for ref in sorted(set(_IMG_RE.findall(path.read_text(encoding="utf-8", errors="replace")))):
+ if ref.startswith(("http://", "https://", "data:")):
+ continue
+ resolved = _resolve_image(ref)
+ if not resolved.is_file():
+ # A missing figure is `_assert_images_resolved`'s refusal to make,
+ # at render time, where it can say which caption is empty.
+ continue
+ # The path is part of the digest, so repointing a caption at a
+ # different file counts as a change even if the bytes are identical.
+ digest.update(b"\x00" + ref.encode("utf-8") + b"\x00")
+ digest.update(resolved.read_bytes())
+ return digest.hexdigest()
def _display(path: Path) -> str:
@@ -427,18 +460,24 @@ def _injected_defect_cases() -> list[tuple[str, bool]]:
everything would "detect" all four defects and be useless, so a
well-formed paper is required to pass.
"""
- global DEFAULT_SRC_DIR, DEFAULT_OUT_DIR, MANIFEST_PATH
+ global DEFAULT_SRC_DIR, DEFAULT_OUT_DIR, MANIFEST_PATH, IMAGE_ROOT
- def plant(root: Path, *, pdf: bool, recorded: str | None, orphan: bool) -> None:
+ def plant(root: Path, *, pdf: bool, recorded: str | None, orphan: bool, figure: str | None = None) -> None:
# Without this, the assignments below bind locals to `plant` and every
# case silently grades the real tree instead of the planted one.
- global DEFAULT_SRC_DIR, DEFAULT_OUT_DIR, MANIFEST_PATH
+ global DEFAULT_SRC_DIR, DEFAULT_OUT_DIR, MANIFEST_PATH, IMAGE_ROOT
src_dir, out_dir = root / "content", root / "public"
src_dir.mkdir(parents=True)
out_dir.mkdir(parents=True)
+ IMAGE_ROOT = root
source = src_dir / "paper.md"
- source.write_text("# a paper\n")
+ source.write_text("# a paper\n" if figure is None else "# a paper\n\n![a console](shot.png)\n")
+ if figure is not None:
+ # Record the digest against the *first* capture, then replace the
+ # file the way a re-capture does. The markdown never changes, which
+ # is precisely how the real staleness went unnoticed.
+ (root / "shot.png").write_bytes(b"first capture")
if pdf:
(out_dir / "paper.pdf").write_bytes(b"%PDF-1.7 not a real render\n")
entries: dict[str, str] = {}
@@ -448,6 +487,8 @@ def _injected_defect_cases() -> list[tuple[str, bool]]:
entries["paper.md"] = "0" * 64
if orphan:
entries["deleted.md"] = "0" * 64
+ if figure == "refreshed":
+ (root / "shot.png").write_bytes(b"re-captured against a later release")
DEFAULT_SRC_DIR, DEFAULT_OUT_DIR = src_dir, out_dir
MANIFEST_PATH = out_dir / "render-manifest.json"
MANIFEST_PATH.write_text(json.dumps({"sources": entries}) + "\n")
@@ -458,9 +499,17 @@ def _injected_defect_cases() -> list[tuple[str, bool]]:
("a source with no PDF at all is refused", {"pdf": False, "recorded": "match", "orphan": False}, False),
("a PDF with no recorded render is refused", {"pdf": True, "recorded": None, "orphan": False}, False),
("a recorded paper whose source is gone is refused", {"pdf": True, "recorded": "match", "orphan": True}, False),
+ # The control for the pair below: an embedded figure that has not moved
+ # must still pass, or the two cases together prove nothing.
+ ("a paper whose embedded figure is unchanged passes", {"pdf": True, "recorded": "match", "orphan": False, "figure": "kept"}, True),
+ (
+ "a figure re-captured since the PDF was rendered is refused",
+ {"pdf": True, "recorded": "match", "orphan": False, "figure": "refreshed"},
+ False,
+ ),
]
- saved = (DEFAULT_SRC_DIR, DEFAULT_OUT_DIR, MANIFEST_PATH)
+ saved = (DEFAULT_SRC_DIR, DEFAULT_OUT_DIR, MANIFEST_PATH, IMAGE_ROOT)
results: list[tuple[str, bool]] = []
try:
for description, shape, should_pass in cases:
@@ -468,7 +517,7 @@ def _injected_defect_cases() -> list[tuple[str, bool]]:
plant(Path(tmp), **shape) # type: ignore[arg-type]
results.append((description, (_check() == 0) is should_pass))
finally:
- DEFAULT_SRC_DIR, DEFAULT_OUT_DIR, MANIFEST_PATH = saved
+ DEFAULT_SRC_DIR, DEFAULT_OUT_DIR, MANIFEST_PATH, IMAGE_ROOT = saved
return results