Skip to content

High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.

fix: a schema version bump is invisible to the Go tests (PIN-109)

Luigi Agostiea277e478ec4
diff --git a/docs/scrape.md b/docs/scrape.md
index 7b53a49d..fbe37f8a 100644
--- a/docs/scrape.md
+++ b/docs/scrape.md
@@ -137,6 +137,29 @@ summary: contentTypes, httpPages, browserPages, failedPages, recommendations
without `--output-dir`): a single digest with the site overview, the page tree
by URL pattern, and each page's content (or its snippet in preview mode).
+### Schema history
+
+`schemaVersion` is stamped into every report so a consumer can detect format
+changes. The current version is **`2.0`**.
+
+**`1.0` → `2.0` (breaking).** `site.totalDiscovered` was **removed** and replaced
+by `site.totalURLsInSitemap`.
+
+The number itself did not change — the name did. `totalDiscovered` only ever
+counted URLs listed in the site's sitemap, never everything the crawl found, so on
+a site with no sitemap it read `0` while pages were plainly being scraped
+(`N page(s) discovered of 0`). It was renamed to state what it holds.
+
+Two things to change in a `1.0` reader:
+
+- Read `totalURLsInSitemap` where you read `totalDiscovered`; the value is the
+ same measurement.
+- It is now `omitempty`, so it is **absent** rather than `0` when the site has no
+ sitemap. Pair it with `sitemapFound` rather than treating a missing key as zero.
+
+If what you actually wanted was how many pages the crawl reached, that is
+`site.sampledPages` (or the `pages[]` length) — it never was `totalDiscovered`.
+
## HTTP API
The CLI is a thin client over one endpoint:
diff --git a/internal/audit/types_test.go b/internal/audit/types_test.go
index 928ccb8f..ad6b5fe8 100644
--- a/internal/audit/types_test.go
+++ b/internal/audit/types_test.go
@@ -140,6 +140,29 @@ func TestNewAuditReportSchemaVersion(t *testing.T) {
}
}
+// TestNewAuditReportSchemaVersion above compares the report to the constant that
+// stamped it, so it proves the report is stamped and the JSON field is spelled
+// right but cannot notice the constant changing. The literal here is the second
+// copy that can, and it is the point of the test.
+//
+// The audit version happens to be rendered into the report goldens, so a bump
+// also reds internal/audit/report — but incidentally, and only because those
+// goldens compare content: 1.0 and 9.9 are the same width, so a length-only
+// golden would miss it. This assertion does not depend on that.
+func TestSchemaVersionIsPinnedSoABumpMustBeAcknowledged(t *testing.T) {
+ const pinned = "1.0"
+
+ if SchemaVersion != pinned {
+ t.Fatalf("audit SchemaVersion = %q, pinned at %q.\n"+
+ "An audit schema bump changes what report consumers receive. If it is intended, update all three:\n"+
+ " 1. this literal,\n"+
+ " 2. the audit report goldens — run `go test ./internal/audit/report -update` and review the diff,\n"+
+ " 3. schemaVersion in tests/e2e/fixtures/audit-site/golden-report.json.\n"+
+ "The audit e2e scenarios assert the field EXISTS rather than pinning a value, so there is no EXPECTED_*_SCHEMA to change — unlike scrape.",
+ SchemaVersion, pinned)
+ }
+}
+
// The audit snapshot path never measures layout (no bounds pass, no on-screen
// test), so the report must make no visibility claim at all. The positive
// assertions are here because an empty payload would satisfy the negative.
diff --git a/internal/scrape/types_test.go b/internal/scrape/types_test.go
new file mode 100644
index 00000000..1d2bba5e
--- /dev/null
+++ b/internal/scrape/types_test.go
@@ -0,0 +1,24 @@
+package scrape
+
+import "testing"
+
+// The literal here is deliberately a second copy of SchemaVersion, and it is the
+// point of the test. Every other assertion compares the report's field to the
+// constant that stamped it, so it can prove the report is stamped but never that
+// the version is the intended one — a bump passes the whole Go suite and surfaces
+// only in the Docker e2e, long after it lands.
+//
+// This does not check correctness. It forces a bump to be acknowledged, together
+// with the sibling artefacts that carry the same number.
+func TestSchemaVersionIsPinnedSoABumpMustBeAcknowledged(t *testing.T) {
+ const pinned = "2.0"
+
+ if SchemaVersion != pinned {
+ t.Fatalf("scrape SchemaVersion = %q, pinned at %q.\n"+
+ "A scrape schema bump is a breaking change for report consumers. If it is intended, update all three:\n"+
+ " 1. this literal,\n"+
+ " 2. the schema history in docs/scrape.md — say which fields changed, as 1.0 -> 2.0 did for totalDiscovered -> totalURLsInSitemap,\n"+
+ " 3. EXPECTED_SCRAPE_SCHEMA in tests/e2e/scenarios/cli/scrape-basic.sh.",
+ SchemaVersion, pinned)
+ }
+}