High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
fix: a schema version bump is invisible to the Go tests (PIN-109)
Luigi Agostiea277e478ec4
diff --git a/docs/scrape.md b/docs/scrape.mdindex 7b53a49d..fbe37f8a 100644--- a/docs/scrape.md+++ b/docs/scrape.md@@ -137,6 +137,29 @@ summary: contentTypes, httpPages, browserPages, failedPages, recommendationswithout `--output-dir`): a single digest with the site overview, the page treeby URL pattern, and each page's content (or its snippet in preview mode).+### Schema history++`schemaVersion` is stamped into every report so a consumer can detect format+changes. The current version is **`2.0`**.++**`1.0` → `2.0` (breaking).** `site.totalDiscovered` was **removed** and replaced+by `site.totalURLsInSitemap`.++The number itself did not change — the name did. `totalDiscovered` only ever+counted URLs listed in the site's sitemap, never everything the crawl found, so on+a site with no sitemap it read `0` while pages were plainly being scraped+(`N page(s) discovered of 0`). It was renamed to state what it holds.++Two things to change in a `1.0` reader:++- Read `totalURLsInSitemap` where you read `totalDiscovered`; the value is the+ same measurement.+- It is now `omitempty`, so it is **absent** rather than `0` when the site has no+ sitemap. Pair it with `sitemapFound` rather than treating a missing key as zero.++If what you actually wanted was how many pages the crawl reached, that is+`site.sampledPages` (or the `pages[]` length) — it never was `totalDiscovered`.+## HTTP APIThe CLI is a thin client over one endpoint:diff --git a/internal/audit/types_test.go b/internal/audit/types_test.goindex 928ccb8f..ad6b5fe8 100644--- a/internal/audit/types_test.go+++ b/internal/audit/types_test.go@@ -140,6 +140,29 @@ func TestNewAuditReportSchemaVersion(t *testing.T) {}}+// TestNewAuditReportSchemaVersion above compares the report to the constant that+// stamped it, so it proves the report is stamped and the JSON field is spelled+// right but cannot notice the constant changing. The literal here is the second+// copy that can, and it is the point of the test.+//+// The audit version happens to be rendered into the report goldens, so a bump+// also reds internal/audit/report — but incidentally, and only because those+// goldens compare content: 1.0 and 9.9 are the same width, so a length-only+// golden would miss it. This assertion does not depend on that.+func TestSchemaVersionIsPinnedSoABumpMustBeAcknowledged(t *testing.T) {+ const pinned = "1.0"++ if SchemaVersion != pinned {+ t.Fatalf("audit SchemaVersion = %q, pinned at %q.\n"++ "An audit schema bump changes what report consumers receive. If it is intended, update all three:\n"++ " 1. this literal,\n"++ " 2. the audit report goldens — run `go test ./internal/audit/report -update` and review the diff,\n"++ " 3. schemaVersion in tests/e2e/fixtures/audit-site/golden-report.json.\n"++ "The audit e2e scenarios assert the field EXISTS rather than pinning a value, so there is no EXPECTED_*_SCHEMA to change — unlike scrape.",+ SchemaVersion, pinned)+ }+}+// The audit snapshot path never measures layout (no bounds pass, no on-screen// test), so the report must make no visibility claim at all. The positive// assertions are here because an empty payload would satisfy the negative.diff --git a/internal/scrape/types_test.go b/internal/scrape/types_test.gonew file mode 100644index 00000000..1d2bba5e--- /dev/null+++ b/internal/scrape/types_test.go@@ -0,0 +1,24 @@+package scrape++import "testing"++// The literal here is deliberately a second copy of SchemaVersion, and it is the+// point of the test. Every other assertion compares the report's field to the+// constant that stamped it, so it can prove the report is stamped but never that+// the version is the intended one — a bump passes the whole Go suite and surfaces+// only in the Docker e2e, long after it lands.+//+// This does not check correctness. It forces a bump to be acknowledged, together+// with the sibling artefacts that carry the same number.+func TestSchemaVersionIsPinnedSoABumpMustBeAcknowledged(t *testing.T) {+ const pinned = "2.0"++ if SchemaVersion != pinned {+ t.Fatalf("scrape SchemaVersion = %q, pinned at %q.\n"++ "A scrape schema bump is a breaking change for report consumers. If it is intended, update all three:\n"++ " 1. this literal,\n"++ " 2. the schema history in docs/scrape.md — say which fields changed, as 1.0 -> 2.0 did for totalDiscovered -> totalURLsInSitemap,\n"++ " 3. EXPECTED_SCRAPE_SCHEMA in tests/e2e/scenarios/cli/scrape-basic.sh.",+ SchemaVersion, pinned)+ }+}