High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
test(extract): add CC BY / CC BY-SA attribution for the corpus page copies and correct the Wikinews licence to CC BY 4.0 (PIN-395)
Luigi Agostib992caea0e26
diff --git a/internal/extract/testdata/corpus/README.md b/internal/extract/testdata/corpus/README.mdindex 25f27a5c..3db3278e 100644--- a/internal/extract/testdata/corpus/README.md+++ b/internal/extract/testdata/corpus/README.md@@ -92,6 +92,26 @@ floors in step with this directory, and thepage through the live pipeline, reporting every field and failing an entrywhose hits fall below its floor.-The toscrape pages are Zyte's public scraping sandbox. The Wikinews article is-CC BY 2.5 and the Wikipedia pages CC BY-SA 4.0, both credited to their-contributors at the URLs above.+## Attribution and licences++The snapshots here and the HTML copies under `tests/e2e/fixtures/corpus/` carry+third-party page content, captured on 2026-09-13 from the URLs in the table+above:++- `article`: "Grateful Dead rhythm guitarist Bob Weir dies, aged 78", Wikinews+ contributors, licensed CC BY 4.0+ (https://creativecommons.org/licenses/by/4.0/; the page's own footer puts+ text created after 2024-12-16 under 4.0). Attributed to Wikinews.+- `infobox`: "Mount Twynam", Wikipedia contributors, and `search`: the+ Wikipedia search results page for "headless browser", whose snippets are+ excerpts of Wikipedia articles by Wikipedia contributors; both licensed+ CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/). The copies+ are adaptations and are shared under the same licence.+- `product`, `listing`, `login`: Zyte's public scraping sandboxes+ (books.toscrape.com, quotes.toscrape.com), published for scraping practice.+- `pricing`, `stories`: snapshot only, no HTML copy; see the e2e manifest.++Changes from the originals: the snapshots are reformatted to one node per line;+the HTML copies have scripts, iframes and external resources removed,+stylesheets reduced and inlined, and images replaced by a 1x1 pixel. No text+was edited.diff --git a/tests/e2e/fixtures/corpus/README.md b/tests/e2e/fixtures/corpus/README.mdnew file mode 100644index 00000000..88837cf5--- /dev/null+++ b/tests/e2e/fixtures/corpus/README.md@@ -0,0 +1,23 @@+# e2e extract corpus++Static copies of the pages in `internal/extract/testdata/corpus/`, used by+`tests/e2e/scenarios/api/extract-corpus-extended.sh`. That README describes how+they were made and how `manifest.json` gates them.++## Attribution and licences++All pages captured on 2026-09-13.++| file | source | author / licence |+| -------------- | ---------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- |+| `article.html` | https://en.wikinews.org/wiki/Grateful_Dead_rhythm_guitarist_Bob_Weir_dies,_aged_78 | Wikinews contributors, CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) |+| `infobox.html` | https://en.wikipedia.org/wiki/Mount_Twynam | Wikipedia contributors, CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/) |+| `search.html` | https://en.wikipedia.org/w/index.php?search=headless+browser&title=Special:Search&profile=advanced&fulltext=1&ns0=1 | Wikipedia contributors (article excerpts), CC BY-SA 4.0 |+| `product.html` | https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html | Zyte's public scraping sandbox |+| `listing.html` | https://books.toscrape.com/catalogue/category/books/travel_2/index.html | Zyte's public scraping sandbox |+| `login.html` | https://quotes.toscrape.com/login | Zyte's public scraping sandbox |++Changes from the originals: scripts, iframes and external resources removed,+stylesheets reduced to the matching rules and inlined, images replaced by a+1x1 pixel. No text was edited. The Wikipedia-derived copies are adaptations+shared under CC BY-SA 4.0.