Scrapy, a fast high-level web crawling & scraping framework for Python.
Commits on master
chore: Update outdated GitHub Actions versions (#7223)
49930df Pádraic Slattery committed 10 months ago
Fix omitting repeated dataloss warnings in HTTP11DownloadHandler. (#7222)
3ec6ae0 Andrey Rakhmatullin committed 10 months ago
Remove the non-standard 307/308 handling, and align other aspects with the standard
ba3d7bc Adrian Chaves committed 10 months ago
Lazy creation of Downloader._slot_gc_loop. (#7210)
9bae1ee Andrey Rakhmatullin committed 10 months ago
Remove body on 307/308 redirects and switch to GET on 301
f7f1812 Adrian Chaves committed 10 months ago
Fix arbitrary callable execution from the Referrer-Policy response header
b6e5c58 Adrian Chaves committed 10 months ago
(#7180) Update codecov test results action to v5 on all CI workflows. (#7215)
e30ba7d Santiago Beltrán Ruiz committed 10 months ago
Pass the spider arg to custom stat collectors {open,close}_spider(). (#7213)
0f07b2e Andrey Rakhmatullin committed 10 months ago
Allow Crawler.crawl_async() without asyncio. (#7202)
a246332 Andrey Rakhmatullin committed 10 months ago
Remove outdated version marker directives (< 2.8.0) from documentation (#7198)
9381ad8 Adham 3llam committed 11 months ago
Deprecate returning Deferreds from pipeline methods (#7179)
180ca39 Andrey Rakhmatullin committed 11 months ago
Restore and deprecate the spider arg of Downloader._get_slot(). (#7178)
5a7e132 Andrey Rakhmatullin committed 11 months ago
Refactor MediaPipeline.process_item() to async def (#7177)
c49ae21 Andrey Rakhmatullin committed 11 months ago
Remove docs mentioning deprecated spider attrs. (#7176)
588f3d4 Andrey Rakhmatullin committed 11 months ago
Change default priority queue to DownloaderAwarePriorityQueue (#6940)
d8583a8 Thalison Fernandes committed 11 months ago