Scrapy, a fast high-level web crawling & scraping framework for Python.
Commits on master
Type scrapy/settings/default_settings.py, drop mypy override (#8077)
941d7eb Adrian committed 2 months ago
Re-enable tests with DirtyReactorAggregateError comments. (#8073)
6ed70e8 Andrey Rakhmatullin committed 2 months ago
Interleave subprocess-heavy tests to avoid an xdist scheduling straggler (#8039)
4ab7cf3 Adrian committed 2 months ago
Fix open_in_browser ReDoS test flakiness and a broken exploit input (#8064)
2382636 Adrian committed 2 months ago
Document setting request priority in CrawlSpider rules (#8070)
753f3af Adrian committed 2 months ago
Support non-text responses in open_in_browser via Content-Type (#8067)
6073742 Adrian committed 2 months ago
Type the last tests.* modules under check_untyped_defs = false (#8017)
e9830e6 Adrian committed 2 months ago
Suppress parso's DEBUG logging noise in the Scrapy shell (#8060)
3857c69 Adrian committed 3 months ago
docs: document subclassing RedirectMiddleware to allow/deny redirects by target URL (#8061)
8d390db Adrian committed 3 months ago
Document that concurrent spiders get independent crawlers, middlewares, and settings (#8055)
45c837c Adrian committed 3 months ago
Document that get_project_settings() returns project settings only (#8051)
f6b1f54 Adrian committed 3 months ago
Crawl more pages in the download delay tests instead of skipping them (#8027)
a466ced Adrian committed 3 months ago
docs: improve ScrapyCommand documentation for issue #6844 (#6904)
ed6e481 alchemypls committed 3 months ago
CI: Fix concurrency group collision between merged-PR and master-push runs (#8047)
9da1a32 Adrian committed 3 months ago
Warn that XMLFeedSpider's html iterator can mangle void-like tags (#8045)
5938dde Adrian committed 3 months ago
Document where the project name appears in a Scrapy project (#8044)
bbb5a26 Adrian committed 3 months ago
Document deriving the job directory from the spider name (#8035)
ab27dfd Adrian committed 3 months ago