Scrapy, a fast high-level web crawling & scraping framework for Python.
Commits on master
Revert "expose allowed_status tuple for media pipeline"
72fbb68 Paul Tremberth committed 9 years ago
add tests for media pipeline MEDIA_ALLOW_REDIRECTS and MEDIA_HTTPSTATUS_LIST settings
6a42214 Bernardas committed 10 years ago
add description for media pipeline MEDIA_ALLOW_REDIRECTS and MEDIA_HTTPSTATUS_LIST settings
25ed491 Bernardas committed 10 years ago
add media pipeline settings to enable redirection and handling of certain http statuses
5d00584 Bernardas committed 10 years ago
Moved cache dir logging to `open_spider` in FilesystemCacheStorage for consistency
5e89db5 jorenham committed 9 years ago
Log full cache file path instead of cache directory for the storages that cache to single files.
42b429d jorenham committed 9 years ago
Merge pull request #2306 from redapple/referrer-policy
7e8453c Mikhail Korobov committed 9 years ago
Remove Legacy Policy which is equivalent to UnsafeUrl Policy
db176f8 Paul Tremberth committed 9 years ago
Logging the cache directory at HttpCacheMiddleware instantiation #2604
97d84d9 jorenham committed 9 years ago
Add tests for policy fallback on unknown policies from meta and headers
efa5003 Paul Tremberth committed 9 years ago