Scrapy, a fast high-level web crawling & scraping framework for Python.
Commits on master
Add tests for stripping userinfo with percent-encoded delimiters
0a0b60a Paul Tremberth committed 10 years ago
Rename helper function to strip_url() + add more tests
8864d0e Paul Tremberth committed 10 years ago
Add tests for referrer-policy set in response HTTP headers
d3d4d66 Paul Tremberth committed 10 years ago
Add tests for referrer policy via settings and via Request meta
e72b6e3 Paul Tremberth committed 10 years ago
Make default referrer policy customizable via settings
842ce13 Paul Tremberth committed 10 years ago