Scrapy, a fast high-level web crawling & scraping framework for Python.
Commits on master
fix make_requests_from_url deprcation implementation, add tests
a1e8a85 Mikhail Korobov committed 9 years ago
deprecate Spider.make_requests_from_url. Fixes #1495.
692975a Mikhail Korobov committed 10 years ago
use w3lib.html.strip_html5_whitespace function; expand docs; strip consistently before calling process_value
d09eed7 Mikhail Korobov committed 9 years ago
Strip leading/trailing whitespaces in link extractors. Fixes GH-838.
d079e15 Mikhail Korobov committed 9 years ago
RegexLinkExtractor: add \x0c to whitespace characters, as per html5 standard
ad36a4a Mikhail Korobov committed 9 years ago
FormRequest: handle whitespaces in action attribute properly
074caf4 Mikhail Korobov committed 9 years ago
DOC document response.follow methods; expand the tutorial
5b79c6a Mikhail Korobov committed 9 years ago
strip URL extracted from selectors (as per html5 standard)
71dd5d0 Mikhail Korobov committed 9 years ago
Merge pull request #2530 from elacuesta/proxy_credentials
afac3fd Mikhail Korobov committed 9 years ago
Separate building request from _requests_to_follow in CrawlSpider
5b31dfe terut committed 9 years ago