Scrapy, a fast high-level web crawling & scraping framework for Python.
Commits on master
Add link extractor test for non-ASCII characters in query part of URL
7b5243a Paul Tremberth committed 10 years ago
Fix link extractor tests for non-ASCII characters from latin1 document
1656fbc Paul Tremberth committed 10 years ago
Merge pull request #1891 from djunzu/update_files_images_pipelines
0ede017 Paul Tremberth committed 10 years ago
Merge pull request #1902 from starrify/case-insensitive-robots-txt-for-sitemap
642fedb Paul Tremberth committed 10 years ago
Added: Making it case-insensitive when extracting sitemap URLs from a robots.txt
103f6ea Pengyu CHEN committed 10 years ago
Merge pull request #1847 from aron-bordin/add_blocking_storage_path_setting
bf7f675 Paul Tremberth committed 10 years ago
Change ImagesPipeline class attributes to instance attributes.
5370835 djunzu committed 10 years ago
Merge pull request #1883 from lopuhin/botocore-files-store-fix
9ae4e46 Paul Tremberth committed 10 years ago
Merge pull request #1857 from pawelmhm/fix_response_status_msg
3a763f7 Paul Tremberth committed 10 years ago
Fix handling of meta=None in S3FilesStore.persist_file
5045a4f Konstantin Lopuhin committed 10 years ago
response_status_message should not fail on non-standard HTTP codes
65c7c05 pawelmhm committed 10 years ago
Merge pull request #1848 from aron-bordin/small_doc_style_fixes
ebef6d7 Mikhail Korobov committed 10 years ago