Scrapy, a fast high-level web crawling & scraping framework for Python.
Commits on master
tutorial: updating extracting data section to introduce CSS and XPath equally
31545a9 Elias Dorneles committed 10 years ago
Merge pull request #2249 from scrapy/fix-overview-spider
de1a6ac Elias Dorneles committed 10 years ago
tutorial: update section about following links, expand examples adding an AuthorSpider to demonstrate further a different crawling arrangement.
b2a5cdd Elias Dorneles committed 10 years ago
use harcoded URLs, remove item reference on second spider
c508f40 Elias Dorneles committed 10 years ago
tutorial: remove item class definition and present start_requests first This changes the tutorial, removing the step of creating an item class and also starts by presenting the start_requests method instead of start_urls.
2427791 Elias Dorneles committed 10 years ago
docs: update overview spider code to use toscrape.com and minor changes So, this will replace the spider example code from the overview that scrapes questions from StackOverflow by a spider scraping quotes (much like the one in the tutorial), and upates the text around it to be consistent.
18bd0b0 Elias Dorneles committed 10 years ago
Make scrapy available in shell without explicit import statement
105163f Paul Tremberth committed 10 years ago
Add shell test for using scrapy.Request() directly without importing scrapy
b828fac Paul Tremberth committed 10 years ago
Merge pull request #2236 from stummjr/new-tutorial-toscrape
2f60f2a Paul Tremberth committed 10 years ago
changed tutorial examples from dmoz to quotes.toscrape.com
10f8c52 Valdir Stumm Junior committed 10 years ago
Merge pull request #1503 from demelziraptor/amazon-json-response
129421c Elias Dorneles committed 10 years ago