Scrapy, a fast high-level web crawling & scraping framework for Python.
Merge pull request #2410 from redapple/fetch-transparent-redirect
Elias Dornelesd09ec3db6871
diff --cc docs/topics/shell.rstindex da91108b2,6eb81a71f..ef6aeeed3--- a/docs/topics/shell.rst+++ b/docs/topics/shell.rst@@@ -173,23 -180,12 +180,12 @@@ all start with the ``[s]`` prefix):After that, we can start playing with the objects::>>> response.xpath('//title/text()').extract_first()- u'Scrapy | A Fast and Powerful Scraping and Web Crawling Framework'+ 'Scrapy | A Fast and Powerful Scraping and Web Crawling Framework'>>> fetch("http://reddit.com")- [s] Available Scrapy objects:- [s] crawler <scrapy.crawler.Crawler object at 0x7fb3ed9c9c90>- [s] item {}- [s] request <GET http://reddit.com>- [s] response <200 https://www.reddit.com/>- [s] settings <scrapy.settings.Settings object at 0x7fb3ed9c9c10>- [s] spider <DefaultSpider 'default' at 0x7fb3ecdd3390>- [s] Useful shortcuts:- [s] shelp() Shell help (print this help)- [s] fetch(req_or_url) Fetch request (or URL) and update local objects- [s] view(response) View response in a browser>>> response.xpath('//title/text()').extract()- [u'reddit: the front page of the internet']+ ['reddit: the front page of the internet']>>> request = request.replace(method="POST")