Scrapy, a fast high-level web crawling & scraping framework for Python.
DOC more Python 3 in examples
Mikhail Korobov0ccead6681c9
diff --git a/docs/topics/commands.rst b/docs/topics/commands.rstindex 3088017cb..ef9c45196 100644--- a/docs/topics/commands.rst+++ b/docs/topics/commands.rst@@ -458,9 +458,9 @@ Usage example::>>> STATUS DEPTH LEVEL 1 <<<# Scraped Items ------------------------------------------------------------- [{'name': u'Example item',- 'category': u'Furniture',- 'length': u'12 cm'}]+ [{'name': 'Example item',+ 'category': 'Furniture',+ 'length': '12 cm'}]# Requests -----------------------------------------------------------------[]diff --git a/docs/topics/items.rst b/docs/topics/items.rstindex 4423bbda2..ae44aecd3 100644--- a/docs/topics/items.rst+++ b/docs/topics/items.rst@@ -86,7 +86,7 @@ Creating items::>>> product = Product(name='Desktop PC', price=1000)- >>> print product+ >>> print(product)Product(name='Desktop PC', price=1000)Getting field values@@ -161,11 +161,11 @@ Other common tasksCopying items::>>> product2 = Product(product)- >>> print product2+ >>> print(product2)Product(name='Desktop PC', price=1000)>>> product3 = product2.copy()- >>> print product3+ >>> print(product3)Product(name='Desktop PC', price=1000)Creating dicts from items::diff --git a/docs/topics/jobs.rst b/docs/topics/jobs.rstindex 06c7fff3d..8e1574376 100644--- a/docs/topics/jobs.rst+++ b/docs/topics/jobs.rst@@ -84,7 +84,7 @@ So, for example, this won't work::return scrapy.Request('http://www.example.com', callback=lambda r: self.other_callback(r, somearg))def other_callback(self, response, somearg):- print "the argument passed is:", somearg+ print("the argument passed is: %s" % somearg)But this will::@@ -94,7 +94,7 @@ But this will::def other_callback(self, response):somearg = response.meta['somearg']- print "the argument passed is:", somearg+ print("the argument passed is: %s" % somearg)If you wish to log the requests that couldn't be serialized, you can set the:setting:`SCHEDULER_DEBUG` setting to ``True`` in the project's settings page.diff --git a/docs/topics/loaders.rst b/docs/topics/loaders.rstindex a895b535c..f3b6aa4a1 100644--- a/docs/topics/loaders.rst+++ b/docs/topics/loaders.rst@@ -678,10 +678,10 @@ Here is a list of all built-in processors:>>> from scrapy.loader.processors import Join>>> proc = Join()>>> proc(['one', 'two', 'three'])- u'one two three'+ 'one two three'>>> proc = Join('<br>')>>> proc(['one', 'two', 'three'])- u'one<br>two<br>three'+ 'one<br>two<br>three'.. class:: Compose(\*functions, \**default_loader_context)@@ -744,9 +744,9 @@ Here is a list of all built-in processors:... return None if x == 'world' else x...>>> from scrapy.loader.processors import MapCompose- >>> proc = MapCompose(filter_world, unicode.upper)- >>> proc([u'hello', u'world', u'this', u'is', u'scrapy'])- [u'HELLO, u'THIS', u'IS', u'SCRAPY']+ >>> proc = MapCompose(filter_world, str.upper)+ >>> proc(['hello', 'world', 'this', 'is', 'scrapy'])+ ['HELLO, 'THIS', 'IS', 'SCRAPY']As with the Compose processor, functions can receive Loader contexts, andconstructor keyword arguments are used as default context values. See@@ -772,7 +772,7 @@ Here is a list of all built-in processors:>>> import json>>> proc_single_json_str = Compose(json.loads, SelectJmes("foo"))>>> proc_single_json_str('{"foo": "bar"}')- u'bar'+ 'bar'>>> proc_json_list = Compose(json.loads, MapCompose(SelectJmes('foo')))>>> proc_json_list('[{"foo":"bar"}, {"baz":"tar"}]')- [u'bar']+ ['bar']diff --git a/docs/topics/selectors.rst b/docs/topics/selectors.rstindex 8ac40c3cc..25c1f0aab 100644--- a/docs/topics/selectors.rst+++ b/docs/topics/selectors.rst@@ -235,17 +235,17 @@ Here's an example used to extract image names from the :ref:`HTML code<topics-selectors-htmlcode>` above::>>> response.xpath('//a[contains(@href, "image")]/text()').re(r'Name:\s*(.*)')- [u'My image 1',- u'My image 2',- u'My image 3',- u'My image 4',- u'My image 5']+ ['My image 1',+ 'My image 2',+ 'My image 3',+ 'My image 4',+ 'My image 5']There's an additional helper reciprocating ``.extract_first()`` for ``.re()``,named ``.re_first()``. Use it to extract just the first matching string::>>> response.xpath('//a[contains(@href, "image")]/text()').re_first(r'Name:\s*(.*)')- u'My image 1'+ 'My image 1'.. _topics-selectors-relative-xpaths:@@ -431,26 +431,26 @@ with groups of itemscopes and corresponding itemprops::... print " properties:", props.extract()- current scope: [u'http://schema.org/Product']- properties: [u'name', u'aggregateRating', u'offers', u'description', u'review', u'review']+ current scope: ['http://schema.org/Product']+ properties: ['name', 'aggregateRating', 'offers', 'description', 'review', 'review']- current scope: [u'http://schema.org/AggregateRating']- properties: [u'ratingValue', u'reviewCount']+ current scope: ['http://schema.org/AggregateRating']+ properties: ['ratingValue', 'reviewCount']- current scope: [u'http://schema.org/Offer']- properties: [u'price', u'availability']+ current scope: ['http://schema.org/Offer']+ properties: ['price', 'availability']- current scope: [u'http://schema.org/Review']- properties: [u'name', u'author', u'datePublished', u'reviewRating', u'description']+ current scope: ['http://schema.org/Review']+ properties: ['name', 'author', 'datePublished', 'reviewRating', 'description']- current scope: [u'http://schema.org/Rating']- properties: [u'worstRating', u'ratingValue', u'bestRating']+ current scope: ['http://schema.org/Rating']+ properties: ['worstRating', 'ratingValue', 'bestRating']- current scope: [u'http://schema.org/Review']- properties: [u'name', u'author', u'datePublished', u'reviewRating', u'description']+ current scope: ['http://schema.org/Review']+ properties: ['name', 'author', 'datePublished', 'reviewRating', 'description']- current scope: [u'http://schema.org/Rating']- properties: [u'worstRating', u'ratingValue', u'bestRating']+ current scope: ['http://schema.org/Rating']+ properties: ['worstRating', 'ratingValue', 'bestRating']>>>@@ -543,22 +543,22 @@ Example::This gets all first ``<li>`` elements under whatever it is its parent::>>> xp("//li[1]")- [u'<li>1</li>', u'<li>4</li>']+ ['<li>1</li>', '<li>4</li>']And this gets the first ``<li>`` element in the whole document::>>> xp("(//li)[1]")- [u'<li>1</li>']+ ['<li>1</li>']This gets all first ``<li>`` elements under an ``<ul>`` parent::>>> xp("//ul/li[1]")- [u'<li>1</li>', u'<li>4</li>']+ ['<li>1</li>', '<li>4</li>']And this gets the first ``<li>`` element under an ``<ul>`` parent in the whole document::>>> xp("(//ul/li)[1]")- [u'<li>1</li>']+ ['<li>1</li>']When querying by class, consider using CSS~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~diff --git a/docs/topics/settings.rst b/docs/topics/settings.rstindex 1f1217770..47b6cf13d 100644--- a/docs/topics/settings.rst+++ b/docs/topics/settings.rst@@ -871,7 +871,7 @@ LOG_STDOUTDefault: ``False``If ``True``, all standard output (and error) of your process will be redirected-to the log. For example if you ``print 'hello'`` it will appear in the Scrapy+to the log. For example if you ``print('hello')`` it will appear in the Scrapylog... setting:: LOG_SHORT_NAMES