Scrapy, a fast high-level web crawling & scraping framework for Python.
Add Request.kwargs docs
Eugenio Lacuestabddfeaba4c17
diff --git a/docs/topics/request-response.rst b/docs/topics/request-response.rstindex e29914dbf..d12766676 100644--- a/docs/topics/request-response.rst+++ b/docs/topics/request-response.rst@@ -24,7 +24,7 @@ below in :ref:`topics-request-response-ref-request-subclasses` andRequest objects===============-.. class:: Request(url[, callback, method='GET', headers, body, cookies, meta, encoding='utf-8', priority=0, dont_filter=False, errback, flags])+.. class:: Request(url[, callback, method='GET', headers, body, cookies, meta, encoding='utf-8', priority=0, dont_filter=False, errback, flags, kwargs])A :class:`Request` object represents an HTTP request, which is usuallygenerated in the Spider and executed by the Downloader, and thus generating@@ -126,6 +126,9 @@ Request objects:param flags: Flags sent to the request, can be used for logging or similar purposes.:type flags: list+ :param kwargs: A dict with arbitrary data that will be passed as keyword arguments to the Request's callback.+ :type kwargs: dict+.. attribute:: Request.urlA string containing the URL of this request. Keep in mind that this@@ -165,6 +168,17 @@ Request objects``copy()`` or ``replace()`` methods, and can also be accessed, in yourspider, from the ``response.meta`` attribute.+ .. attribute:: Request.kwargs++ A dictionary that contains arbitrary metadata for this request. Its contents+ will be passed to the Request's callback as keyword arguments. It is empty+ for new Requests, which means by default callbacks only get a :class:`Response`+ object as argument.++ This dict is `shallow copied`_ when the request is cloned using the+ ``copy()`` or ``replace()`` methods, and can also be accessed, in your+ spider, from the ``response.kwargs`` attribute.+.. _shallow copied: https://docs.python.org/2/library/copy.html.. method:: Request.copy()@@ -200,11 +214,9 @@ Example::self.logger.info("Visited %s", response.url)In some cases you may be interested in passing arguments to those callback-functions so you can receive the arguments later, in the second callback. You-can use the :attr:`Request.meta` attribute for that.--Here's an example of how to pass an item using this mechanism, to populate-different fields from different pages::+functions so you can receive the arguments later, in the second callback.+The following two examples show how to achieve this by using the+:attr:`Request.meta` and :attr:`Request.kwargs` attributes respectively::def parse_page1(self, response):item = MyItem()@@ -219,6 +231,22 @@ different fields from different pages::item['other_url'] = response.urlyield item+::++ def parse_page1(self, response):+ item = MyItem()+ item['main_url'] = response.url+ request = scrapy.Request("http://www.example.com/some_page.html",+ callback=self.parse_page2)+ request.kwargs['item'] = item+ request.kwargs['foo'] = 'bar'+ yield request++ def parse_page2(self, response, item, foo):+ item['other_url'] = response.url+ item['foo'] = foo+ yield item+.. _topics-request-response-ref-errbacks: