Scrapy, a fast high-level web crawling & scraping framework for Python.
Update docs on accessing callback arguments in errback (#4634)
Devi Sandeep5d541731870e
diff --git a/docs/topics/request-response.rst b/docs/topics/request-response.rstindex bbd715766..d88d40b00 100644--- a/docs/topics/request-response.rst+++ b/docs/topics/request-response.rst@@ -189,6 +189,10 @@ Request objectscloned using the ``copy()`` or ``replace()`` methods, and can also beaccessed, in your spider, from the ``response.cb_kwargs`` attribute.+ In case of a failure to process the request, this dict can be accessed as+ ``failure.request.cb_kwargs`` in the request's errback. For more information,+ see :ref:`topics-request-response-ref-accessing-callback-arguments-in-errback`.+.. method:: Request.copy()Return a new Request which is a copy of this Request. See also:@@ -312,6 +316,31 @@ errors if needed::request = failure.requestself.logger.error('TimeoutError on %s', request.url)+.. _topics-request-response-ref-accessing-callback-arguments-in-errback:++Accessing additional data in errback functions+----------------------------------------------++In case of a failure to process the request, you may be interested in+accessing arguments to the callback functions so you can process further+based on the arguments in the errback. The following example shows how to+achieve this by using ``Failure.request.cb_kwargs``::++ def parse(self, response):+ request = scrapy.Request('http://www.example.com/index.html',+ callback=self.parse_page2,+ errback=self.errback_page2,+ cb_kwargs=dict(main_url=response.url))+ yield request++ def parse_page2(self, response, main_url):+ pass++ def errback_page2(self, failure):+ yield dict(+ main_url=failure.request.cb_kwargs['main_url'],+ )+.. _topics-request-meta:Request.meta special keys