Skip to content

Scrapy, a fast high-level web crawling & scraping framework for Python.

Rule.process_request: docs

Eugenio Lacuestab30ca379b678
diff --git a/docs/topics/spiders.rst b/docs/topics/spiders.rst
index 742a88659..24b6f7ec9 100644
--- a/docs/topics/spiders.rst
+++ b/docs/topics/spiders.rst
@@ -402,10 +402,12 @@ Crawling rules
of links extracted from each response using the specified ``link_extractor``.
This is mainly used for filtering purposes.
- ``process_request`` is a callable, or a string (in which case a method from
- the spider object with that name will be used) which will be called with
- every request extracted by this rule, and must return a request or None (to
- filter out the request).
+ ``process_request`` is a callable (or a string, in which case a method from
+ the spider object with that name will be used) which will be called for
+ every request extracted by this rule. This callable should take a Request object
+ as first positional argument and, optionally, the Response object from which the
+ Request originated as second positional argument. It must return a request or None
+ (to filter out the request).
CrawlSpider example
~~~~~~~~~~~~~~~~~~~