Skip to content

Scrapy, a fast high-level web crawling & scraping framework for Python.

docs: update data flow description and image (fixes: #2278) This fixes the explanation to use Requests instead of URLs, which is what actually happens, and is also consistent with the new tutorial, which already explains how URLs become Request objects.

Elias Dorneles3ac3ac4d925e
diff --git a/docs/topics/_images/scrapy_architecture_02.png b/docs/topics/_images/scrapy_architecture_02.png
index 80be02078..5fe393f22 100644
Binary files a/docs/topics/_images/scrapy_architecture_02.png and b/docs/topics/_images/scrapy_architecture_02.png differ
diff --git a/docs/topics/architecture.rst b/docs/topics/architecture.rst
index 39e54ee99..84c053660 100644
--- a/docs/topics/architecture.rst
+++ b/docs/topics/architecture.rst
@@ -29,20 +29,20 @@ Data flow
The data flow in Scrapy is controlled by the execution engine, and goes like
this:
-1. The :ref:`Engine <component-engine>` gets the first URLs to crawl from the
+1. The :ref:`Engine <component-engine>` gets the first Requests to crawl from the
:ref:`Spider <component-spiders>`.
-2. The :ref:`Engine <component-engine>` schedules the URLs in the
- :ref:`Scheduler <component-scheduler>` as Requests and asks for the
- next URLs to crawl.
+2. The :ref:`Engine <component-engine>` schedules the Requests in the
+ :ref:`Scheduler <component-scheduler>` and asks for the
+ next Requests to crawl.
-3. The :ref:`Scheduler <component-scheduler>` returns the next URLs to crawl
+3. The :ref:`Scheduler <component-scheduler>` returns the next Requests
to the :ref:`Engine <component-engine>`.
-4. The :ref:`Engine <component-engine>` sends the URLs to the
+4. The :ref:`Engine <component-engine>` sends the Requests to the
:ref:`Downloader <component-downloader>`, passing through the
:ref:`Downloader Middleware <component-downloader-middleware>`
- (request direction).
+ (requests direction).
5. Once the page finishes downloading the
:ref:`Downloader <component-downloader>` generates a Response (with
@@ -62,10 +62,11 @@ this:
:ref:`Spider Middleware <component-spider-middleware>` (output direction).
8. The :ref:`Engine <component-engine>` sends processed items to
- :ref:`Item Pipelines <component-pipelines>` and processed Requests to
- the :ref:`Scheduler <component-scheduler>`.
+ :ref:`Item Pipelines <component-pipelines>`, then sends processed Requests to
+ the :ref:`Scheduler <component-scheduler>` and asks for the next
+ Requests to crawl.
-9. The process repeats (from step 1) until there are no more requests from the
+9. The process repeats (from step 2) until there are no more requests from the
:ref:`Scheduler <component-scheduler>`.
Components
@@ -102,7 +103,7 @@ Spiders
-------
Spiders are custom classes written by Scrapy users to parse responses and
-extract items (aka scraped items) from them or additional URLs (requests) to
+extract items (aka scraped items) from them or additional requests to
follow. For more information see :ref:`topics-spiders`.
.. _component-pipelines: