Scrapy, a fast high-level web crawling & scraping framework for Python.
Clarify the docs about reactor settings (#7880)
Adrianfc082aa914cf
diff --git a/docs/topics/settings.rst b/docs/topics/settings.rstindex de665f42e..de74ad5cf 100644--- a/docs/topics/settings.rst+++ b/docs/topics/settings.rst@@ -69,9 +69,10 @@ Example::precedence and override the project ones... note:: :ref:`Pre-crawler settings <pre-crawler-settings>` cannot be defined- per spider, and :ref:`reactor settings <reactor-settings>` should not have- a different value per spider when :ref:`running multiple spiders in the- same process <run-multiple-spiders>`.+ per spider, and :ref:`reactor settings <reactor-settings>` and+ :ref:`logging settings <logging-settings>` are subject to restrictions when+ :ref:`running multiple spiders in the same process+ <run-multiple-spiders>`.One way to do so is by setting their :attr:`~scrapy.Spider.custom_settings`attribute:@@ -329,32 +330,41 @@ Reactor settings**Reactor settings** are settings tied to the :doc:`Twisted reactor<twisted:core/howto/reactor-basics>`.-These settings can be defined from a spider. However, because only 1 reactor-can be used per process, these settings cannot use a different value per spider-when :ref:`running multiple spiders in the same process-<run-multiple-spiders>`.--In general, if different spiders define different values, the first defined-value is used. However, if two spiders request a different reactor, an-exception is raised.+Because only 1 reactor can be used per process, these settings cannot use a+different value per spider when :ref:`running multiple spiders in the same+process <run-multiple-spiders>`.-These settings are:+These settings are used upon installing the reactor:- :setting:`ASYNCIO_EVENT_LOOP` (not possible to set per-spider when using:class:`~scrapy.crawler.AsyncCrawlerProcess`, see below)+- :setting:`TWISTED_REACTOR` (ignored when using+ :class:`~scrapy.crawler.AsyncCrawlerProcess`, see below)++They can be :ref:`set from a spider <spider-settings>`, but only the values+from the first spider that runs are used, since that is when the reactor is+installed. If a later spider asks for a different reactor or a different event+loop, an exception is raised. With+:class:`~scrapy.crawler.CrawlerRunner` and+:class:`~scrapy.crawler.AsyncCrawlerRunner` the reactor must be installed+beforehand, so these settings are only used to check that the installed reactor+and event loop match them.++These settings are applied when starting the reactor:+- :setting:`TWISTED_DNS_RESOLVER` and settings used by the correspondingcomponent, e.g. :setting:`DNSCACHE_ENABLED`, :setting:`DNSCACHE_SIZE`and :setting:`DNS_TIMEOUT` for the default one.- :setting:`REACTOR_THREADPOOL_MAXSIZE`-- :setting:`TWISTED_REACTOR` (ignored when using- :class:`~scrapy.crawler.AsyncCrawlerProcess`, see below)--:setting:`ASYNCIO_EVENT_LOOP` and :setting:`TWISTED_REACTOR` are used upon-installing the reactor. The rest of the settings are applied when starting-the reactor.+They are read from the settings of the+:class:`~scrapy.crawler.CrawlerProcess` or+:class:`~scrapy.crawler.AsyncCrawlerProcess` object, so setting them from a+spider or an :ref:`add-on <topics-addons>` has no effect. They are ignored+altogether when using :class:`~scrapy.crawler.CrawlerRunner` or+:class:`~scrapy.crawler.AsyncCrawlerRunner`, which do not start the reactor.There is an additional restriction for :setting:`TWISTED_REACTOR` and:setting:`ASYNCIO_EVENT_LOOP` when using