Scrapy, a fast high-level web crawling & scraping framework for Python.
Improve pre-crawler setting docs (#7835)
Adrian434fd1154ad2
diff --git a/docs/topics/commands.rst b/docs/topics/commands.rstindex ee2c3a3cd..e8a843e80 100644--- a/docs/topics/commands.rst+++ b/docs/topics/commands.rst@@ -665,6 +665,8 @@ Example:COMMANDS_MODULE = "mybot.commands"+.. note:: This is a :ref:`pre-crawler setting <pre-crawler-settings>`.+.. _Deploying your project: https://scrapyd.readthedocs.io/en/latest/deploy.htmlRegister commands via setup.py entry pointsdiff --git a/docs/topics/settings.rst b/docs/topics/settings.rstindex 1b6851d04..81055afc2 100644--- a/docs/topics/settings.rst+++ b/docs/topics/settings.rst@@ -305,10 +305,21 @@ These settings cannot be :ref:`set from a spider <spider-settings>`.These settings are:-- :setting:`TWISTED_REACTOR_ENABLED`+- :setting:`ADDONS`+- :setting:`COMMANDS_MODULE`+- :setting:`FORCE_CRAWLER_PROCESS`- :setting:`SPIDER_LOADER_CLASS` and settings used by the correspondingspider loader class, e.g. :setting:`SPIDER_MODULES` and:setting:`SPIDER_LOADER_WARN_ONLY` for the default spider loader class.+- :setting:`TWISTED_REACTOR_ENABLED`++:setting:`ADDONS` is a special case: it can be set from a spider, but the+``update_pre_crawler_settings()`` method of :ref:`add-ons <topics-addons>`+enabled that way is not called.++:setting:`TWISTED_REACTOR` also acts as a pre-crawler setting when running a+:ref:`command that needs a CrawlerProcess <topics-commands-crawlerprocess>`,+since its project-level value determines the crawler process class... _reactor-settings:@@ -409,6 +420,9 @@ Default: ``{}``A dict containing paths to the add-ons enabled in your project and theirpriorities. For more information, see :ref:`topics-addons`.+.. note:: This is a :ref:`pre-crawler setting <pre-crawler-settings>`, with a+ caveat described in that section.+.. setting:: ASYNCIO_EVENT_LOOPASYNCIO_EVENT_LOOP@@ -1402,6 +1416,8 @@ When :setting:`TWISTED_REACTOR_ENABLED` is set to ``False``,Set this to ``True`` if you want to set :setting:`TWISTED_REACTOR` to anon-default value in :ref:`per-spider settings <spider-settings>`.+.. note:: This is a :ref:`pre-crawler setting <pre-crawler-settings>`.+.. setting:: FTP_PASSIVE_MODEFTP_PASSIVE_MODE