Skip to content

Scrapy, a fast high-level web crawling & scraping framework for Python.

Improve docs about setting settings in addons.

Andrey Rakhmatullina2264d3b8b70
diff --git a/docs/topics/addons.rst b/docs/topics/addons.rst
index f1cc070ad..901a8bf8f 100644
--- a/docs/topics/addons.rst
+++ b/docs/topics/addons.rst
@@ -61,6 +61,14 @@ They can also have the following method:
:param crawler: The crawler that uses this addon
:type crawler: :class:`~scrapy.crawler.Crawler`
+The settings set by the addon should use the ``addon`` priority (see
+:ref:`populating-settings` and :func:`scrapy.settings.BaseSettings.set`). This
+allows users to override these settings in the project or spider configuration.
+This is not possible with settings that are mutable objects, such as the dict
+that is a value of :setting:`ITEM_PIPELINES`. In these cases you can provide an
+addon-specific setting that governs whether the addon will modify
+:setting:`ITEM_PIPELINES`.
+
Add-on examples
===============
@@ -70,7 +78,7 @@ Set some basic configuration::
class MyAddon:
def update_settings(self, settings):
settings["ITEM_PIPELINES"]["path.to.mypipeline"] = 200
- settings["DNSCACHE_ENABLED"] = True
+ settings.set("DNSCACHE_ENABLED", True, "addon")
Check dependencies::
diff --git a/docs/topics/api.rst b/docs/topics/api.rst
index bb46b2b7d..d1a5497fb 100644
--- a/docs/topics/api.rst
+++ b/docs/topics/api.rst
@@ -137,6 +137,7 @@ Settings API
SETTINGS_PRIORITIES = {
"default": 0,
"command": 10,
+ "addon": 15,
"project": 20,
"spider": 30,
"cmdline": 40,
diff --git a/docs/topics/settings.rst b/docs/topics/settings.rst
index 143002360..139e0a35f 100644
--- a/docs/topics/settings.rst
+++ b/docs/topics/settings.rst
@@ -40,8 +40,9 @@ precedence:
1. Command line options (most precedence)
2. Settings per-spider
3. Project settings module
- 4. Default settings per-command
- 5. Default global settings (less precedence)
+ 4. Settings set by addons
+ 5. Default settings per-command
+ 6. Default global settings (less precedence)
The population of these settings sources is taken care of internally, but a
manual handling is possible using API calls. See the
@@ -89,7 +90,13 @@ project, it's where most of your custom settings will be populated. For a
standard Scrapy project, this means you'll be adding or changing the settings
in the ``settings.py`` file created for your project.
-4. Default settings per-command
+4. Settings set by addons
+-------------------------
+
+:ref:`Addons <topics-addons>` can modify settings. They should do this with
+this priority, though this is not enforced.
+
+5. Default settings per-command
-------------------------------
Each :doc:`Scrapy tool </topics/commands>` command can have its own default
@@ -97,7 +104,7 @@ settings, which override the global default settings. Those custom command
settings are specified in the ``default_settings`` attribute of the command
class.
-5. Default global settings
+6. Default global settings
--------------------------
The global defaults are located in the ``scrapy.settings.default_settings``