Skip to content

Scrapy, a fast high-level web crawling & scraping framework for Python.

Add more docs for update_settings().

Andrey Rakhmatullin72462a53e2cf
diff --git a/docs/topics/settings.rst b/docs/topics/settings.rst
index 219509c1e..d0f2acd89 100644
--- a/docs/topics/settings.rst
+++ b/docs/topics/settings.rst
@@ -66,8 +66,8 @@ Example::
----------------------
Spiders (See the :ref:`topics-spiders` chapter for reference) can define their
-own settings that will take precedence and override the project ones. They can
-do so by setting their :attr:`~scrapy.Spider.custom_settings` attribute:
+own settings that will take precedence and override the project ones. One way
+to do so is by setting their :attr:`~scrapy.Spider.custom_settings` attribute:
.. code-block:: python
@@ -81,6 +81,22 @@ do so by setting their :attr:`~scrapy.Spider.custom_settings` attribute:
"SOME_SETTING": "some value",
}
+It's often better to provide a :meth:`~scrapy.Spider.update_settings` instead,
+and settings set there should use the "spider" priority explicitly:
+
+.. code-block:: python
+
+ import scrapy
+
+
+ class MySpider(scrapy.Spider):
+ name = "myspider"
+
+ @classmethod
+ def update_settings(cls, settings):
+ settings.set("SOME_SETTING", "some value", priority="spider")
+ super().update_settings(settings)
+
3. Project settings module
--------------------------
diff --git a/docs/topics/spiders.rst b/docs/topics/spiders.rst
index 0f6c2b1ba..97b525bd6 100644
--- a/docs/topics/spiders.rst
+++ b/docs/topics/spiders.rst
@@ -151,13 +151,18 @@ scrapy.Spider
and is called during initialization of a spider instance.
It takes a :class:`~scrapy.settings.Settings` object as a parameter and
- can add or update the spider's configuration values. This method is a class method,
- meaning that it is called on the :class:`~scrapy.Spider` class and allows all instances
- of the spider to share the same configuration.
-
- One of the main advantages of ``update_settings()`` is that it allows
- you to dynamically add, remove or change settings based on other settings
- or other external factors.
+ can add or update the spider's configuration values. This method is a
+ class method, meaning that it is called on the :class:`~scrapy.Spider`
+ class and allows all instances of the spider to share the same
+ configuration.
+
+ While per-spider settings can be set in
+ :attr:`~scrapy.Spider.custom_settings`, using ``update_settings()``
+ allows you to dynamically add, remove or change settings based on other
+ settings, spider attributes or other factors and use setting priorities
+ other than ``'spider'``. Also, it's easy to extend ``update_settings()``
+ in a subclass by overriding it, while doing the same with
+ :attr:`~scrapy.Spider.custom_settings` is hard or impossible.
For example, suppose a spider needs to modify :setting:`FEEDS`: