Skip to content

Scrapy, a fast high-level web crawling & scraping framework for Python.

More quick doc fixes.

Andrey Rakhmatullin27f5f3513437
diff --git a/docs/topics/addons.rst b/docs/topics/addons.rst
index 25d7da50b..523f6e86e 100644
--- a/docs/topics/addons.rst
+++ b/docs/topics/addons.rst
@@ -40,10 +40,10 @@ This is an example where an internal add-on and two third-party add-ons (in this
case with one requiring no configuration) are enabled/configured in a project's
``settings.py``::
- ADDONS = (
- 'httpcache',
- 'path.to.some.addon',
- )
+ ADDONS = {
+ 'httpcache': 0,
+ 'path.to.some.addon': 0,
+ }
HTTPCACHE = {
'expiration_secs': 60,
@@ -385,126 +385,3 @@ Forward to other add-ons depending on Python version::
_addon = 'path.to.addon'
else:
_addon = 'path.to.other.addon'
-
-
-Built-in add-on reference
-=========================
-
-Scrapy comes with gateway add-ons that you can use to configure the built-in
-middlewares and extensions. For example, to activate and configure the
-:class:`~scrapy.downloadermiddlewares.httpcache.HttpCacheMiddleware`, instead
-of placing this in your ``settings.py``::
-
- HTTPCACHE_ENABLED = True
- HTTPCACHE_EXPIRATION_SECS = 60
- HTTPCACHE_IGNORE_HTTP_CODES = [404]
-
-you can also use the add-on framework::
-
- ADDONS = (
- # ...,
- 'httpcache',
- )
-
- HTTPCACHE = {
- 'expiration_secs': 60,
- 'ignore_http_codes': [404],
- }
-
-Note that you *must* enable built-in addons by placing them in your
-``ADDONS`` setting before you can use them for configuring built-in
-components. I.e., configuring the ``HTTPCACHE`` setting will have no effect
-when ``httpcache`` is not listed in ``ADDONS``.
-
-In general, the add-on names match the lowercase name of the component, with its
-type suffix removed (i.e. the add-on configuring the
-:class:`~scrapy.downloadermiddlewares.httpcache.HttpCacheMiddleware` is called
-``httpcache``), and the configuration option names match the names of the
-settings they map to, with the component prefix removed (i.e.
-``expiration_secs`` maps to :setting:`HTTPCACHE_EXPIRATION_SECS`, as above).
-The available add-ons are:
-
-
-+--------------------------------------+--------------------------------------+
-| Add-on | Notes |
-+======================================+======================================+
-| **Spider middlewares** |
-+--------------------------------------+--------------------------------------+
-| depth (:class:`~scrapy.spidermi\ | |
-| ddlewares.depth.DepthMiddleware`) | |
-+--------------------------------------+--------------------------------------+
-| httperror (:class:`~scrapy.spid\ | |
-| ermiddlewares.httperror.HttpErrorMi\ | |
-| ddleware`) | |
-+--------------------------------------+--------------------------------------+
-| offsite (:class:`~scrapy.spid\ | |
-| ermiddlewares.offsite.OffsiteMiddle\ | |
-| ware`) | |
-+--------------------------------------+--------------------------------------+
-| referer (:class:`~scrapy.spid\ | |
-| ermiddlewares.referer.RefererMiddle\ | |
-| ware`) | |
-+--------------------------------------+--------------------------------------+
-| urllength (:class:`~scrapy.spid\ | |
-| ermiddlewares.urllength.UrlLengthMi\ | |
-| ddleware`) | |
-+--------------------------------------+--------------------------------------+
-| **Downloader middlewares** |
-+--------------------------------------+--------------------------------------+
-| ajaxcrawl (:class:`~scrapy.download\ | |
-| ermiddlewares.ajaxcrawl.AjaxCrawlMi\ | |
-| ddleware`) | |
-+--------------------------------------+--------------------------------------+
-| chunked (:class:`~scrapy.download\ | |
-| ermiddlewares.chunked.ChunkedTrans\ | |
-| ferMiddleware`) | |
-+--------------------------------------+--------------------------------------+
-| cookies (:class:`~scrapy.download\ | |
-| ermiddlewares.cookies.CookiesMiddle\ | |
-| ware`) | |
-+--------------------------------------+--------------------------------------+
-| defaultheaders (:class:`~scrapy.down\| Every configuration entry is treated |
-| loadermiddlewares.defaultheaders.Def\| as a default header. |
-| aultHeadersMiddleware`) | |
-+--------------------------------------+--------------------------------------+
-| **Extensions** |
-+--------------------------------------+--------------------------------------+
-| autothrottle | Installing sets |
-| (:ref:`topics-autothrottle`) | :setting:`AUTOTHROTTLE_ENABLED` to |
-| | ``True``. |
-+--------------------------------------+--------------------------------------+
-| corestats (:class:`~scrapy.exten\ | |
-| sions.corestats.CoreStats`) | |
-+--------------------------------------+--------------------------------------+
-| closespider (:class:`~scrapy.exten\ | |
-| sions.closespider.CloseSpider`) | |
-+--------------------------------------+--------------------------------------+
-| debugger (:class:`~scrapy.exten\ | |
-| sions.debug.Debugger`) | |
-+--------------------------------------+--------------------------------------+
-| feedexport (:ref:`topics-feed-expor\ | |
-| ts`) | |
-+--------------------------------------+--------------------------------------+
-| logstats (:class:`~scrapy.exten\ | |
-| sions.logstats.LogStats`) | |
-+--------------------------------------+--------------------------------------+
-| memdebug (:class:`~scrapy.exten\ | Installing sets |
-| sions.memdebug.MemoryDebugger`) | :setting:`MEMDEBUG_ENABLED` to |
-| | ``True``. |
-+--------------------------------------+--------------------------------------+
-| memusage (:class:`~scrapy.exten\ | Installing sets |
-| sions.memusage.MemoryUsage`) | :setting:`MEMUSAGE_ENABLED` to |
-| | ``True``. |
-+--------------------------------------+--------------------------------------+
-| spiderstate (:class:`~scrapy.exten\ | |
-| sions.spiderstate.SpiderState`) | |
-+--------------------------------------+--------------------------------------+
-| stacktracedump (:class:`~scrapy.ext\ | |
-| ensions.debug.StackTraceDump`) | |
-+--------------------------------------+--------------------------------------+
-| statsmailer (:class:`~scrapy.exten\ | |
-| sions.statsmailer.StatsMailer`) | |
-+--------------------------------------+--------------------------------------+
-| telnetconsole (:ref:`topics-telnet\ | |
-| console`) | |
-+--------------------------------------+--------------------------------------+
diff --git a/docs/topics/settings.rst b/docs/topics/settings.rst
index aa09abbd5..143002360 100644
--- a/docs/topics/settings.rst
+++ b/docs/topics/settings.rst
@@ -206,10 +206,10 @@ component must be enabled in order for the setting to have any effect.
ADDONS
------
-Default: ``()``
+Default: ``{}``
-A tuple containing paths to the add-ons enabled in your project. For more
-information, see :ref:`topics-addons`.
+A dict containing paths to the add-ons enabled in your project and their
+priorities. For more information, see :ref:`topics-addons`.
.. setting:: AWS_ACCESS_KEY_ID