Skip to content

Scrapy, a fast high-level web crawling & scraping framework for Python.

Remove ChunkedTransferMiddleware from default settings

Paul Tremberth03cf5f1bd201
diff --git a/docs/topics/downloader-middleware.rst b/docs/topics/downloader-middleware.rst
index 15069e56e..dca5ec6a0 100644
--- a/docs/topics/downloader-middleware.rst
+++ b/docs/topics/downloader-middleware.rst
@@ -665,7 +665,13 @@ ChunkedTransferMiddleware
.. class:: ChunkedTransferMiddleware
- This middleware adds support for `chunked transfer encoding`_
+ This middleware adds support for `chunked transfer encoding`_.
+
+.. note::
+ This middleware is not enabled nor used by Scrapy downloader anymore.
+ In fact, Scrapy downloader has built-in support for chunked transfers,
+ so this middleware has no effect in practice.
+
HttpProxyMiddleware
-------------------
diff --git a/docs/topics/settings.rst b/docs/topics/settings.rst
index 8540308fe..2195e4233 100644
--- a/docs/topics/settings.rst
+++ b/docs/topics/settings.rst
@@ -468,7 +468,6 @@ Default::
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware': 600,
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware': 700,
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware': 750,
- 'scrapy.downloadermiddlewares.chunked.ChunkedTransferMiddleware': 830,
'scrapy.downloadermiddlewares.stats.DownloaderStats': 850,
'scrapy.downloadermiddlewares.httpcache.HttpCacheMiddleware': 900,
}
diff --git a/scrapy/settings/default_settings.py b/scrapy/settings/default_settings.py
index 18d5ebbbb..61f4bd567 100644
--- a/scrapy/settings/default_settings.py
+++ b/scrapy/settings/default_settings.py
@@ -102,7 +102,6 @@ DOWNLOADER_MIDDLEWARES_BASE = {
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware': 600,
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware': 700,
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware': 750,
- 'scrapy.downloadermiddlewares.chunked.ChunkedTransferMiddleware': 830,
'scrapy.downloadermiddlewares.stats.DownloaderStats': 850,
'scrapy.downloadermiddlewares.httpcache.HttpCacheMiddleware': 900,
# Downloader side