Skip to content

Scrapy, a fast high-level web crawling & scraping framework for Python.

Sort the default RETRY_HTTP_CODES (#8415)

Kishan Mehta5c7886a3f27d
diff --git a/docs/topics/downloader-middleware.rst b/docs/topics/downloader-middleware.rst
index 195b939f3..a4abddd40 100644
--- a/docs/topics/downloader-middleware.rst
+++ b/docs/topics/downloader-middleware.rst
@@ -1186,7 +1186,7 @@ precedence over the :setting:`RETRY_TIMES` setting.
RETRY_HTTP_CODES
^^^^^^^^^^^^^^^^
-Default: ``[500, 502, 503, 504, 522, 524, 408, 429]``
+Default: ``[408, 429, 500, 502, 503, 504, 522, 524]``
Which HTTP response codes to retry. Other errors (DNS lookup issues,
connections lost, etc) are always retried.
diff --git a/scrapy/settings/default_settings.py b/scrapy/settings/default_settings.py
index 0c162114b..ede9d09c5 100644
--- a/scrapy/settings/default_settings.py
+++ b/scrapy/settings/default_settings.py
@@ -591,7 +591,7 @@ RETRY_EXCEPTIONS = [
"scrapy.core.downloader.handlers.http11.TunnelError",
]
RETRY_GIVE_UP_LOG_LEVEL = "ERROR"
-RETRY_HTTP_CODES = [500, 502, 503, 504, 522, 524, 408, 429]
+RETRY_HTTP_CODES = [408, 429, 500, 502, 503, 504, 522, 524]
RETRY_PRIORITY_ADJUST = -1
RETRY_TIMES = 2 # initial response + 2 retries = 3 requests