Scrapy, a fast high-level web crawling & scraping framework for Python.
Reword warning on no-referrer-when-downgrade policy
Paul Trembertheb07285a63d6
diff --git a/docs/topics/spider-middleware.rst b/docs/topics/spider-middleware.rstindex a4ac45b41..ada4c46c0 100644--- a/docs/topics/spider-middleware.rst+++ b/docs/topics/spider-middleware.rst@@ -349,7 +349,8 @@ This setting accepts:- or one of the standard W3C-defined string values:- `"no-referrer"`_,- - `"no-referrer-when-downgrade"`_,+ - `"no-referrer-when-downgrade"`_+ (the W3C-recommended default, used by major web browsers),- `"same-origin"`_,- `"origin"`_,- `"strict-origin"`_,@@ -358,18 +359,19 @@ This setting accepts:- or `"unsafe-url"`_(not recommended).- (It can also be the non-standard value ``"scrapy-default"`` to use- Scrapy's default referrer policy.)+It can also be the non-standard value ``"scrapy-default"`` to use+Scrapy's default referrer policy.Scrapy's default referrer policy is a variant of `"no-referrer-when-downgrade"`_,with the addition that "Referrer" is not sent if the parent request wasusing ``file://`` or ``s3://`` scheme... warning::- Scrapy's default referrer policy, just like `"no-referrer-when-downgrade"`_,- will send a non-empty "Referer" header from any ``https://`` to any ``https://`` URL,+ Scrapy's default referrer policy—just like `"no-referrer-when-downgrade"`_,+ the W3C-recommended value for browsers—will send a non-empty+ "Referer" header from any ``http(s)://`` to any ``https://`` URL,even if the domain is different.- ``same-origin`` may be a better choice if you want to remove referrer+ `"same-origin"`_ may be a better choice if you want to remove referrerinformation for cross-domain requests... note::