Scrapy, a fast high-level web crawling & scraping framework for Python.
Update docs for new "referrer_policy" Request.meta key
Paul Tremberth03ff19d1882e
diff --git a/docs/topics/request-response.rst b/docs/topics/request-response.rstindex 3d110b02d..67f8ec285 100644--- a/docs/topics/request-response.rst+++ b/docs/topics/request-response.rst@@ -307,6 +307,7 @@ Those are:* :reqmeta:`proxy`* ``ftp_user`` (See :setting:`FTP_USER` for more info)* ``ftp_password`` (See :setting:`FTP_PASSWORD` for more info)+* :reqmeta:`referrer_policy`.. reqmeta:: bindaddressdiff --git a/docs/topics/spider-middleware.rst b/docs/topics/spider-middleware.rstindex 0ddf027ea..a4ac45b41 100644--- a/docs/topics/spider-middleware.rst+++ b/docs/topics/spider-middleware.rst@@ -337,6 +337,8 @@ REFERER_POLICYDefault: ``'scrapy.spidermiddlewares.referer.DefaultReferrerPolicy'``+.. reqmeta:: referrer_policy+`Referrer Policy`_ to apply when populating Request "Referer" header.This setting accepts:@@ -370,6 +372,11 @@ using ``file://`` or ``s3://`` scheme.``same-origin`` may be a better choice if you want to remove referrerinformation for cross-domain requests.+.. note::+ You can also override the Referrer Policy per request,+ using the special ``"referrer_policy"`` :ref:`Request.meta <topics-request-meta>` key,+ with the same acceptable values as for the ``REFERER_POLICY`` setting.+.. _Referrer Policy: https://www.w3.org/TR/referrer-policy.. _"no-referrer": https://www.w3.org/TR/referrer-policy/#referrer-policy-no-referrer.. _"no-referrer-when-downgrade": https://www.w3.org/TR/referrer-policy/#referrer-policy-no-referrer-when-downgrade