Scrapy, a fast high-level web crawling & scraping framework for Python.
Warn against using keep_fragments for JS-rendered fragment content (#8033)
Adriand2ec88fb4024
diff --git a/scrapy/utils/request.py b/scrapy/utils/request.pyindex b2fd9a834..b8eb633c2 100644--- a/scrapy/utils/request.py+++ b/scrapy/utils/request.py@@ -69,6 +69,10 @@ def fingerprint(so they are also ignored by default when calculating the fingerprint.If you want to include them, set the keep_fragments argument to True(for instance when handling requests with a headless browser).++ keep_fragments is not a substitute for rendering JavaScript to reach+ content loaded based on the URL fragment; see+ :ref:`topics-dynamic-content` for that instead."""processed_include_headers: tuple[bytes, ...] | None = Noneif include_headers: