Scrapy, a fast high-level web crawling & scraping framework for Python.
Warn that XMLFeedSpider's html iterator can mangle void-like tags (#8045)
Adrian5938dde913a0
diff --git a/docs/topics/spiders.rst b/docs/topics/spiders.rstindex 95c80d5dc..bfa70e8b5 100644--- a/docs/topics/spiders.rst+++ b/docs/topics/spiders.rst@@ -624,7 +624,11 @@ XMLFeedSpider- ``'html'`` - an iterator which uses :class:`~scrapy.Selector`.Keep in mind this uses DOM parsing and must load all DOM in memory- which could be a problem for big feeds+ which could be a problem for big feeds. It also parses the feed+ with an HTML parser, which can silently mangle tags that HTML+ treats as void elements, such as ``<link>``, dropping their+ content and closing tag. Use ``xml`` or ``iternodes`` instead+ for feeds affected by this.- ``'xml'`` - an iterator which uses :class:`~scrapy.Selector`.Keep in mind this uses DOM parsing and must load all DOM in memory