Skip to content

Scrapy, a fast high-level web crawling & scraping framework for Python.

Warn that XMLFeedSpider's html iterator can mangle void-like tags (#8045)

Adrian5938dde913a0
diff --git a/docs/topics/spiders.rst b/docs/topics/spiders.rst
index 95c80d5dc..bfa70e8b5 100644
--- a/docs/topics/spiders.rst
+++ b/docs/topics/spiders.rst
@@ -624,7 +624,11 @@ XMLFeedSpider
- ``'html'`` - an iterator which uses :class:`~scrapy.Selector`.
Keep in mind this uses DOM parsing and must load all DOM in memory
- which could be a problem for big feeds
+ which could be a problem for big feeds. It also parses the feed
+ with an HTML parser, which can silently mangle tags that HTML
+ treats as void elements, such as ``<link>``, dropping their
+ content and closing tag. Use ``xml`` or ``iternodes`` instead
+ for feeds affected by this.
- ``'xml'`` - an iterator which uses :class:`~scrapy.Selector`.
Keep in mind this uses DOM parsing and must load all DOM in memory