Skip to content

Scrapy, a fast high-level web crawling & scraping framework for Python.

documenting sitemap entries as suggested by @kmike

Victor Torres10f46bca54b2
diff --git a/docs/topics/spiders.rst b/docs/topics/spiders.rst
index 918f1cc36..9d4ed6ca6 100644
--- a/docs/topics/spiders.rst
+++ b/docs/topics/spiders.rst
@@ -703,6 +703,16 @@ SitemapSpider
This would retrieve only ``entries`` modified on 2005 and the following
years.
+ Entries are dict objects extracted from the sitemap document.
+ Usually, the key is the tag name and the value is the text inside it.
+
+ It's important to notice that:
+
+ - as the loc attribute is required, entries without this tag are discarded
+ - alternate links are stored in a list with the key ``alternate``
+ (see ``sitemap_alternate_links``)
+ - namespaces are removed, so lxml tags named as ``{foo}bar`` become only ``bar``
+
If you omit this method, all entries found in sitemaps will be
processed, observing other attributes and their settings.