Skip to content

Scrapy, a fast high-level web crawling & scraping framework for Python.

Merge branch 'master' into spider-name-collision

Adrián Chaves57e36b5f0d76
diff --cc docs/faq.rst
index 8de680816,080d81981..ee1a75cdd
--- a/docs/faq.rst
+++ b/docs/faq.rst
@@@ -319,17 -329,31 +329,43 @@@ I'm scraping a XML document and my XPat
You may need to remove namespaces. See :ref:`removing-namespaces`.
++
+ .. _faq-split-item:
+
+ How to split an item into multiple items in an item pipeline?
+ -------------------------------------------------------------
+
+ :ref:`Item pipelines <topics-item-pipeline>` cannot yield multiple items per
+ input item. :ref:`Create a spider middleware <custom-spider-middleware>`
+ instead, and use its
+ :meth:`~scrapy.spidermiddlewares.SpiderMiddleware.process_spider_output`
+ method for this puspose. For example::
+
+ from copy import deepcopy
+
+ from scrapy.item import BaseItem
+
+
+ class MultiplyItemsMiddleware:
+
+ def process_spider_output(self, response, result, spider):
+ for item in result:
+ if isinstance(item, (BaseItem, dict)):
+ for _ in range(item['multiply_by']):
+ yield deepcopy(item)
+
+
+Running ``runspider`` I get ``error: No spider found in file: <filename>``
+--------------------------------------------------------------------------
+
+This may happen if your Scrapy project has a spider module with a name that
+conflicts with the name of one of the `Python standard library modules`_, such
+as ``csv.py`` or ``os.py``, or any `Python package`_ that you have installed.
+See :issue:`2680`.
+
++
+.. _Python standard library modules: https://docs.python.org/py-modindex.html
+.. _Python package: https://pypi.org/
-
.. _user agents: https://en.wikipedia.org/wiki/User_agent
.. _LIFO: https://en.wikipedia.org/wiki/Stack_(abstract_data_type)
.. _DFO order: https://en.wikipedia.org/wiki/Depth-first_search