Scrapy, a fast high-level web crawling & scraping framework for Python.
Merge branch 'master' into spider-name-collision
Adrián Chaves57e36b5f0d76
diff --cc docs/faq.rstindex 8de680816,080d81981..ee1a75cdd--- a/docs/faq.rst+++ b/docs/faq.rst@@@ -319,17 -329,31 +329,43 @@@ I'm scraping a XML document and my XPatYou may need to remove namespaces. See :ref:`removing-namespaces`.+++ .. _faq-split-item:++ How to split an item into multiple items in an item pipeline?+ -------------------------------------------------------------++ :ref:`Item pipelines <topics-item-pipeline>` cannot yield multiple items per+ input item. :ref:`Create a spider middleware <custom-spider-middleware>`+ instead, and use its+ :meth:`~scrapy.spidermiddlewares.SpiderMiddleware.process_spider_output`+ method for this puspose. For example::++ from copy import deepcopy++ from scrapy.item import BaseItem+++ class MultiplyItemsMiddleware:++ def process_spider_output(self, response, result, spider):+ for item in result:+ if isinstance(item, (BaseItem, dict)):+ for _ in range(item['multiply_by']):+ yield deepcopy(item)+++Running ``runspider`` I get ``error: No spider found in file: <filename>``+--------------------------------------------------------------------------++This may happen if your Scrapy project has a spider module with a name that+conflicts with the name of one of the `Python standard library modules`_, such+as ``csv.py`` or ``os.py``, or any `Python package`_ that you have installed.+See :issue:`2680`.++++.. _Python standard library modules: https://docs.python.org/py-modindex.html+.. _Python package: https://pypi.org/-.. _user agents: https://en.wikipedia.org/wiki/User_agent.. _LIFO: https://en.wikipedia.org/wiki/Stack_(abstract_data_type).. _DFO order: https://en.wikipedia.org/wiki/Depth-first_search