Scrapy, a fast high-level web crawling & scraping framework for Python.
Merge branch 'master' into spider-name-collision
Adrián Chavesec0a0eb3ab8c
diff --cc docs/faq.rstindex ee1a75cdd,75a0f4864..84c664d77--- a/docs/faq.rst+++ b/docs/faq.rst@@@ -354,18 -353,25 +354,36 @@@ method for this purpose. For example:for _ in range(item['multiply_by']):yield deepcopy(item)+ Does Scrapy support IPv6 addresses?+ -----------------------------------++ Yes, by setting :setting:`DNS_RESOLVER` to ``scrapy.resolver.CachingHostnameResolver``.+ Note that by doing so, you lose the ability to set a specific timeout for DNS requests+ (the value of the :setting:`DNS_TIMEOUT` setting is ignored).+++ .. _faq-specific-reactor:++ How to deal with ``<class 'ValueError'>: filedescriptor out of range in select()`` exceptions?+ ----------------------------------------------------------------------------------------------++ This issue `has been reported`_ to appear when running broad crawls in macOS, where the default+ Twisted reactor is :class:`twisted.internet.selectreactor.SelectReactor`. Switching to a+ different reactor is possible by using the :setting:`TWISTED_REACTOR` setting.++Running ``runspider`` I get ``error: No spider found in file: <filename>``+--------------------------------------------------------------------------++This may happen if your Scrapy project has a spider module with a name that+conflicts with the name of one of the `Python standard library modules`_, such+as ``csv.py`` or ``os.py``, or any `Python package`_ that you have installed.+See :issue:`2680`.+++ .. _has been reported: https://github.com/scrapy/scrapy/issues/2905+.. _Python standard library modules: https://docs.python.org/py-modindex.html+.. _Python package: https://pypi.org/.. _user agents: https://en.wikipedia.org/wiki/User_agent.. _LIFO: https://en.wikipedia.org/wiki/Stack_(abstract_data_type).. _DFO order: https://en.wikipedia.org/wiki/Depth-first_search