Scrapy, a fast high-level web crawling & scraping framework for Python.
added upstream issue for not supported python version
databenderbcf38a67194f
diff --git a/docs/topics/downloader-middleware.rst b/docs/topics/downloader-middleware.rstindex 4d7c87404..089d5683a 100644--- a/docs/topics/downloader-middleware.rst+++ b/docs/topics/downloader-middleware.rst@@ -1070,7 +1070,9 @@ In order to use this parser:* Install `Reppy <https://github.com/seomoz/reppy/>`_ by running ``pip install reppy``-.. warning:: Does not support Python 3.9++ .. warning:: `Upstream issue #122+ <https://github.com/seomoz/reppy/issues/122>`_ prevents reppy usage in+ Python 3.9+.* Set :setting:`ROBOTSTXT_PARSER` setting to``scrapy.robotstxt.ReppyRobotParser``