Scrapy, a fast high-level web crawling & scraping framework for Python.
Add sphinx-lint. (#6920)
Andrey Rakhmatullin95172659af21
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yamlindex 0d1a76247..6fbf74058 100644--- a/.pre-commit-config.yaml+++ b/.pre-commit-config.yaml@@ -15,3 +15,7 @@ repos:rev: v5.0.0hooks:- id: trailing-whitespace+- repo: https://github.com/sphinx-contrib/sphinx-lint+ rev: v1.0.0+ hooks:+ - id: sphinx-lintdiff --git a/docs/news.rst b/docs/news.rstindex 05ad611ef..7a235787e 100644--- a/docs/news.rst+++ b/docs/news.rst@@ -340,7 +340,7 @@ Deprecations(:issue:`6708`, :issue:`6714`)- ``scrapy.utils.versions.scrapy_components_versions()`` is deprecated, use- :func:`scrapy.utils.versions.get_versions()` instead.+ :func:`scrapy.utils.versions.get_versions` instead.(:issue:`6582`)- ``BaseDupeFilter.log()`` is deprecated. It does nothing and shouldn't be@@ -1786,7 +1786,7 @@ Bug fixes(:issue:`5914`, :issue:`5917`)- Fixed an error breaking user handling of send failures in- :meth:`scrapy.mail.MailSender.send()`. (:issue:`1611`, :issue:`5880`)+ :meth:`scrapy.mail.MailSender.send`. (:issue:`1611`, :issue:`5880`)Documentation~~~~~~~~~~~~~@@ -5336,7 +5336,7 @@ Docs- Added missing bullet point for the ``AUTOTHROTTLE_TARGET_CONCURRENCY``setting. (:issue:`2756`)- Update Contributing docs, document new support channels- (:issue:`2762`, issue:`3038`)+ (:issue:`2762`, :issue:`3038`)- Include references to Scrapy subreddit in the docs- Fix broken links; use ``https://`` for external links(:issue:`2978`, :issue:`2982`, :issue:`2958`)diff --git a/docs/topics/benchmarking.rst b/docs/topics/benchmarking.rstindex b704e54ed..e8ddec00c 100644--- a/docs/topics/benchmarking.rst+++ b/docs/topics/benchmarking.rst@@ -83,4 +83,4 @@ and how well it's written.Use scrapy-bench_ for more complex benchmarking.-.. _scrapy-bench: https://github.com/scrapy/scrapy-bench\ No newline at end of file+.. _scrapy-bench: https://github.com/scrapy/scrapy-benchdiff --git a/docs/topics/developer-tools.rst b/docs/topics/developer-tools.rstindex 89a4d32d8..3d3f04793 100644--- a/docs/topics/developer-tools.rst+++ b/docs/topics/developer-tools.rst@@ -280,7 +280,7 @@ In more complex websites, it could be difficult to easily reproduce therequests, as we could need to add ``headers`` or ``cookies`` to make it work.In those cases you can export the requests in `cURL <https://curl.se/>`_format, by right-clicking on each of them in the network tool and using the-:meth:`~scrapy.Request.from_curl()` method to generate an equivalent+:meth:`~scrapy.Request.from_curl` method to generate an equivalentrequest:.. code-block:: pythondiff --git a/docs/topics/dynamic-content.rst b/docs/topics/dynamic-content.rstindex 6c57a88f1..18b3ce244 100644--- a/docs/topics/dynamic-content.rst+++ b/docs/topics/dynamic-content.rst@@ -86,7 +86,7 @@ method and URL. However, you may also need to reproduce the body, headers andform parameters (see :class:`~scrapy.FormRequest`) of that request.As all major browsers allow to export the requests in curl_ format, Scrapy-incorporates the method :meth:`~scrapy.Request.from_curl()` to generate an equivalent+incorporates the method :meth:`~scrapy.Request.from_curl` to generate an equivalent:class:`~scrapy.Request` from a cURL command. To get more informationvisit :ref:`request from curl <requests-from-curl>` inside the networktool section.diff --git a/docs/topics/exporters.rst b/docs/topics/exporters.rstindex 5c078568b..2fbe3f75d 100644--- a/docs/topics/exporters.rst+++ b/docs/topics/exporters.rst@@ -116,10 +116,10 @@ Example:2. Overriding the serialize_field() method-------------------------------------------You can also override the :meth:`~BaseItemExporter.serialize_field()` method to+You can also override the :meth:`~BaseItemExporter.serialize_field` method tocustomize how your field value will be exported.-Make sure you call the base class :meth:`~BaseItemExporter.serialize_field()` method+Make sure you call the base class :meth:`~BaseItemExporter.serialize_field` methodafter your custom code.Example:diff --git a/docs/topics/leaks.rst b/docs/topics/leaks.rstindex bbe1f3dd4..e61f33aed 100644--- a/docs/topics/leaks.rst+++ b/docs/topics/leaks.rst@@ -162,7 +162,7 @@ Too many spiders?-----------------If your project has too many spiders executed in parallel,-the output of :func:`prefs()` can be difficult to read.+the output of :func:`prefs` can be difficult to read.For this reason, that function has a ``ignore`` argument which can be used toignore a particular class (and all its subclasses). Forexample, this won't show any live references to spiders:diff --git a/docs/topics/request-response.rst b/docs/topics/request-response.rstindex 6ca0973d8..8a907e377 100644--- a/docs/topics/request-response.rst+++ b/docs/topics/request-response.rst@@ -873,7 +873,7 @@ It is usual for web sites to provide pre-populated form fields through ``<inputtype="hidden">`` elements, such as session related data or authenticationtokens (for login pages). When scraping, you'll want these fields to beautomatically pre-populated and only override a couple of them, such as the-user name and password. You can use the :meth:`.FormRequest.from_response()`+user name and password. You can use the :meth:`.FormRequest.from_response`method for this job. Here's an example spider which uses it:.. code-block:: pythondiff --git a/sep/sep-008.rst b/sep/sep-008.rstindex 1c38b1c40..baf1fde23 100644--- a/sep/sep-008.rst+++ b/sep/sep-008.rst@@ -37,8 +37,8 @@ Modules and classes- ``scrapy.contrib.itemparser.ItemParser``- ``scrapy.contrib.itemparser.XPathItemParser``-- ``scrapy.contrib.itemparser.parsers.``MapConcat`` *(ex. ``TreeExpander``)*-- ``scrapy.contrib.itemparser.parsers.``TakeFirst``+- ``scrapy.contrib.itemparser.parsers.MapConcat`` *(ex. ``TreeExpander``)*+- ``scrapy.contrib.itemparser.parsers.TakeFirst``- ``scrapy.contrib.itemparser.parsers.Join``- ``scrapy.contrib.itemparser.parsers.Identity``@@ -70,7 +70,7 @@ Alternative Public API Proposal- ``ItemLoader.replace_value()``- ``ItemLoader.load_item()`` *(returns loaded item)*-- ``ItemLoader.get_stored_values()`` or ``ItemLoader.get_values()`` *(returns the ``ItemLoader values)*+- ``ItemLoader.get_stored_values()`` or ``ItemLoader.get_values()`` *(returns the ``ItemLoader`` values)*- ``ItemLoader.get_output_value()``- ``ItemLoader.get_input_processor()`` or ``ItemLoader.get_in_processor()`` *(short version)*