Scrapy, a fast high-level web crawling & scraping framework for Python.
Documentation improvements (#6429)
mlmsmithd8ecd28c5557
diff --git a/docs/intro/install.rst b/docs/intro/install.rstindex c90c1d2bf..e6c9a683b 100644--- a/docs/intro/install.rst+++ b/docs/intro/install.rst@@ -37,7 +37,7 @@ Note that sometimes this may require solving compilation issues for some Scrapydependencies depending on your operating system, so be sure to check the:ref:`intro-install-platform-notes`.-For more detailed and platform specifics instructions, as well as+For more detailed and platform-specific instructions, as well astroubleshooting information, read on.@@ -101,7 +101,7 @@ Windows-------Though it's possible to install Scrapy on Windows using pip, we recommend you-to install `Anaconda`_ or `Miniconda`_ and use the package from the+install `Anaconda`_ or `Miniconda`_ and use the package from the`conda-forge`_ channel, which will avoid most installation issues.Once you've installed `Anaconda`_ or `Miniconda`_, install Scrapy with::@@ -141,7 +141,7 @@ But it should support older versions of Ubuntu too, like Ubuntu 14.04,albeit with potential issues with TLS connections.**Don't** use the ``python-scrapy`` package provided by Ubuntu, they are-typically too old and slow to catch up with latest Scrapy.+typically too old and slow to catch up with the latest Scrapy release.To install Scrapy on Ubuntu (or Ubuntu-based) systems, you need to install@@ -170,7 +170,7 @@ macOSBuilding Scrapy's dependencies requires the presence of a C compiler anddevelopment headers. On macOS this is typically provided by Apple’s Xcode-development tools. To install the Xcode command line tools open a terminal+development tools. To install the Xcode command-line tools, open a terminalwindow and run::xcode-select --install@@ -200,11 +200,6 @@ solutions:brew install python- * Latest versions of python have ``pip`` bundled with them so you won't need- to install it separately. If this is not the case, upgrade python::-- brew update; brew upgrade python-* *(Optional)* :ref:`Install Scrapy inside a Python virtual environment<intro-using-virtualenv>`.diff --git a/docs/intro/overview.rst b/docs/intro/overview.rstindex ef1294470..cd17b1968 100644--- a/docs/intro/overview.rst+++ b/docs/intro/overview.rst@@ -65,7 +65,7 @@ When you ran the command ``scrapy runspider quotes_spider.py``, Scrapy looked foSpider definition inside it and ran it through its crawler engine.The crawl started by making requests to the URLs defined in the ``start_urls``-attribute (in this case, only the URL for quotes in *humor* category)+attribute (in this case, only the URL for quotes in the *humor* category)and called the default callback method ``parse``, passing the response object asan argument. In the ``parse`` callback, we loop through the quote elementsusing a CSS Selector, yield a Python dict with the extracted quote text and author,@@ -83,9 +83,9 @@ While this enables you to do very fast crawls (sending multiple concurrentrequests at the same time, in a fault-tolerant way) Scrapy also gives youcontrol over the politeness of the crawl through :ref:`a few settings<topics-settings-ref>`. You can do things like setting a download delay between-each request, limiting amount of concurrent requests per domain or per IP, and+each request, limiting the amount of concurrent requests per domain or per IP, andeven :ref:`using an auto-throttling extension <topics-autothrottle>` that tries-to figure out these automatically.+to figure these settings out automatically... note::@@ -106,10 +106,10 @@ scraping easy and efficient, such as:* Built-in support for :ref:`selecting and extracting <topics-selectors>` datafrom HTML/XML sources using extended CSS selectors and XPath expressions,- with helper methods to extract using regular expressions.+ with helper methods for extraction using regular expressions.* An :ref:`interactive shell console <topics-shell>` (IPython aware) for trying- out the CSS and XPath expressions to scrape data, very useful when writing or+ out the CSS and XPath expressions to scrape data, which is very useful when writing ordebugging your spiders.* Built-in support for :ref:`generating feed exports <topics-feed-exports>` in@@ -124,7 +124,7 @@ scraping easy and efficient, such as:well-defined API (middlewares, :ref:`extensions <topics-extensions>`, and:ref:`pipelines <topics-item-pipeline>`).-* Wide range of built-in extensions and middlewares for handling:+* A wide range of built-in extensions and middlewares for handling:- cookies and session handling- HTTP features like compression, authentication, caching