Scrapy, a fast high-level web crawling & scraping framework for Python.
Cover Scrapy 2.1 in the release notes (#4499)
Adrián Chavesffe576c4ed19
diff --git a/docs/news.rst b/docs/news.rstindex e9b7140cd..a158246eb 100644--- a/docs/news.rst+++ b/docs/news.rst@@ -3,6 +3,153 @@Release notes=============+.. _release-2.1.0:++Scrapy 2.1.0 (2020-04-24)+-------------------------++Highlights:++* New :setting:`FEEDS` setting to export to multiple feeds+* New :attr:`Response.ip_address <scrapy.http.Response.ip_address>` attribute++Backward-incompatible changes+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~++* :exc:`AssertionError` exceptions triggered by :ref:`assert <assert>`+ statements have been replaced by new exception types, to support running+ Python in optimized mode (see :option:`-O`) without changing Scrapy’s+ behavior in any unexpected ways.++ If you catch an :exc:`AssertionError` exception from Scrapy, update your+ code to catch the corresponding new exception.++ (:issue:`4440`)+++Deprecation removals+~~~~~~~~~~~~~~~~~~~~++* The ``LOG_UNSERIALIZABLE_REQUESTS`` setting is no longer supported, use+ :setting:`SCHEDULER_DEBUG` instead (:issue:`4385`)++* The ``REDIRECT_MAX_METAREFRESH_DELAY`` setting is no longer supported, use+ :setting:`METAREFRESH_MAXDELAY` instead (:issue:`4385`)++* The :class:`~scrapy.downloadermiddlewares.chunked.ChunkedTransferMiddleware`+ middleware has been removed, including the entire+ :class:`scrapy.downloadermiddlewares.chunked` module; chunked transfers+ work out of the box (:issue:`4431`)++* The ``spiders`` property has been removed from+ :class:`~scrapy.crawler.Crawler`, use :class:`CrawlerRunner.spider_loader+ <scrapy.crawler.CrawlerRunner.spider_loader>` or instantiate+ :setting:`SPIDER_LOADER_CLASS` with your settings instead (:issue:`4398`)++* The ``MultiValueDict``, ``MultiValueDictKeyError``, and ``SiteNode``+ classes have been removed from :mod:`scrapy.utils.datatypes`+ (:issue:`4400`)+++Deprecations+~~~~~~~~~~~~++* The ``FEED_FORMAT`` and ``FEED_URI`` settings have been deprecated in+ favor of the new :setting:`FEEDS` setting (:issue:`1336`, :issue:`3858`,+ :issue:`4507`)+++New features+~~~~~~~~~~~~++* A new setting, :setting:`FEEDS`, allows configuring multiple output feeds+ with different settings each (:issue:`1336`, :issue:`3858`, :issue:`4507`)++* The :command:`crawl` and :command:`runspider` commands now support multiple+ ``-o`` parameters (:issue:`1336`, :issue:`3858`, :issue:`4507`)++* The :command:`crawl` and :command:`runspider` commands now support+ specifying an output format by appending ``:<format>`` to the output file+ (:issue:`1336`, :issue:`3858`, :issue:`4507`)++* The new :attr:`Response.ip_address <scrapy.http.Response.ip_address>`+ attribute gives access to the IP address that originated a response+ (:issue:`3903`, :issue:`3940`)++* A warning is now issued when a value in+ :attr:`~scrapy.spiders.Spider.allowed_domains` includes a port+ (:issue:`50`, :issue:`3198`, :issue:`4413`)++* Zsh completion now excludes used option aliases from the completion list+ (:issue:`4438`)+++Bug fixes+~~~~~~~~~++* :ref:`Request serialization <request-serialization>` no longer breaks for+ callbacks that are spider attributes which are assigned a function with a+ different name (:issue:`4500`)++* ``None`` values in :attr:`~scrapy.spiders.Spider.allowed_domains` no longer+ cause a :exc:`TypeError` exception (:issue:`4410`)++* Zsh completion no longer allows options after arguments (:issue:`4438`)++* zope.interface 5.0.0 and later versions are now supported+ (:issue:`4447`, :issue:`4448`)++* :meth:`Spider.make_requests_from_url+ <scrapy.spiders.Spider.make_requests_from_url>`, deprecated in Scrapy+ 1.4.0, now issues a warning when used (:issue:`4412`)+++Documentation+~~~~~~~~~~~~~++* Improved the documentation about signals that allow their handlers to+ return a :class:`~twisted.internet.defer.Deferred` (:issue:`4295`,+ :issue:`4390`)++* Our PyPI entry now includes links for our documentation, our source code+ repository and our issue tracker (:issue:`4456`)++* Covered the `curl2scrapy <https://michael-shub.github.io/curl2scrapy/>`_+ service in the documentation (:issue:`4206`, :issue:`4455`)++* Removed references to the Guppy library, which only works in Python 2+ (:issue:`4285`, :issue:`4343`)++* Extended use of InterSphinx to link to Python 3 documentation+ (:issue:`4444`, :issue:`4445`)++* Added support for Sphinx 3.0 and later (:issue:`4475`, :issue:`4480`,+ :issue:`4496`, :issue:`4503`)+++Quality assurance+~~~~~~~~~~~~~~~~~++* Removed warnings about using old, removed settings (:issue:`4404`)++* Removed a warning about importing+ :class:`~twisted.internet.testing.StringTransport` from+ ``twisted.test.proto_helpers`` in Twisted 19.7.0 or newer (:issue:`4409`)++* Removed outdated Debian package build files (:issue:`4384`)++* Removed :class:`object` usage as a base class (:issue:`4430`)++* Removed code that added support for old versions of Twisted that we no+ longer support (:issue:`4472`)++* Fixed code style issues (:issue:`4468`, :issue:`4469`, :issue:`4471`,+ :issue:`4481`)++* Removed :func:`twisted.internet.defer.returnValue` calls (:issue:`4443`,+ :issue:`4446`, :issue:`4489`)++.. _release-2.0.1:Scrapy 2.0.1 (2020-03-18)diff --git a/docs/topics/request-response.rst b/docs/topics/request-response.rstindex 5eb4915cd..024f46466 100644--- a/docs/topics/request-response.rst+++ b/docs/topics/request-response.rst@@ -619,6 +619,9 @@ Response objects:param ip_address: The IP address of the server from which the Response originated.:type ip_address: :class:`ipaddress.IPv4Address` or :class:`ipaddress.IPv6Address`+ .. versionadded:: 2.1.0+ The ``ip_address`` parameter.+.. attribute:: Response.urlA string containing the URL of the response.@@ -710,6 +713,8 @@ Response objects.. attribute:: Response.ip_address+ .. versionadded:: 2.1.0+The IP address of the server from which the Response originated.This attribute is currently only populated by the HTTP 1.1 download