Skip to content

Scrapy, a fast high-level web crawling & scraping framework for Python.

Merge branch '2.11-compression-bomb' into 2.11

Adrián Chaves809bfac4890f
diff --cc docs/news.rst
index 5a3d5e783,a12bda53f..ad931d6db
--- a/docs/news.rst
+++ b/docs/news.rst
@@@ -8,70 -8,18 +8,80 @@@ Release note
Scrapy 2.11.1 (unreleased)
--------------------------
-**Security bug fix:**
+Highlights:
+
+- Security bug fixes.
+
+- Support for Twisted >= 23.8.0.
+
+- Documentation improvements.
+
+Security bug fixes
+~~~~~~~~~~~~~~~~~~
+ - :setting:`DOWNLOAD_MAXSIZE` and :setting:`DOWNLOAD_WARNSIZE` now also apply
+ to the decompressed response body. Please, see the `7j7m-v7m3-jqm7 security
+ advisory`_ for more information.
+
+ .. _7j7m-v7m3-jqm7 security advisory: https://github.com/scrapy/scrapy/security/advisories/GHSA-7j7m-v7m3-jqm7
+
+ - Also in relation with the `7j7m-v7m3-jqm7 security advisory`_, the
+ deprecated ``scrapy.downloadermiddlewares.decompression`` module has been
+ removed.
+
+- The ``Authorization`` header is now dropped on redirects to a different
+ domain. Please, see the `cw9j-q3vf-hrrv security advisory`_ for more
+ information.
+
+ .. _cw9j-q3vf-hrrv security advisory: https://github.com/scrapy/scrapy/security/advisories/GHSA-cw9j-q3vf-hrrv
+
+Modified requirements
+~~~~~~~~~~~~~~~~~~~~~
+
+- The Twisted dependency is no longer restricted to < 23.8.0. (:issue:`6024`,
+ :issue:`6064`, :issue:`6142`)
+
+Bug fixes
+~~~~~~~~~
+
+- The OS signal handling code was refactored to no longer use private Twisted
+ functions. (:issue:`6024`, :issue:`6064`, :issue:`6112`)
+
+Documentation
+~~~~~~~~~~~~~
+
+- Improved documentation for :class:`~scrapy.crawler.Crawler` initialization
+ changes made in the 2.11.0 release. (:issue:`6057`, :issue:`6147`)
+
+- Extended documentation for :attr:`Request.meta <scrapy.http.Request.meta>`.
+ (:issue:`5565`)
+
+- Fixed the :reqmeta:`dont_merge_cookies` documentation. (:issue:`5936`,
+ :issue:`6077`)
+
+- Added a link to Zyte's export guides to the :ref:`feed exports
+ <topics-feed-exports>` documentation. (:issue:`6183`)
+
+- Added a missing note about backward-incompatible changes in
+ :class:`~scrapy.exporters.PythonItemExporter` to the 2.11.0 release notes.
+ (:issue:`6060`, :issue:`6081`)
+
+- Added a missing note about removing the deprecated
+ ``scrapy.utils.boto.is_botocore()`` function to the 2.8.0 release notes.
+ (:issue:`6056`, :issue:`6061`)
+
+- Other documentation improvements. (:issue:`6128`, :issue:`6144`,
+ :issue:`6163`, :issue:`6190`, :issue:`6192`)
+
+Quality assurance
+~~~~~~~~~~~~~~~~~
+
+- Added Python 3.12 to the CI configuration, re-enabled tests that were
+ disabled when the pre-release support was added. (:issue:`5985`,
+ :issue:`6083`, :issue:`6098`)
+
+- Fixed a test issue on PyPy 7.3.14. (:issue:`6204`, :issue:`6205`)
+
.. _release-2.11.0:
@@@ -2941,19 -2889,21 +2951,28 @@@ affect subclasses
(:issue:`3884`)
++
.. _release-1.8.4:
Scrapy 1.8.4 (unreleased)
-------------------------
-**Security bug fix:**
+**Security bug fixes:**
+ - :setting:`DOWNLOAD_MAXSIZE` and :setting:`DOWNLOAD_WARNSIZE` now also apply
+ to the decompressed response body. Please, see the `7j7m-v7m3-jqm7 security
+ advisory`_ for more information.
+
+ - Also in relation with the `7j7m-v7m3-jqm7 security advisory`_, use of the
+ ``scrapy.downloadermiddlewares.decompression`` module is discouraged and
+ will trigger a warning.
+
+- The ``Authorization`` header is now dropped on redirects to a different
+ domain. Please, see the `cw9j-q3vf-hrrv security advisory`_ for more
+ information.
+
+ .. _cw9j-q3vf-hrrv security advisory: https://github.com/scrapy/scrapy/security/advisories/GHSA-cw9j-q3vf-hrrv
+
.. _release-1.8.3: