Scrapy, a fast high-level web crawling & scraping framework for Python.
Merge branch '2.11-compression-bomb' into 2.11
Adrián Chaves809bfac4890f
diff --cc docs/news.rstindex 5a3d5e783,a12bda53f..ad931d6db--- a/docs/news.rst+++ b/docs/news.rst@@@ -8,70 -8,18 +8,80 @@@ Release noteScrapy 2.11.1 (unreleased)---------------------------**Security bug fix:**+Highlights:++- Security bug fixes.++- Support for Twisted >= 23.8.0.++- Documentation improvements.++Security bug fixes+~~~~~~~~~~~~~~~~~~+ - :setting:`DOWNLOAD_MAXSIZE` and :setting:`DOWNLOAD_WARNSIZE` now also apply+ to the decompressed response body. Please, see the `7j7m-v7m3-jqm7 security+ advisory`_ for more information.++ .. _7j7m-v7m3-jqm7 security advisory: https://github.com/scrapy/scrapy/security/advisories/GHSA-7j7m-v7m3-jqm7++ - Also in relation with the `7j7m-v7m3-jqm7 security advisory`_, the+ deprecated ``scrapy.downloadermiddlewares.decompression`` module has been+ removed.++- The ``Authorization`` header is now dropped on redirects to a different+ domain. Please, see the `cw9j-q3vf-hrrv security advisory`_ for more+ information.++ .. _cw9j-q3vf-hrrv security advisory: https://github.com/scrapy/scrapy/security/advisories/GHSA-cw9j-q3vf-hrrv++Modified requirements+~~~~~~~~~~~~~~~~~~~~~++- The Twisted dependency is no longer restricted to < 23.8.0. (:issue:`6024`,+ :issue:`6064`, :issue:`6142`)++Bug fixes+~~~~~~~~~++- The OS signal handling code was refactored to no longer use private Twisted+ functions. (:issue:`6024`, :issue:`6064`, :issue:`6112`)++Documentation+~~~~~~~~~~~~~++- Improved documentation for :class:`~scrapy.crawler.Crawler` initialization+ changes made in the 2.11.0 release. (:issue:`6057`, :issue:`6147`)++- Extended documentation for :attr:`Request.meta <scrapy.http.Request.meta>`.+ (:issue:`5565`)++- Fixed the :reqmeta:`dont_merge_cookies` documentation. (:issue:`5936`,+ :issue:`6077`)++- Added a link to Zyte's export guides to the :ref:`feed exports+ <topics-feed-exports>` documentation. (:issue:`6183`)++- Added a missing note about backward-incompatible changes in+ :class:`~scrapy.exporters.PythonItemExporter` to the 2.11.0 release notes.+ (:issue:`6060`, :issue:`6081`)++- Added a missing note about removing the deprecated+ ``scrapy.utils.boto.is_botocore()`` function to the 2.8.0 release notes.+ (:issue:`6056`, :issue:`6061`)++- Other documentation improvements. (:issue:`6128`, :issue:`6144`,+ :issue:`6163`, :issue:`6190`, :issue:`6192`)++Quality assurance+~~~~~~~~~~~~~~~~~++- Added Python 3.12 to the CI configuration, re-enabled tests that were+ disabled when the pre-release support was added. (:issue:`5985`,+ :issue:`6083`, :issue:`6098`)++- Fixed a test issue on PyPy 7.3.14. (:issue:`6204`, :issue:`6205`)+.. _release-2.11.0:@@@ -2941,19 -2889,21 +2951,28 @@@ affect subclasses(:issue:`3884`)++.. _release-1.8.4:Scrapy 1.8.4 (unreleased)--------------------------**Security bug fix:**+**Security bug fixes:**+ - :setting:`DOWNLOAD_MAXSIZE` and :setting:`DOWNLOAD_WARNSIZE` now also apply+ to the decompressed response body. Please, see the `7j7m-v7m3-jqm7 security+ advisory`_ for more information.++ - Also in relation with the `7j7m-v7m3-jqm7 security advisory`_, use of the+ ``scrapy.downloadermiddlewares.decompression`` module is discouraged and+ will trigger a warning.++- The ``Authorization`` header is now dropped on redirects to a different+ domain. Please, see the `cw9j-q3vf-hrrv security advisory`_ for more+ information.++ .. _cw9j-q3vf-hrrv security advisory: https://github.com/scrapy/scrapy/security/advisories/GHSA-cw9j-q3vf-hrrv+.. _release-1.8.3: