Scrapy, a fast high-level web crawling & scraping framework for Python.
Merge branch 'master' into bytes_received_signal
Eugenio Lacuesta21f1eddc90fa
diff --cc docs/topics/signals.rstindex dfb87cef3,2def53848..97be46f2a--- a/docs/topics/signals.rst+++ b/docs/topics/signals.rst@@@ -141,10 -142,10 +142,10 @@@ item_erro.. signal:: item_error.. function:: item_error(item, response, spider, failure)- Sent when a :ref:`topics-item-pipeline` generates an error (ie. raises+ Sent when a :ref:`topics-item-pipeline` generates an error (i.e. raisesan exception), except :exc:`~scrapy.exceptions.DropItem` exception.- This signal supports returning deferreds from their handlers.+ This signal supports returning deferreds from its handlers.:param item: the item dropped from the :ref:`topics-item-pipeline`:type item: dict or :class:`~scrapy.item.Item` object@@@ -257,9 -233,9 +258,9 @@@ spider_erro.. signal:: spider_error.. function:: spider_error(failure, response, spider)- Sent when a spider callback generates an error (ie. raises an exception).+ Sent when a spider callback generates an error (i.e. raises an exception).- This signal does not support returning deferreds from their handlers.+ This signal does not support returning deferreds from its handlers.:param failure: the exception raised:type failure: twisted.python.failure.Failurediff --cc scrapy/core/downloader/handlers/http11.pyindex 7a1a77b23,93951d3b5..93fdb20a8--- a/scrapy/core/downloader/handlers/http11.py+++ b/scrapy/core/downloader/handlers/http11.py@@@ -474,9 -458,13 +476,15 @@@ class _ResponseReader(protocol.Protocolself._fail_on_dataloss_warned = Falseself._reached_warnsize = Falseself._bytes_received = 0+ self._certificate = None+ self._crawler = crawler+ self._source = source+ def connectionMade(self):+ if self._certificate is None:+ with suppress(AttributeError):+ self._certificate = ssl.Certificate(self.transport._producer.getPeerCertificate())+def dataReceived(self, bodyBytes):# This maybe called several times after cancel was called with buffered data.if self._finished.called:diff --cc scrapy/signals.pyindex 590421893,cd7ed7fb1..c61ae6ec3--- a/scrapy/signals.py+++ b/scrapy/signals.py@@@ -14,9 -14,9 +14,10 @@@ spider_error = object(request_scheduled = object()request_dropped = object()request_reached_downloader = object()+ request_left_downloader = object()response_received = object()response_downloaded = object()+bytes_received = object()item_scraped = object()item_dropped = object()item_error = object()