Scrapy, a fast high-level web crawling & scraping framework for Python.
DOC -> Add missing documentation for CloseSpider & CoreStats (#7421)
SpiliosDmk4a165508591d
diff --git a/docs/topics/extensions.rst b/docs/topics/extensions.rstindex f0a8bb5ed..735e46c29 100644--- a/docs/topics/extensions.rst+++ b/docs/topics/extensions.rst@@ -136,6 +136,19 @@ Core Stats extensionEnable the collection of core statistics, provided the stats collection isenabled (see :ref:`topics-stats`).+The following stats are collected:++* ``start_time``: start date/time of the crawl (:class:`~datetime.datetime`).+* ``finish_time``: end date/time of the crawl (:class:`~datetime.datetime`).+* ``elapsed_time_seconds``: total crawl duration in seconds (:class:`float`).+* ``finish_reason``: the closing reason string (e.g. ``"finished"``,+ ``"closespider_timeout"``).+* ``item_scraped_count``: total number of items that passed all pipelines.+* ``item_dropped_count``: total number of items dropped by a pipeline.+* ``item_dropped_reasons_count/<ExceptionName>``: per-exception drop count+ (e.g. ``item_dropped_reasons_count/DropItem``).+* ``response_received_count``: total number of HTTP responses received.+.. _topics-extensions-ref-telnetconsole:Log Count extension@@ -247,6 +260,7 @@ settings:* :setting:`CLOSESPIDER_TIMEOUT_NO_ITEM`* :setting:`CLOSESPIDER_ITEMCOUNT`* :setting:`CLOSESPIDER_PAGECOUNT`+* :setting:`CLOSESPIDER_PAGECOUNT_NO_ITEM`* :setting:`CLOSESPIDER_ERRORCOUNT`.. note::@@ -260,12 +274,11 @@ settings:CLOSESPIDER_TIMEOUT"""""""""""""""""""-Default: ``0``+Default: ``0.0``-An integer which specifies a number of seconds. If the spider remains open for-more than that number of seconds, it will be automatically closed with the-reason ``closespider_timeout``. If zero (or non set), spiders won't be closed by-timeout.+If the spider remains open for more than this number of seconds, it will be+automatically closed with the reason ``closespider_timeout``. If zero (or non+set), spiders won't be closed by timeout... setting:: CLOSESPIDER_TIMEOUT_NO_ITEM