Scrapy, a fast high-level web crawling & scraping framework for Python.
Update docs.
djunzu6988e9cd4ba4
diff --git a/docs/topics/media-pipeline.rst b/docs/topics/media-pipeline.rstindex a34f4c053..3da243d29 100644--- a/docs/topics/media-pipeline.rst+++ b/docs/topics/media-pipeline.rst@@ -77,30 +77,6 @@ PIL... _Python Imaging Library: http://www.pythonware.com/products/pil/-Usage example-=============--In order to use a media pipeline first, :ref:`enable it-<topics-media-pipeline-enabling>`.--Then, if a spider returns a dict with the URLs key ('file_urls' or-'image_urls', for the Files or Images Pipeline respectively), the pipeline will-put the results under respective key ('files' or images').--If you prefer to use :class:`~.Item`, then define a custom item with the-necessary fields, like in this example for Images Pipeline::-- import scrapy-- class MyItem(scrapy.Item):-- # ... other item fields ...- image_urls = scrapy.Field()- images = scrapy.Field()--If you need something more complex and want to override the custom pipeline-behaviour, see :ref:`topics-media-pipeline-override`.-.. _topics-media-pipeline-enabling:Enabling your Media Pipeline@@ -171,6 +147,51 @@ Where:used). For more info see :ref:`topics-images-thumbnails`.+Usage example+=============++.. setting:: FILES_URLS_FIELD+.. setting:: FILES_RESULT_FIELD+.. setting:: IMAGES_URLS_FIELD+.. setting:: IMAGES_RESULT_FIELD++In order to use a media pipeline first, :ref:`enable it+<topics-media-pipeline-enabling>`.++Then, if a spider returns a dict with the URLs key (``file_urls`` or+``image_urls``, for the Files or Images Pipeline respectively), the pipeline will+put the results under respective key (``files`` or ``images``).++If you prefer to use :class:`~.Item`, then define a custom item with the+necessary fields, like in this example for Images Pipeline::++ import scrapy++ class MyItem(scrapy.Item):++ # ... other item fields ...+ image_urls = scrapy.Field()+ images = scrapy.Field()++If you want to use another field name for the URLs key or for the results key,+it is also possible to override it.++For the Files Pipeline, set :setting:`FILES_URLS_FIELD` and/or+:setting:`FILES_RESULT_FIELD` settings::++ FILES_URLS_FIELD = 'field_name_for_your_files_urls'+ FILES_RESULT_FIELD = 'field_name_for_your_processed_files'++For the Images Pipeline, set :setting:`IMAGES_URLS_FIELD` and/or+:setting:`IMAGES_RESULT_FIELD` settings::++ IMAGES_URLS_FIELD = 'field_name_for_your_images_urls'+ IMAGES_RESULT_FIELD = 'field_name_for_your_processed_images'++If you need something more complex and want to override the custom pipeline+behaviour, see :ref:`topics-media-pipeline-override`.++Additional features===================@@ -185,12 +206,14 @@ adjust this retention delay use the :setting:`FILES_EXPIRES` setting (or:setting:`IMAGES_EXPIRES`, in case of Images Pipeline), whichspecifies the delay in number of days::- # 90 days of delay for files expiration- FILES_EXPIRES = 90+ # 120 days of delay for files expiration+ FILES_EXPIRES = 120# 30 days of delay for images expirationIMAGES_EXPIRES = 30+The default value for both settings is 90 days.+.. _topics-images-thumbnails:Thumbnail generation for images@@ -249,7 +272,13 @@ For example::IMAGES_MIN_HEIGHT = 110IMAGES_MIN_WIDTH = 110-Note: these size constraints don't affect thumbnail generation at all.+.. note::+ The size constraints don't affect thumbnail generation at all.++It is possible to set just one size constraint or both. When setting both of+them, only images that satisfy both minimum sizes will be saved. For the+above example, images of sizes (105 x 105) or (105 x 200) or (200 x 105) will+all be dropped because at least one dimension is shorter than the constraint.By default, there are no size constraints, so all images are processed.