Skip to content

Scrapy, a fast high-level web crawling & scraping framework for Python.

Document JOBDIR option issue #5173

Aaron Tan8e7d2ef13312
diff --git a/docs/topics/settings.rst b/docs/topics/settings.rst
index 1a1a833df..5e820b0a9 100644
--- a/docs/topics/settings.rst
+++ b/docs/topics/settings.rst
@@ -989,6 +989,17 @@ Default: ``{}``
A dict containing the pipelines enabled by default in Scrapy. You should never
modify this setting in your project, modify :setting:`ITEM_PIPELINES` instead.
+.. setting:: JOBDIR
+
+JOBDIR
+------
+
+Default: ``''``
+
+A string indicating the directory for storing the required data to keep the state of a single job to enable persistence support. This directory must not be shared by different spiders or jobs/runs of the same spider.
+
+For more info on this setting, see :ref:`topics-jobs`
+
.. setting:: LOG_ENABLED
LOG_ENABLED
diff --git a/scrapy/settings/default_settings.py b/scrapy/settings/default_settings.py
index 4ef330dd2..9137086c0 100644
--- a/scrapy/settings/default_settings.py
+++ b/scrapy/settings/default_settings.py
@@ -199,6 +199,8 @@ ITEM_PROCESSOR = 'scrapy.pipelines.ItemPipelineManager'
ITEM_PIPELINES = {}
ITEM_PIPELINES_BASE = {}
+JOBDIR = ''
+
LOG_ENABLED = True
LOG_ENCODING = 'utf-8'
LOG_FORMATTER = 'scrapy.logformatter.LogFormatter'