Scrapy, a fast high-level web crawling & scraping framework for Python.
More documentation for setting spider atributes
Shinichi Takayanagibd60c3f41fd2
diff --git a/docs/topics/feed-exports.rst b/docs/topics/feed-exports.rstindex e772a461c..26c247cdd 100644--- a/docs/topics/feed-exports.rst+++ b/docs/topics/feed-exports.rst@@ -135,6 +135,9 @@ Here are some examples to illustrate:- ``s3://mybucket/scraping/feeds/%(name)s/%(time)s.json``+.. note:: :ref:`Spider arguments <spiderargs>` become spider attributes, hence+ they can also be used as storage URI parameters.+.. _topics-feed-storage-backends:diff --git a/docs/topics/spiders.rst b/docs/topics/spiders.rstindex 2056664c7..a3e9f410f 100644--- a/docs/topics/spiders.rst+++ b/docs/topics/spiders.rst@@ -294,6 +294,14 @@ The above example can also be written as follows::def start_requests(self):yield scrapy.Request(f'http://www.example.com/categories/{self.category}')+If you are :ref:`running Scrapy from a script <run-from-script>`, you can+specify spider arguments when calling+:class:`CrawlerProcess.crawl <scrapy.crawler.CrawlerProcess.crawl>` or+:class:`CrawlerRunner.crawl <scrapy.crawler.CrawlerRunner.crawl>`::++ process = CrawlerProcess()+ process.crawl(MySpider, category="electronics")+Keep in mind that spider arguments are only strings.The spider will not do any parsing on its own.If you were to set the ``start_urls`` attribute from the command line,