Scrapy, a fast high-level web crawling & scraping framework for Python.
Fix some async docs examples. (#8404)
Andrey Rakhmatullin41a2ded027c6
diff --git a/docs/topics/coroutines.rst b/docs/topics/coroutines.rstindex 4c488d66e..df07c7dae 100644--- a/docs/topics/coroutines.rst+++ b/docs/topics/coroutines.rst@@ -208,8 +208,10 @@ means you can use many useful Python libraries that provide such code:class MySpiderDeferred(Spider):# ...async def parse(self, response):- additional_response = await treq.get("https://additional.url")- additional_data = await treq.content(additional_response)+ additional_response = await deferred_to_future(+ treq.get("https://additional.url")+ )+ additional_data = await deferred_to_future(treq.content(additional_response))# ... use response and additional_data to yield items and requests@@ -249,7 +251,7 @@ it in a separate thread and returns an awaitable with the result:class MySpider(Spider):async def parse(self, response):- result = await blocking_call(response.url)+ result = await deferred_to_future(blocking_call(response.url))# ... use result to yield items and requests.. autofunction:: scrapy.utils.decorators.inthreaddiff --git a/docs/topics/signals.rst b/docs/topics/signals.rstindex cc1b12d9f..41e6fc193 100644--- a/docs/topics/signals.rst+++ b/docs/topics/signals.rst@@ -69,10 +69,8 @@ Let's take an example using :ref:`coroutines <topics-coroutines>`:.. skip: next.. code-block:: python- import json-+ import aiohttpimport scrapy- import treqclass SignalSpider(scrapy.Spider):@@ -87,13 +85,8 @@ Let's take an example using :ref:`coroutines <topics-coroutines>`:async def item_scraped(self, item):# Send the scraped item to the server- response = await treq.post(- "http://example.com/post",- json.dumps(item).encode("ascii"),- headers={b"Content-Type": [b"application/json"]},- )-- return response+ async with aiohttp.ClientSession() as session:+ await session.post("http://example.com/post", json=item)def parse(self, response):for quote in response.css("div.quote"):