Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
examples - added missing awaits (#944)
petrpatek863856243279
diff --git a/docs/guides/migration_to_v1.md b/docs/guides/migration_to_v1.mdindex 45f8de9a9..caba6ffc1 100644--- a/docs/guides/migration_to_v1.md+++ b/docs/guides/migration_to_v1.md@@ -8,21 +8,21 @@ title: Migration to version 1- [Installation](#installation)- [Running on Apify Platform](#running-on-apify-platform)- [Handler arguments are now Crawling Context](#handler-arguments-are-now-crawling-context)- * [`Map` of crawling contexts and their IDs](#map-of-crawling-contexts-and-their-ids)- * [`autoscaledPool` was moved under `crawlingContext.crawler`](#autoscaledpool-was-moved-under-crawlingcontextcrawler)+ - [`Map` of crawling contexts and their IDs](#map-of-crawling-contexts-and-their-ids)+ - [`autoscaledPool` was moved under `crawlingContext.crawler`](#autoscaledpool-was-moved-under-crawlingcontextcrawler)- [Replacement of `PuppeteerPool` with `BrowserPool`](#replacement-of-puppeteerpool-with-browserpool)- * [Access to running `BrowserPool`](#access-to-running-browserpool)- * [Pages now have IDs](#pages-now-have-ids)- * [Configuration and lifecycle hooks](#configuration-and-lifecycle-hooks)- * [Introduction of `BrowserController`](#introduction-of-browsercontroller)- * [`BrowserPool` methods vs `PuppeteerPool`](#browserpool-methods-vs-puppeteerpool)+ - [Access to running `BrowserPool`](#access-to-running-browserpool)+ - [Pages now have IDs](#pages-now-have-ids)+ - [Configuration and lifecycle hooks](#configuration-and-lifecycle-hooks)+ - [Introduction of `BrowserController`](#introduction-of-browsercontroller)+ - [`BrowserPool` methods vs `PuppeteerPool`](#browserpool-methods-vs-puppeteerpool)- [Updated `PuppeteerCrawlerOptions`](#updated-puppeteercrawleroptions)- * [Removal of `gotoFunction`](#removal-of-gotofunction)- * [`launchPuppeteerOptions` => `launchContext`](#launchpuppeteeroptions--launchcontext)- * [Removal of `launchPuppeteerFunction`](#removal-of-launchpuppeteerfunction)+ - [Removal of `gotoFunction`](#removal-of-gotofunction)+ - [`launchPuppeteerOptions` => `launchContext`](#launchpuppeteeroptions--launchcontext)+ - [Removal of `launchPuppeteerFunction`](#removal-of-launchpuppeteerfunction)- [Launch functions](#launch-functions)- * [Updated arguments](#updated-arguments)- * [Custom modules](#custom-modules)+ - [Updated arguments](#updated-arguments)+ - [Custom modules](#custom-modules)## SummaryAfter 3.5 years of rapid development and a lot of breaking changes and deprecations,@@ -228,17 +228,17 @@ you should use a `browserController` to do it. You can find it in the handle pag```jsconst handlePageFunction = async ({ page, browserController }) => {// Wrong usage. Could backfire because it bypasses BrowserPool.- page.browser().close();+ await page.browser().close();// Correct usage. Allows graceful shutdown.- browserController.close();+ await browserController.close();const cookies = [/* some cookie objects */];// Wrong usage. Will only work in Puppeteer and not Playwright.- page.setCookies(...cookies);+ await page.setCookies(...cookies);// Correct usage. Will work in both.- browserController.setCookies(page, cookies);+ await browserController.setCookies(page, cookies);}```@@ -260,15 +260,15 @@ Some functions were removed (in line with earlier deprecations), and some were c```js// OLD-puppeteerPool.recyclePage(page);+await puppeteerPool.recyclePage(page);// NEW-page.close();+await page.close();``````js// OLD-puppeteerPool.retire(page.browser());+await puppeteerPool.retire(page.browser());// NEWbrowserPool.retireBrowserByPage(page);@@ -276,7 +276,7 @@ browserPool.retireBrowserByPage(page);```js// OLD-puppeteerPool.serveLiveViewSnapshot();+await puppeteerPool.serveLiveViewSnapshot();// NEW// There's no LiveView in BrowserPool@@ -298,7 +298,7 @@ const gotoFunction = async ({ request, page }) => {await makePageStealthy(page);// Have to remember how to do this:- const response = gotoExtended(page, request, {/* have to remember the defaults */});+ const response = await gotoExtended(page, request, {/* have to remember the defaults */});// post-processingawait page.evaluate(() => {diff --git a/website/sidebars.json b/website/sidebars.jsonindex 4348c176f..763f5737e 100644--- a/website/sidebars.json+++ b/website/sidebars.json@@ -182,4 +182,4 @@}]}-}+}\ No newline at end of file