Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

fix: random puppeteer crawler (running in headful mode) failure (#1348)

Andrey Bykove2d923368734
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 64af307c6..28cc41e78 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -3,6 +3,7 @@
- fix: `utils.apifyClient` early instantiation (#1330)
- feat: `utils.playwright.injectJQuery()` (#1337)
- feat: add `keyValueStore` option to `Statistics` class (#1345)
+- fix: random puppeteer crawler (running in headful mode) failure (#1348)
2.3.0 / 2022/04/07
====================
diff --git a/src/browser_launchers/puppeteer_launcher.js b/src/browser_launchers/puppeteer_launcher.js
index 56decd040..347d8e2c8 100644
--- a/src/browser_launchers/puppeteer_launcher.js
+++ b/src/browser_launchers/puppeteer_launcher.js
@@ -135,6 +135,12 @@ export class PuppeteerLauncher extends BrowserLauncher {
const launchOptions = super.createLaunchOptions();
launchOptions.args = launchOptions.args || [];
+ // Adding the following flag should prevent actor run failure for puppeteer crawler running in headful mode.
+ // Related to https://github.com/puppeteer/puppeteer/issues/7050.
+ if (!launchOptions.headless) {
+ launchOptions.args.push('--disable-site-isolation-trials');
+ }
+
if (isAtHome()) launchOptions.args.push('--no-sandbox');
if (launchOptions.defaultViewport === undefined) {