Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Adding `--disable-background-timer-throttling` and `--enable-resource-load-scheduler=false` Chrome flags to launchPuppeteer()

Jan Curn773065299e84
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 7b87826f5..0ea6f239f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,3 +1,9 @@
+xxx
+==================
+- Automatically adding `--disable-background-timer-throttling` and `--enable-resource-load-scheduler=false`
+ Chrome flags in `Apify.launchPuppeteer()`
+ to make crawling of pages in all tabs run equally fast.
+
0.9.3 / 2018-11-12
==================
- Bug fixes and improvements of internals.
diff --git a/src/puppeteer.js b/src/puppeteer.js
index 235a13627..8eb4413bf 100644
--- a/src/puppeteer.js
+++ b/src/puppeteer.js
@@ -157,6 +157,9 @@ const getPuppeteerOrThrow = () => {
* <li>
* The function adds <code>--no-sandbox</code> to <code>args</code> to enable running
* headless Chrome in a Docker container on the Apify platform.
+ * Also it adds <code>--disable-background-timer-throttling</code>
+ * and <code>--enable-resource-load-scheduler=false</code>
+ * to make crawling of pages in all tabs run equally fast.
* </li>
* </ul>
*
@@ -194,6 +197,9 @@ export const launchPuppeteer = (options = {}) => {
optsCopy.args = optsCopy.args || [];
optsCopy.args.push('--no-sandbox');
+ optsCopy.args.push('-—disable-background-timer-throttling');
+ // TODO: It's not clear that this works, keep eye on https://bugs.chromium.org/p/chromium/issues/detail?id=723233
+ optsCopy.args.push('--enable-resource-load-scheduler=false');
if (optsCopy.headless == null) {
// Forcing headless with liveView, otherwise screenshots redirect user to new browser window
optsCopy.headless = optsCopy.liveView || (process.env[ENV_VARS.HEADLESS] === '1' && process.env[ENV_VARS.XVFB] !== '1');