Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
chore: increase timeout between scrolls to 250ms (#1185)
Vlad Frangu2bc2e6b68be8
diff --git a/src/puppeteer_utils.js b/src/puppeteer_utils.jsindex 322eaa2d9..10967cf0c 100644--- a/src/puppeteer_utils.js+++ b/src/puppeteer_utils.js@@ -475,7 +475,7 @@ export const infiniteScroll = async (page, options = {}) => {while (!finished) {await doScroll();- await page.waitForTimeout(100);+ await page.waitForTimeout(250);if (scrollDownAndUp) {await page.mouse.wheel({ deltaY: -1000 });}