Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Merge branch 'develop' into feature/block-resources
Jan Čurn779f5cb2603a
diff --cc CHANGELOG.mdindex 1cbd20969,ab77ff181..0e925f00f--- a/CHANGELOG.md+++ b/CHANGELOG.md@@@ -1,8 -1,7 +1,9 @@@xxx===================+- Improved logging of URL and port in `PuppeteerLiveViewBrowser`.- PuppeteerCrawler's default page load timeout changed from 30 to 60 seconds.+- Added `Apify.utils.puppeteer.blockResources()` function+ - More efficient implementation of `getMemoryInfo` function0.5.51 / 2018-08-09===================