Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Fix merge conflicts
Ondra Urban096d3f7d8e06
diff --cc CHANGELOG.mdindex e8e343c1e,e8096f3c6..bace0fb82--- a/CHANGELOG.md+++ b/CHANGELOG.md@@@ -1,8 -1,9 +1,10 @@@-xxx+xxxxxxxxxxxxxxxxxxx===================+- Added `utils.createRequestDebugInfo()` function to create a standardized debug info from request and response.- `PseudoUrl` can now be constructed with a `RegExp`.- `Apify.utils.enqueueLinks()` now accepts `RegExp` instances in its `pseudoUrls` parameter.+ - `Apify.utils.enqueueLinks()` now accepts a `baseUrl` option that enables resolution of relative URLs+ when parsing a Cheerio object. (It's done automatically in browser when using Puppeteer).- Better error message for an invalid `launchPuppeteerFunction` passed to `PuppeteerPool`.0.11.6 / 2018-01-24