Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Updated docs
Jan Curn80e5483a0156
diff --git a/src/request_queue.js b/src/request_queue.jsindex a32ec6c07..6e0dee2aa 100644--- a/src/request_queue.js+++ b/src/request_queue.js@@ -110,6 +110,9 @@ const getRequestId = (uniqueKey) => {};/**+ * A helper class that is used to report results from the+ * {@linkcode enqueueLinks|Apify.utils.puppeteer.enqueueLinks()} function.+ ** @typedef {Object} RequestOperationInfo* @property {Boolean} wasAlreadyPresent Indicates if request was already present in the queue.* @property {Boolean} wasAlreadyHandled Indicates if request was already marked as handled.