Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Merge branch 'master' of github.com:Apifier/apify-runtime-js into develop
Marek Trunkatf4e2115df260
diff --cc src/request_queue.jsindex 808f434e5,801b37cd7..c0956617b--- a/src/request_queue.js+++ b/src/request_queue.js@@@ -204,11 -209,7 +209,11 @@@ export class RequestQueue/*** Adds a request to the queue.*+ * If the request with the same `Request.uniqueKey` property is already present in the queue,+ * it will not be updated. You can find out this happened from the resulting+ * {@linkcode RequestOperationInfo} object.+ *- * @param {Request} request Request object+ * @param {Request|Object} request Request object, or object to construct a Request* @param {Object} [opts]* @param {Boolean} [opts.forefront] If `true`, the request will be added to the foremost position in the queue.* @return {RequestOperationInfo}