Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Added a TODO
Jan Curnc23d13e3473e
diff --git a/src/request_queue.js b/src/request_queue.jsindex 84d0409b3..eff12bf37 100644--- a/src/request_queue.js+++ b/src/request_queue.js@@ -305,6 +305,7 @@ export class RequestQueue {return this.getRequest(nextId).then((request) => {+ // TODO: What happens when getRequest() fails? Shouldn't we run the code below too???// We need to handle this situation because request may not be available// immediately after adding to the queue.if (!request) {