Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Improving documentation
Jan Curn6e6a2aef8197
diff --git a/src/request_list.js b/src/request_list.jsindex b04758793..364585314 100644--- a/src/request_list.js+++ b/src/request_list.js@@ -268,7 +268,12 @@ export default class RequestList {}/**- * Returns next request which is the reclaimed one if available or next upcoming request otherwise.+ * Gets the next `Request` to process. First, the function gets a request previously reclaimed+ * using `reclaimRequest()` function, if there is any.+ * Otherwise it gets a next request from the sources.+ *+ * The function gets `null` if there are no more+ * requests to process.** @returns {Promise<Request>}*/