Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Add throw error option to request.skip()

Ondra Urbane984186c02ab
diff --git a/src/request.js b/src/request.js
index 0088add30..58363d6f7 100644
--- a/src/request.js
+++ b/src/request.js
@@ -198,9 +198,15 @@ class Request {
* Marks the request as skipped which in turn prevents {@link BasicCrawler}
* (and {@PuppeteerCrawler} + {@CheerioCrawler}, since they use {@BasicCrawler} internally)
* from retrying the request after an error occurs.
+ *
+ * Optionally accepts a message that will be used to construct
+ * and throw an Error.
+ *
+ * @param {String} [message]
*/
- skip() {
+ skip(message) {
this.skipped = true;
+ if (message) throw new Error(message);
}
}