Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Upgrading PPTR to 2.0.0 and fixing clicking

Marek Trunkat368352f09a98
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 3be15cb3e..262dc3bd7 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,6 +1,6 @@
NEXT
====================
-- Bump Puppeteer to 1.19.0 and use `{ pipe: true }` again because upstream bug has been fixed.
+- Bump Puppeteer to 2.0.0 and use `{ pipe: true }` again because upstream bug has been fixed.
- Add `useIncognitoPages` option to `PuppeteerPool` to enable opening new pages in incognito
browser contexts. This is useful to keep cookies and cache unique for each page.
diff --git a/package.json b/package.json
index 45dbc6757..493e69642 100644
--- a/package.json
+++ b/package.json
@@ -105,6 +105,6 @@
"why-is-node-running": "^2.0.3"
},
"optionalDependencies": {
- "puppeteer": "1.19.0"
+ "puppeteer": "^2.0.0"
}
}
diff --git a/src/enqueue_links/click_elements.js b/src/enqueue_links/click_elements.js
index 3451c1f7e..f1dffbb47 100644
--- a/src/enqueue_links/click_elements.js
+++ b/src/enqueue_links/click_elements.js
@@ -215,9 +215,12 @@ function createInterceptRequestHandler(page, requests) {
method: req.method(),
payload: req.postData(),
}));
- req.respond(req.redirectChain().length
- ? { body: '' } // Prevents 301/302 redirect
- : { status: 204 }); // Prevents navigation by js
+
+ if (req.redirectChain().length) {
+ req.respond({ body: '' }); // Prevents 301/302 redirect
+ } else {
+ req.abort('aborted'); // Prevents navigation by js
+ }
};
}