Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

6bf2dc5
refactor!: Make AutoscaledPool internal (#3947)
Jan Buchar committed 3 months ago
83451f2
fc2930f
6be354a
3471064
refactor: move debug helpers to `@crawlee/core` (#3916)
Jindřich Bär committed 3 months ago
39f626d
b9e9d9d
4dff929
aeee91d
feat: Finalize AdaptivePlaywrightCrawler (#3907)
Jan Buchar committed 3 months ago
2b07f02
refactor: Introduce IProxyConfiguration (#3892)
Jan Buchar committed 3 months ago
219be89
fix: Fix ErrorHandler type (#3881)
Jan Buchar committed 4 months ago
5098248
83003fb
92a356f
de94603
feat: move autoscaling helpers to `@crawlee/core` (#3835)
Jindřich Bär committed 4 months ago
e614738
f745f87