Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

MAking request list more efficient (#104)
Marek Trunkát committed 8 years ago
c1f6509
Adding delete methods to stores (#106)
Marek Trunkát committed 8 years ago
f50636b
f18417b
add router and html templates
Ondrej Urban committed 8 years ago
5979906
c5473a2
Version bump
Marek Trunkat committed 8 years ago
7a07089
Fix/autoscale pool (#101)
Jan Čurn committed 8 years ago
659fef9
Feature/request list length (#100)
Jan Čurn committed 8 years ago
129da3a
Improved docs
Jan Curn committed 8 years ago
24c8f82
Improved docs
Jan Curn committed 8 years ago
4e0be80
Improved docs
Jan Curn committed 8 years ago
58b0f20
c19bcdc
Fixing typo
Marek Trunkat committed 8 years ago
84ef14f
Version bump
Marek Trunkat committed 8 years ago
3d681fa
Better help for RequestList
Jan Curn committed 8 years ago
e3400a3
Version bump
Marek Trunkat committed 8 years ago
e0fb617
Feature/ignore error (#97)
Marek Trunkát committed 8 years ago
d8187df
Fix/in progress (#96)
Jan Čurn committed 8 years ago
ddf5e99
Feature/clicking (#95)
Marek Trunkát committed 8 years ago
262a2e6
b6e2f5a
3da1fb3
Bumped package.json version
Jan Curn committed 8 years ago
4b7a7b3
e2cce68
Updating puppeteer
Marek Trunkat committed 8 years ago
97a0545
Version bump
Marek Trunkat committed 8 years ago
31ba0e4
Adding util function to click elements (#91)
Marek Trunkát committed 8 years ago
7df7fac