Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Fixing README.md
Marek Trunkat committed 8 years ago
fcc6e2c
Updated README
Jan Curn committed 8 years ago
0f31df7
Updated docs and README
Jan Curn committed 8 years ago
3263cc8
4ab1654
Removing legacy apifier package
Marek Trunkat committed 8 years ago
3529042
Fix snapshotter empty array issue
Ondra Urban committed 8 years ago
08ebc6e
Fix unstable tests
Ondra Urban committed 8 years ago
471fc78
Increase CPU overload threshold; Bump version
Ondra Urban committed 8 years ago
b728ecd
Fix docs
Ondra Urban committed 8 years ago
80cf4b0
Remove old TODO
Ondra Urban committed 8 years ago
b9957cc
Updated docs
Jan Curn committed 8 years ago
efb0782
Updated docs
Jan Curn committed 8 years ago
6c6aaf4
Add test for autoscaledpool.abort()
Ondra Urban committed 8 years ago
d25281d
Bump version
Ondra Urban committed 8 years ago
aec1611
fba5c6f
Updated docs
Jan Curn committed 8 years ago
80e5483
Updated docs and README
Jan Curn committed 8 years ago
334cd87
Updated README
Jan Curn committed 8 years ago
9ea67ce
Update key_value_store.js
davidjohnbarton committed 8 years ago
e9bdc6a
Update events.js
davidjohnbarton committed 8 years ago
67b839f
Update dataset.js
davidjohnbarton committed 8 years ago
bfd3d54
Update constants.js
davidjohnbarton committed 8 years ago
72e8c9c
Updated docs
Jan Curn committed 8 years ago
e4263e9
Fix missing var in LiveView
Ondra Urban committed 8 years ago
ab4661d
Version bump
Marek Trunkat committed 8 years ago
42c9b78
Merge pull request #180 from apifytech/feature/mix
Marek Trunkát committed 8 years ago
2f845c9