Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Update tests
Ondra Urban committed 7 years ago
0c8a226
Merge branch 'master' into feature/save-snapshot
Ondra Urban committed 7 years ago
f7d6eef
923dbf7
Fix test broken by updating error message
Ondra Urban committed 7 years ago
72f584c
Merge branch 'master' into fix/retryHistogram
Ondra Urban committed 7 years ago
9284d2c
Update dataset.js
Ondra Urban committed 7 years ago
cdcfcff
fixed stub.
Petr Patek committed 7 years ago
6b98357
added async
Petr Patek committed 7 years ago
cb0691d
df1b7cd
Try Ubuntu Bionic on Travis
Ondra Urban committed 7 years ago
d476627
Merge pull request #497 from apifytech/feature/jest
Ondra Urban committed 7 years ago
dc7be4d
Attempt to fix flaky tests
Ondra Urban committed 7 years ago
d8369d2
Move user-agent check to Test stage on Travis
Ondra Urban committed 7 years ago
6b0c748
Remove time reliant expects
Ondra Urban committed 7 years ago
245d275
Merge branch 'master' into feature/jest
Ondra Urban committed 7 years ago
7a23ddc
Fix interfering Apify.events tests
Ondra Urban committed 7 years ago
774703c
Try limiting maxWorkers to fix CI issues
Ondra Urban committed 7 years ago
ccd9ab9
Merge branch 'master' into feature/jest
Ondra Urban committed 7 years ago
cc9fa40
f802368
f2b2659
Merge branch 'master' into fix/cannot-reclaim
Ondra Urban committed 7 years ago
6201a32
Fix docs [skip ci]
Ondra Urban committed 7 years ago
89aff8d
Fix requests not correctly marked 'in progress'
Ondra Urban committed 7 years ago
a9f36ed
Fixed docs
Jan Curn committed 7 years ago
46ac6a1
Merge remote-tracking branch 'origin/master'
Jan Curn committed 7 years ago
82d4e9c
Updated docs
Jan Curn committed 7 years ago
a8c6610
Changelog
drobnikj committed 7 years ago
bf69507