Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Kill missed browsers
Ondra Urban committed 7 years ago
e797bed
Add more tests
Ondra Urban committed 7 years ago
16a310d
Remove deprecated warnings in test
Ondra Urban committed 7 years ago
38bdf33
Fix tests
Ondra Urban committed 7 years ago
559027c
Add handleRequestTimeout test
Ondra Urban committed 7 years ago
fdfd437
Add handleRequestTimeout to CheerioCrawler
Ondra Urban committed 7 years ago
918e56f
Add implementation
Ondra Urban committed 7 years ago
2270f77
Update changelog, bump version
Ondra Urban committed 7 years ago
d5c8e58
Add focusing of oldest tabs in browser
Ondra Urban committed 7 years ago
47a9861
Build docs
Marek Trunkat committed 7 years ago
1b21214
Version bump
Marek Trunkat committed 7 years ago
668b3bf
594fb30
446c080
Merge branch 'develop' into feature/metamorph
Marek Trunkát committed 7 years ago
a2c7d54
Merge pull request #316 from apifytech/feature/getEnv
Marek Trunkát committed 7 years ago
41781d1
Update actor.js
Marek Trunkát committed 7 years ago
a68e3c0
Improve docs
Ondra Urban committed 7 years ago
fef2d74
Update actor.js
Marek Trunkát committed 7 years ago
89e87a7
Removing www. from apify domain in package.json
Marek Trunkat committed 7 years ago
f86a9e4
Changing www.apify.com to apify.com
Marek Trunkat committed 7 years ago
6e9f4ec
apify-client version bump
Marek Trunkat committed 7 years ago
e0e8798
PR fixes
Marek Trunkat committed 7 years ago
c74e3b0
Adding input and option overrides to actor tasks
Marek Trunkat committed 7 years ago
c80bf2f
Fixing call() failing on run stuck in READY state
Marek Trunkat committed 7 years ago
b87286d
Adding Apify.metamorph()
Marek Trunkat committed 7 years ago
7ae319d
Fixing tests
Marek Trunkat committed 7 years ago
9c4597e
Updating apify-shared
Marek Trunkat committed 7 years ago
0927476
Update changelog
Ondra Urban committed 7 years ago
3def920