Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Version bump
Marek Trunkat committed 8 years ago
2100f72
Feature/request queue fixes (#51)
Marek Trunkát committed 8 years ago
92c6565
Update example_puppeteer_crawler.js
Marek Trunkát committed 8 years ago
344a4d0
Update example_basic_crawler.js
Marek Trunkát committed 8 years ago
82cccd4
Updated README.md
Jan Curn committed 8 years ago
90a8fcf
Updated README.md
Jan Curn committed 8 years ago
f053a7b
0d96150
Updated text in package.json
Jan Curn committed 8 years ago
1a9800b
Updated text in package.json
Jan Curn committed 8 years ago
66d55ec
Version bump
Marek Trunkat committed 8 years ago
9337d0a
Feature/request queue (#48)
Marek Trunkát committed 8 years ago
7b41dea
Bumped package.json version, updated CHANGELOG
Jan Curn committed 8 years ago
90ded14
Updated README
Jan Curn committed 8 years ago
b95e2cd
Updated README and package.json description
Jan Curn committed 8 years ago
3deb5e5
Fix/request list (#47)
Jan Čurn committed 8 years ago
e0504bc
Merging develop
Marek Trunkat committed 8 years ago
290cf60
Fixing Puppeteer disconnected false positive errors
Marek Trunkat committed 8 years ago
ce0fca6
Version bump
Marek Trunkat committed 8 years ago
3bb348a
Updating changelog
Marek Trunkat committed 8 years ago
84435a4
96711a2
Feature/dev shm (#43)
Jan Čurn committed 8 years ago
3929615
Version bump
Marek Trunkat committed 8 years ago
ff7fb35
package.json update apify-client version (#40)
Jakub Drobník committed 8 years ago
2cc1a66
Fixing docs
Marek Trunkat committed 8 years ago
655d738
d84679a