Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Improved comments
drobnikj committed 7 years ago
e155d12
60de0a7
Session Pool implementation proposal.
Petr Patek committed 7 years ago
e8a0b34
Build docs
Ondra Urban committed 7 years ago
791fd23
Build and deploy docs
Ondra Urban committed 7 years ago
a187832
Merge pull request #460 from Masa331/master
Ondra Urban committed 7 years ago
1b0abf3
Bump beta version
Ondra Urban committed 7 years ago
cfebf6c
Build docs
Ondra Urban committed 7 years ago
e1e1ab5
Regenerated docs
Jan Curn committed 7 years ago
14418f6
Fixed Lint errors
Jan Curn committed 7 years ago
692e3ae
Fix missing Puppeteer launch log
Lukáš Křivka committed 7 years ago
bbf20c7
Removed duplicate Puppeteer launch log
Lukáš Křivka committed 7 years ago
4178a04
Remove nonexistant option mention from docs [skip ci]
Premysl Donat committed 7 years ago
a23b417
Merge pull request #459 from cybairfly/master
Jan Čurn committed 7 years ago
a24302d
fix broken link in docs [skip ci]
Tobey committed 7 years ago
b8dd4a7
increased version to 0.16.0.
Petr Patek committed 7 years ago
62d7f4c
builded docs
Petr Patek committed 7 years ago
12af6f4
Regenerated docs
Jan Curn committed 7 years ago
7a279b8
Regenerated docs
Jan Curn committed 7 years ago
8807c93
Added "prettier" dev dependency
Jan Curn committed 7 years ago
39d8eeb
Improved Dataset.getData() docs
Jan Curn committed 7 years ago
42e20df
Add useIncognitoPages to PuppeteerPool
Ondra Urban committed 7 years ago
a93b075
Fix typo in docs [skip ci]
Ondra Urban committed 7 years ago
4a27c4e
Bump version, update Changelog
Ondra Urban committed 7 years ago
6086865
Build docs
Ondra Urban committed 7 years ago
ad7b786