Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Updated README
Jan Curn committed 8 years ago
2b93eac
Update MIGRATIONS.md
Marek Trunkát committed 8 years ago
8118d3e
Update MIGRATIONS.md
Marek Trunkát committed 8 years ago
05d0a72
Updated README
Jan Curn committed 8 years ago
486bbaa
Returning prepublishOnly to package.json
Marek Trunkat committed 8 years ago
55b4f7c
Version bump
Marek Trunkat committed 8 years ago
a34e904
Updated README
Jan Curn committed 8 years ago
2932b01
Updated README
Jan Curn committed 8 years ago
e7a3a42
55710e8
7dacb4f
Fix lint issue
Ondra Urban committed 8 years ago
1f74c10
Updated README
Jan Curn committed 8 years ago
71588ea
Fix lint issues
Ondra Urban committed 8 years ago
4c488d3
Updated README, added examples/hello_world.js
Jan Curn committed 8 years ago
9d6807c
Update dataset.js
Marek Trunkát committed 8 years ago
bfb27e7
Enable null storage name
Ondra Urban committed 8 years ago
eb17bd5
Update request_queue.js
Marek Trunkát committed 8 years ago
d30d51a
Update key_value_store.js
Marek Trunkát committed 8 years ago
9f7490e
Setting log level to INFO by default
Marek Trunkat committed 8 years ago
6c96aef
Fix tests
Ondra Urban committed 8 years ago
ce1dc42
Updated README
Jan Curn committed 8 years ago
11b6dcd
Updated README
Jan Curn committed 8 years ago
5ffe1bc