Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Changed arg order, moved size into object
Jan Novotny committed 7 years ago
484248a
add getPublicUrl function to key-value-store
aarontjdev committed 7 years ago
d153036
add getPublicUrl function to key-value-store
aarontjdev committed 7 years ago
0fd0534
Added a TODO
Jan Curn committed 7 years ago
c23d13e
Fixed forEachKey incosistency
Jan Novotny committed 7 years ago
5f774e3
d198891
Added changelog note
Jan Novotny committed 7 years ago
57d3bda
Fixed forEachKey
Jan Novotny committed 7 years ago
68d9bfe
Updated docs
Jan Curn committed 7 years ago
cce9b1d
Fix tutorial
Ondra Urban committed 7 years ago
f9e47aa
Fix tests
Ondra Urban committed 7 years ago
ca28e13
Add nullification of loadedUrl to PupCrw
Ondra Urban committed 7 years ago
deceabe
Implement, fix docs throughout
Ondra Urban committed 7 years ago
72cb100
Build docs
Ondra Urban committed 7 years ago
b374b64
Update changelog, bump version
Ondra Urban committed 7 years ago
4d333d1
ccfbd8b
Fix unrelated broken test
Ondra Urban committed 7 years ago
845539e
Fix merge conflicts
Ondra Urban committed 7 years ago
096d3f7
e518759
Fix typo
Ondra Urban committed 7 years ago
d881b88
0e51450
Fix tutorial
Ondra Urban committed 7 years ago
7f61b9c
Add next part of tutorial
Ondra Urban committed 7 years ago
b415e3e
Merge branch 'master' into develop
Ondra Urban committed 7 years ago
37311e7
Merge branch 'develop' into feature/dbug_info
Marek Trunkát committed 7 years ago
64f18ff
Update getting_started.md
Jakub Balada committed 7 years ago
674afd7
Improve clarity
Ondra Urban committed 7 years ago
da8dc55